Lucas Caccia

Post-Doc Researcher @ Microsoft Research


Microsoft Research

Mile-Ex, Montréal

I completed in November my Ph.D. at McGill and the Quebec Artificial Intelligence Institute (Mila), where I was advised by Joelle Pineau. Over the course of my graduate studies, I had the opportunity to intern at Meta AI where I worked with Ludovic Denoyer and Marc’Aurelio Rantazo on Continual Learning, and at Microsoft Research under the guidance of Nicolas Le Roux and Alessandro Sordoni, working on modular adaptation in LLMs.

My research centers on building agents that can accumulate knowledge and over time, enabling them to quickly generalize to new tasks in a data and compute efficient way. My work as thus been primarily focused on areas including continual,meta,transfer,multitask learning. More broadly, I believe that a viable solution to efficient adaptation is modularity. This involves creating neural networks that use specialized, independent modules to process information. To this end, I have been focusing lately on designing better algorithms to route and combine modules at a more finegrained level, and on the role of modular methods in multitask learning.

I am currently doing a post-doc at Microsoft Research Montréal, where I am leveraging these principles to adapt LLMs for massively multitask transfer. On one side, as part of project Maia, we are using these modular approaches for generative modeling of individual behavior at scale, efficiently performing stylometry over a pool of several thousand players. Another project I am involved in is the creation of an Adapter Universe, whereby using modularity to learn reusable and composable adapters over pretrained LLMs, we are aiming to build a large repertoire of such adapters that users can leverage for efficient and performant transfer to their downstream tasks of interest.


Nov 9, 2023 I successfully defended my PhD 😁. Thanks to my wonderful committee (Siva Reddy, Adrian Popescu, Steve Liu and Doina Precup) for the great discussions during my defense, and of course, big thank you to Joelle for supporting me throughout my PhD.
Sep 15, 2023 Starting my post-doc at MSR Montréal! I will be continuing my work on efficient adaptation
Sep 1, 2023 Our paper on multi-head routing strategies for MoEs has been accepted at NeurIPS 2023
Mar 7, 2022 Starting a year-long (1 day / week) internship at MSR Montreal with Alessandro Sordoni and Nicolas Le Roux, where I will be working on modular adaptation in LLMs
Feb 1, 2022 Our paper which investigates the root causes of catastrophic forgetting in class-incremental learning settings has been accepted at ICLR 2022

latest posts

Jul 1, 2020 EWC derivation

selected publications