Sound Scientific Discovery: Initial Building Blocks
Laurent Charlin - HEC Montreal
Nov. 20, 2026, 3:30 p.m. - Nov. 20, 2026, 4:30 p.m.
ENGMC 11
Hosted by: Oana Balmau
LLMs are increasingly used across the scientific workflow, from literature review to peer review, yet they routinely produce output that sounds right rather than output that is grounded in evidence. This talk presents methods we developed to close that gap. (1) LitLLM reframes literature review as retrieval-grounded generation: an LLM-driven two-step search over an external corpus, followed by plan-then-write drafting, substantially reducing hallucinated citations. (2) ReviewerToo studies what structured “AI participation” in peer review looks like in practice. It is a modular framework in which specialized reviewer personas evaluate papers against explicit criteria; on 1,963 ICLR 2025 submissions, it approaches the accept/reject accuracy of the average human reviewer, while remaining weaker on novelty judgment and calibration. I will discuss what these results tell us about the complementary strengths of human and AI reviewers, along with our deployments: a pilot at TMLR and the NeurIPS 2026 review experiments. (3) Time permitting, I will present AInstein, which asks whether LLM-generated research ideas actually hold up: we distill problem statements from 1,200+ ICLR papers and task solver agents with proposing and refining solutions through iterative critique, revealing how fragile and framing-sensitive LLM problem-solving remains. I will conclude by sketching the framework these results point toward: verification as reasoning, where papers are decomposed into claims checked against the literature, code, and data, as a path to sound AI-assisted discovery.
Laurent Charlin is an Associate Professor in the Department of Decision Sciences at HEC Montréal, a Canada CIFAR AI Chair, and a co-founding member of Mila – Quebec Artificial Intelligence Institute, where he served as Interim Scientific Director in 2025. His research develops machine learning methods for decision-making, with contributions to recommender and dialogue systems, continual learning, and AI for scientific discovery, including large language models for literature review and peer review. With Richard Zemel, he created the Toronto Paper Matching System (TPMS), used by over 200 scientific conferences. He is Editor-in-Chief of Transactions on Machine Learning Research and Experiment Co-Chair of NeurIPS 2026. He holds a Ph.D. in Computer Science from the University of Toronto and held postdoctoral positions at Princeton, Columbia, and McGill. He regularly speaks about AI to public, government, and judicial audiences.
