Past Event: CosmicAI Seminar

Exploring the LLM universe for astronomy research

Dr. Jessy Li, Associate Professor, Natural Language Processing and Computational Linguistics, UT Austin, & Sebastian Joseph, Ph.D. Student, Computer Science, UT Austin

Abstract

From ChatGPT to Deep Research, the pace of foundation models of AI has generated much excitement as well as reckoning: what are LLMs capable of? What are their limitations? How far can we push for AI models for research? One of the core missions of the Explorable Universe group within CosmicAI is to explore these questions for code LLMs.

In this talk, we will first present a primer that explains how LLMs are trained, especially within the context of our own code LLM training experience, thereby scoping the capabilities of these models. Next, we discuss our recent efforts to build a new coding benchmark that tests the limitations of LLMs, focusing on code execution and visualizations.

Biography

Jessy Li is an associate professor of Natural Language Processing and Computational Linguistics in the Department of Linguistics at UT Austin. She is the AI lead of the Explorable Universe group in CosmicAI.

Sebastian Joseph is a first-year Ph.D. student in Computer Science at UT Austin, advised by Prof. Jessy Li. His research focuses on making large language models more factual, understandable, and adaptable for expert domains.