Simons Foundation Fellow Explores Hidden Math Powering AI
Heidi Opdyke
- Associate Dean of Marketing and Communications, MCS
- Email opdyke@andrew.cmu.edu
- Phone 412-268-9982
When a chatbot answers a question or an AI tool generates a response, an intricate system works behind the scenes to determine how pieces of information relate to one another. These systems, known as transformers, are the foundation of today's large language models and many of the most significant advances in artificial intelligence. But mathematicians still do not fully understand why they work so well.
Kyunghoo Mun, a doctoral student in Carnegie Mellon University's Department of Mathematical Sciences, is working to change that.
Mun recently earned a Simons Dissertation Fellowship in Mathematics for research that aims to uncover the mathematical principles underlying transformers. Working with his advisor, Matthew Rosenzweig, the Gregg Zeitlin Assistant Professor of Mathematical Sciences, Mun studies what happens when randomness, or "noise," is introduced into these systems.
"My project focuses on the mathematical analysis of transformers with noise," Mun said. "The transformer is an architecture widely used in deep learning and is connected to the success of large language models. However, our mathematical understanding of why it works so well remains limited."
Mun’s research has applications beyond AI.
“Machine learning is the most attention-grabbing example,” Rosenzweig said. “It’s more about the mathematical problems that machine learning can suggest.”
Mun approaches the problem using interacting particle systems, a branch of mathematics that examines how large collections of interacting entities behave. This approach has long been used to study complex systems in areas such as physics and probability, including phenomena ranging from water freezing into ice or opinions spreading through social networks.
Transformers are another kind of interacting system.
A key feature of transformers is attention, a mechanism that allows different pieces of information to influence one another. When generating a summary, for example, individual words and phrases "pay attention" to the parts of a sentence that provide the most relevant context.
Mun studies what happens when noise disrupts those interactions.
In the real world, information is messy and can be incomplete or distorted. Mun is determining how much randomness, or noise, these systems can tolerate while functioning in a mathematically predictable way.
Think of it like trying to follow a conversation in a crowded room. Even when some words are lost or distorted, people can often still understand the message. But as the interference grows, there comes a point when the conversation becomes difficult to follow. Mun is interested in the mathematical equivalent of that turning point.
Complex systems can exhibit critical behavior, points at which small changes can lead to dramatically different outcomes. A major focus of Mun’s work is identifying and understanding these transitions.
"My broader goal is to understand increasingly complex interaction structures and the different kinds of critical phenomena that emerge from them," he said.
The noisy transformer model also provides a starting point for studying systems with more complex forms of interaction. Using Fourier analysis, mathematicians can break an interaction into several distinct patterns, known as modes. Mun investigates how these modes reinforce or compete with one another and how their interplay can lead to different kinds of collective behavior and critical transitions.
"Mathematically, the noisy transformer model offers many useful ideas and intuitions," Mun said. "It serves as a starting point for understanding multimodal interactions."
Rosenzweig said Mun had the technical skills and instincts needed to pursue difficult questions.
“Kyunghoo has really come into his own as a researcher,” Rosenzweig said. “Technically he is very strong but more importantly he has good judgment and persistence.”
Mun's fascination with complex structures began in childhood. Drawn to the challenge of thinking deeply about difficult problems, he enjoyed how mathematics uses logic and creativity. The pursuit of uncovering the underlying structure of complex ideas motivates his research today.
That passion also led him to Carnegie Mellon, where the Department of Mathematical Sciences offered both the depth and breadth he was seeking.
"The department has many faculty members working on applied mathematical questions spanning physics, finance and computer science, while also possessing deep expertise in analysis and probability theory," Mun said. “CMU is a perfect fit for me to study and conduct modern mathematical research.”
The Simons Dissertation Fellowship provides research support to excellent graduate students in mathematics in the final years of their Ph.D. This program was created to honor the work of the organization’s late co-founder, Jim Simons, and his commitment to supporting the field of mathematics.
