A Trio of Caltech Experts Explain How AI Is Transforming Scientific Discovery
From Left to Right: Anima Anandkumar (credit: Steve Babuljak), Maria Spiropulu, and Sara Beery (PhD ’23).
Anima Anandkumar, the Bren Professor of Computing and Mathematical Sciences; Maria Spiropulu, the Shang-Yi Ch'en Professor of Physics; and alumnus Sara Beery (PhD ’23), a faculty member at MIT, discuss the potential of AI to understand physical reality—even down to the fundamental quantum level.
by Adam Hadhazy
What is the future of scientific discovery in the new age of artificial intelligence (AI)? This question inspired a recent special double issue of Daedalus, the official quarterly academic journal of the American Academy of Arts and Sciences. Two researchers at Caltech, along with a recent Caltech alumna, contributed essays to the issue exploring how AI is already transforming their fields along with the broader enterprise of advancing human knowledge.
Caltech magazine brought the three researchers together to share their perspectives. They are Anima Anandkumar, the Bren Professor of Computing and Mathematical Sciences, who has done foundational work bringing together AI and science; Maria Spiropulu, the Shang-Yi Ch'en Professor of Physics, who specializes in high-energy particle physics and quantum science and technology; and Sara Beery (PhD ’23), who focuses on ecology and conservation technology as the Homer A. Burnell Career Development Professor on the Massachusetts Institute of Technology faculty of Artificial Intelligence and Decision-Making. (Beery was advised at Caltech by Pietro Perona, the Allen E. Puckett Professor of Electrical Engineering and director of Information Science and Technology.)
The conversation has been edited for length and clarity.
Anima, in your essay titled "How Do We Build AI to Push the Frontiers of Scientific Discovery?" you say that for AI models to generate truly novel discoveries, they need a broader physical understanding of the world. Can you describe what that means?
Anima Anandkumar: What is the true bottleneck for science? Scientists have so many ideas; some of them are sudden eureka ideas, some of them are long-thought-out ideas. But the bottleneck is not a lack of ideas. It’s going and testing which ideas work in the real world. For Maria in high-energy physics, that means running extremely expensive particle colliders. For Sara, that means going into the field, usually somewhere really remote, to collect ecological observations.
With large language models and other AI tools, we have a problem because they’re not grounded in physics and reality. They cannot do real-world tests. They’re just generating more ideas, and they cannot immediately say if every idea they’re generating is correct or not. To give you one example, Google DeepMind [a leading AI research lab] published a paper in 2023 where they claimed their AI tool came up with 380,000 new stable materials. But then later, when scientists checked, many were not even true materials; some were duplicates and others wouldn't work. Instead of AI finding the needle in a haystack, the AI just created more hay.
What I'm pitching instead is that we need to build AI that innately understands the physical world. You could then avoid some of the requirements of real-world observations, and that would accelerate scientific discovery.
Sara Beery: I agree with Anima that scientists have so many ideas that they can't test. I don't think we need models that try to come up with new hypotheses, which has been a focus of the AI science community recently. We need to figure out: How do we allow scientists to efficiently test the hypotheses that they already have, and what does that look like? For instance, how do we try to understand datasets that contain hundreds of millions of images, or thousands of hours of sound recordings, or all the data that's ever been taken looking at Earth from space?
There are a lot of scientific disciplines for which we have a good—maybe not perfect—but a good understanding of the underlying physics of the mathematical structures relevant to that discipline. For ecology, however, we’re far from that. We really can’t just sit and write down the mathematics of communities of living things and how they’re changing because of natural environmental and anthropogenic variables.
Anandkumar: I think of a physics-grounded AI coming up with the best design for a car because the model thoroughly knows fluid dynamics, aerodynamics, and can directly optimize.
Beery: That’s a good example. I imagine our ability to design a perfect car might happen a lot sooner than our ability to build a perfect simulator of Earth, of species, of evolution and ecosystems, of things that we know are highly chaotic, complex, and very difficult to write down mathematical functions for, let alone solve for. Of course, if we could perfectly simulate complex ecosystems in a way that we thought was reliable, we would!
In my essay, “AI & Ecology: From Tool to Transformation,” I talk about how when I started using AI 10 years ago, it was to automate labeling tasks, like for recognizing species or counting trees. I think in another 10 years, AI will have gotten to the point where it can now help us test hypotheses and optimize experimental designs, with scientists providing ethical oversight, context, creativity, and verification.
Maria Spiropulu: Sara, you have a much more difficult science than I do, in that sense, because you don’t have “ground truth,” meaning you only have direct observation and real-world measurements. The scale in ecology is just so big. In high-energy particle physics, we have 70 years of ground truth theory. That is why we test our AI models until they are able to reproduce the theory we have put together with all these decades of data and discoveries in experimental physics.
We still have huge holes, though, in our foundational physics theories, such as the standard model. It can’t explain the force of gravity or why the universe didn’t just self-annihilate because the big bang should produce the same amounts of matter and antimatter. So, we have to keep going, and AI is going to help us.
To that end, Maria, your essay, titled "Quantum + AI = Quantum AI," delves even deeper into this concept of grounding AI models in reality. The essay discusses taking models’ understanding all the way down to the quantum realm—the most fundamental operations that we can articulate. Tell us about this so-called quantum AI.
Spiropulu: In 1989, the physicist John Archibald Wheeler coined the phrase “it from bit,” meaning anything physical has a basis as a unit of information. So, the operational and functional role of every piece of matter in the universe is to be a piece, or quanta, of information. And because nature is quantum to its core, the most powerful AI will have to harness quantum information.
To get there, we’ll need better quantum computers that can accurately simulate nature’s quantum reality. There’s steady progress being made on quantum computers. Just recently, Google’s Willow chip computed quantities—called out-of-time-order correlators—beyond the reach of any classical supercomputer; these can read the geometry of molecules—exactly the kind of data that AI models like the protein structure predictor AlphaFold are starving for. [Editor’s note: Quantum computers use quantum states of objects, such as trapped ions and neutral atoms, to represent and process bits of information, dubbed “qubits,” and are increasingly capable of doing certain calculations—for instance, simulating quantum behavior—in mere fractions of the time it would take for traditional, “classical” computers, potentially slashing both training and compute times.] As we have all been hearing, the AI diet is very, very demanding, both in data for training and all the electricity required. Training a frontier large language model can use up about 10 gigawatt-hours and take months. [Ten gigawatt-hours is roughly a nuclear power plant running for 10 hours—enough to power about a thousand homes for a year.] The hope is that quantum AI needs much less electricity.
My co-author, Hartmut Neven [founder and manager of the Quantum Artificial Intelligence lab at Google], and I predict that before AI turns 100—it is about 75 now—it will have morphed into quantum AI.
Anandkumar: We will need new quantum-AI approaches. What we’ve done so far with language and vision models that has worked the best is we’ve taken the tons of data on the internet; put that into training the AI; used a very expressive, simple architecture—don't worry about anything else—and out comes magic in the form of a well-functioning AI tool. People have called this the “bitter lesson.” It’s bitter in that a simple AI approach is enough when given enough data and compute.
But this is not enough to make new scientific discoveries especially in materials space, because generating enough data with quantum-level calculations is out of reach even on our most powerful supercomputers. Even for a tiny system of just 400 electrons, it would require tracking 2400 quantum states, and that’s more atoms than there are in the universe. We know the precise physics; it's just impossible to calculate.
Yet even before we have quantum computers that can handle these sorts of calculations by designing physics-informed AI that not only relies on data but bakes in enough laws of physics at the quantum level, we’ve seen that we can have models generalize beyond what they're trained on in this way: They can make correct predictions on much bigger systems.
Beery: What Maria and Anima said resonates with me. In ecology, we’re not operating on the quantum level, and we don't have the kind of training paradigm that others have in large-scale internet data, where you're going to see the bitter lesson play out. For some sciences then, what we want to do is bring all of the very hard-won scientific knowledge that we do have to bear. For us, it's things like ecological traits, or the kind of dimensions of variability that you might see across individuals in a species. When we have that knowledge, and we think it’s reliable, and then we bake it in, we're able to generalize and overcome a lot of data limitations.
Do you think AI will be looked back on as a profound new tool in science similar to, say, the invention of the telescope and the microscope?
Anandkumar: Absolutely. One recent example that I think is illustrative is my joint work with Frances Arnold here at Caltech. She was awarded the Nobel Prize for discovering directed evolution, where you mimic and accelerate natural evolution in a lab to discover novel proteins, often in the form of enzymes with promising properties. In a recent paper of ours, we did the same but with AI models. The work traces back to a 2023 paper where we released the first genome-scale large language models that learn on DNA and RNA sequences. We trained a model on all known bacteria and viruses, and it was able to predict new variants of coronavirus and also discover new enzymes.
In collaboration with Frances’s group, we designed filters for these models to say which enzymes are going to be functional and versatile. The filters involved some more machine learning but also a bit of physics, like intuitions of how stable these enzymes are likely to be and other key characteristics like protein folding. We were able to show, to everybody's surprise, an enzyme that has been very well-researched for two decades using directed evolution. We could beat it with an AI-designed enzyme and make it even more capable and versatile.
To me, that's just the cusp of what we can already do. We can come up with better functional enzymes, better cures for disease, and there's so much more promise.
Spiropulu: It’s work like Anima’s that has me thinking that the biggest AI revolution will happen in the bio-chemical-medical world. In other sciences, AI will accelerate discovery, and that is inevitable, and that's how we are using it. I would go further: AI is not merely a scientific instrument like the telescope; it is like writing in Plato’s time and the printing press in Gutenberg’s. It has already profoundly changed everything in all sectors, and it will change more and very rapidly as it integrates into human civilization.
Beery: Clearly AI will be transformative in some spaces, but there are caveats here. We already know that our data-collection systems are imperfect. Now, AI is like another kind of imperfect filter on top of those imperfect data-collection systems. In ecology, we've been grappling with when it comes to actually intervening in real life, for instance, through conservation action or setting biodiversity policy. How do we robustly integrate AI into our systems without having the AI introduce biases that lead us to the wrong conclusions? AI researchers have already shown experimentally that these biases will almost always happen if you don't try to actively account for, understand, and address them. So, not just for science but for society, we should sound a note of caution as we advance further into the age of AI.