Extending AI Architectures to Address Continuous Scientific Problems

Artificial intelligence is proving to be transformative in its ability to work with language and images. Now, with a growing push to apply AI to scientific discovery, Caltech's Anima Anandkumar says there is a crucial ingredient missing from most AI models: the ability to understand the physical world. Take as an example weather models, says Anandkumar, Caltech's Bren Professor of Computing and Mathematical Sciences. If you want an AI model to predict weather, it must understand chaotic physical systems, like how the atmosphere changes around the planet and over time.

The best way to gain that understanding is to learn the continuous functions, the underlying mathematical relationships, that fully describe those systems. But many AI models were created for language processing or computer vision (where AI learns to process visual data). In those categories, data can be treated as collections of isolated points, such as words in a sentence or a fixed set of pixels in an image.

To address this mismatch, Anandkumar and her colleagues from Caltech and tech giant NVIDIA recently described a framework that shows how to extend existing neural network architectures in a way that allows AI systems to learn continuous functions, which are needed to make predictions that are grounded in the physical world. A paper about the framework was published in the journal Nature Machine Intelligence.

Anandkumar first introduced these models, called neural operators, in 2020.

"If you think of weather, it happens everywhere around us. It's not just at the points where the satellites are or where measurements are taken at certain locations. It happens everywhere in between," Anandkumar says. "If you design neural architectures that also directly approximate everywhere in the continuous space—and that imagine what the continuous world looks like rather than only at a few points—that is much more faithful in capturing the physical world."

Read more on the Caltech website.