Fixed or Flexible? Study Shows Vision-Related Neurons Can Rapidly Switch Codes

For many years, a dominant view in neuroscience was that neurons in the inferotemporal (IT) cortex—a critical center in the brain for the recognition of objects—represent the world through fixed tuning functions. Doris Tsao (BS '96), who has been studying how the brain processes visual information for her entire career, believed this to be a fact. But now, with a team of other scientists that includes recent Caltech graduate Yuelin Shi (PhD '26), she has found evidence that the opposite is true.

In a recently published paper in Nature, the team shows that neurons in the IT cortex rapidly switch from a neural code that supports face detection to one optimized for face identification—all within roughly 20 milliseconds after a face is viewed. The results suggest a previously unrecognized neural mechanism that may help explain how the brain flexibly extracts different kinds of information from an image.

"The inferotemporal cortex has been the poster child for saying that the brain works like a deep network in machine learning—in which neurons or 'nodes' have set functions—and we thought we had figured everything out," says Tsao, chief scientist for neuroscience at the Astera Institute, a nonprofit research organization. Tsao was a Caltech faculty member from 2009 to 2021 and is a corresponding author on the paper. "It was really surprising to find that this code the cells are using is changing completely on this very fast timescale. It's still a little hard for me to accept."

The project began with a simple but fundamental question for the researchers: Could they predict how a face cell responds to faces using a model trained only on non-face objects? If so, that would suggest that neurons in the IT rely on a general-purpose code that works across categories, much like units in deep neural networks. If not, it would suggest that these neurons use specialized coding mechanisms for specific categories, making them domain specific. This idea speaks directly to a long-standing debate in neuroscience about whether the IT cortex is organized around general principles that apply to all visual categories or contains category-specific mechanisms. To answer this question, Shi and Tsao studied single neurons in the IT cortex of nonhuman primates as the animals viewed faces and non-face objects.

Read more on the Caltech website.