Fast information processing with slow neurons
Bernese researchers have developed a theory that shows how the brain can efficiently learn extremely fast sequences of sensory stimuli. This can happen much quicker than previously thought if neurons (nerve cells) have a mechanism that allows them to "predict" the future. The Bernese work was selected for presentation from among nearly ten thousand submitted papers at the world's most important conference on artificial intelligence.
This year's NeurIPS conference (Conference on Neural Information Processing Systems) – the most important forum on artificial intelligence for decades – has selected a paper by a team led by Dr. Mihai Petrovici from the University of Bern for oral presentation. This places the selected article among the top 1% of the nearly ten thousand research papers submitted this year.
In their study, the Bernese researchers looked at how the deep, complex neuronal networks in the brain learn to recognize sensory stimuli. "Imagine a child sees a bicycle for the first time," explains Paul Haider, first author of the study. "The information flows from the retina in the eye through many neurons until it reaches a region in the brain where neurons specialize to capture this new concept of a bicycle. However, not only do these few high-level neurons have to learn to recognize bicycles as such, but all the neurons in between must also adapt to process the visual information as efficiently as possible."
This problem is known in neuroscience as the "credit assignment problem": What contribution do individual neurons make to the function of the network as a whole? A major difficulty here is that neurons cannot respond arbitrarily quickly: By the time neurons in the temporal lobe respond to the image of a bicycle, the eye may already be looking at another object nearby. Without a solution to this problem, false associations would be constantly learned. Until now, there has been no explanation of how our brain could overcome this challenge.
Neurons that look into the future
"Neurons may seem slow, but they have a trick," says Benjamin Ellenberger, another member of the research team. "They can use perceived changes to predict the immediate future to some extent. It's a bit like driving a car: if I know where I am and at what speed I'm going, I can roughly predict where I'll be in an hour." This ability compensates for the slowness of neurons, allowing signals to propagate through the brain at lightning speed. As a result, widely separated parts of the brain can synchronize and thus learn correct temporal associations. "Seeing and recognizing a car happens almost simultaneously," Ellenberger continues. "This allows us to quickly shift our gaze from object to object in our environment without limiting our ability to learn."
From biological to artificial intelligence
This idea not only helps to understand learning in the brain. The problem of slow information transfer is found in all physical systems and therefore also has great significance for research into artificial intelligence (AI). Laura Kriener, one of the study's other co-authors, is also involved in the development and application of so-called neuromorphic hardware. "Modern AI goes hand in hand with research in neurobiology," Kriener says. "Neuromorphic hardware combines aspects of both fields." These novel chip architectures contain circuits that behave very much like neurons in the brain. The mechanism now proposed by the researchers therefore also opens new possibilities for these already fast and energy-efficient systems.
"This work is only the beginning," says Mihai Petrovici. "We already plan to collaborate closely with colleagues from different research labs, both to investigate the predictions of our theory for the brain, and to implement its principles in neuromorphic circuits."
This study was funded by the Swiss National Science Foundation SNSF, the Manfred Stärk Foundation and the European Human Brain Project. The project was able to use the European Fenix Infrastructure resources as well as the Insel Data Science Center in Bern.
Interview with Mihai Petrovici
"Neurons have the ability to look into the future"
Mihai A. Petrovici researches the brain's remarkably efficient learning mechanisms at the Department of Physiology. In this interview, he talks about the fascination and progress of modern brain research, but also about the consequences of Switzerland's withdrawal from European research programs.
Paul. Haider, Benjamin Ellenberger, Laura Kriener, Jakob Jordan, Walter Senn, Mihai A. Petrovici: Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons. Advances in Neural Information Processing Systems 34 (NeurIPS), 2021, https://papers.nips.cc/paper/2021/file/94cdbdb84e8e1de8a725fa2ed61498a4-Paper.pdf
The Bernese contribution to the Human Brain Project
The Human Brain Project (HBP) is the largest brain science project in Europe and stands among the biggest research projects ever funded by the European Union. At the interface of neuroscience and information technology, the HBP investigates the brain and its diseases with the help of highly advanced methods from computing, neuroinformatics and artificial intelligence, and in turn drives innovation in fields like brain-inspired computing and neurorobotics. The long-term contribution of the HBP is EBRAINS, a permanent open platform for neuroscience and computing in the form of a European research infrastructure that will remain in place beyond the project period 2023. The Institute of Physiology at the University of Bern is involved in the HBP thanks to the groups of Dr. Mihai Petrovici and his collaborator, Prof. Dr. Walter Senn. In 2020, they received a grant of 2.5 million euros for this purpose. The groups are developing theoretical models of neurons and networks in the brain that link behavior, learning, and the corresponding processes and changes in the brain. The theories are based on biophysical concepts and allow reconstruction of biological processes in so-called neuromorphic hardware, i.e. computer chips that function similarly to the brain itself.