Insider Brief
- A study from Bar-Ilan University published in Physical Review E finds that deep learning systems can eliminate up to 90% of internal parameters through a method called pruning, without compromising performance.
- The researchers showed that identifying which parts of a neural network are essential leads to smaller, faster, and less energy-intensive models—potentially reducing the environmental and computational costs of AI.
- The team validated their method with simulations and mathematical modeling, though further testing in larger systems is needed to confirm broad applicability.
A study funded and conducted by Bar-Ilan University has found that deep learning systems can eliminate up to 90% of their internal parameters without reducing performance, something that could sharply lower AI’s energy and memory demands.
Published in the journal Physical Review E, the research shows that a better understanding of how deep networks learn can help identify which parameters are essential and which are not. By removing the redundant parts of a neural network—a process known as pruning—the team demonstrated that AI systems can run with far less complexity while maintaining accuracy, according to the university. This has major implications for everything from mobile AI applications to large-scale data centers, where memory use and energy consumption are significant concerns.
Deep learning systems power many of today’s AI tools, from image recognition to natural language processing. These systems operate through layers of simulated neurons, each governed by thousands or millions oIsraeli Studyf adjustable parameters that determine how the system learns from data. But as models grow larger, so do their computational and energy costs. The Bar-Ilan team tackled this by studying the underlying mechanics of deep learning to determine which parts of the network were actually doing the work.
The researchers found that when training a deep network, many parameters turn out to be unnecessary for achieving accurate results. Their method allowed for pruning a large fraction of these parameters—in some cases up to 90% in specific layers—without degrading the system’s ability to learn or function. The result is a leaner, more efficient neural network that uses fewer resources.
This efficiency could help democratize AI by making high-performing models more accessible to developers working on limited hardware. It could also reduce the environmental impact of AI, which has come under scrutiny for its growing carbon footprint due to energy-intensive training processes.
“It all hinges on an initial understanding of what happens in deep networks, how they learn and what parameters are essential to its learning,” said Prof. Ido Kanter, of Bar-Ilan’s Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research. “It’s the ever-present reality of scientific research, the more we know the better we understand, and in turn, the better and more efficient the technology we can create.”
