Insider Brief
- Researchers found that matching echo state network hyperparameters to the time scale of target data can improve prediction accuracy and reduce trial-and-error tuning.
- The optimal spectral radius increased for systems with longer time scales, allowing the network to retain information for longer periods.
- The approach could support more efficient time-series forecasting in areas such as weather prediction, robotics and enterprise operations.
PRESS RELEASE — Neural networks, a fascinating technology inspired by the human brain, form the basis of artificial intelligence. These networks consist of layers of interconnected nodes, or artificial neurons, that learn patterns from data and make predictions. For example, large language models generate text by predicting the next word or phrase based on the words that came before it.
Traditional neural networks require many internal connections to be adjusted before the model can make accurate predictions. An approach called echo state networks (ESNs) simplifies this process by keeping most of the network fixed and training only the output layer. This reduces the number of parameters to be optimized, speeding up training while lowering computational demands. However, ESN performance is highly sensitive to hyperparameters, the settings that determine how the model behaves. Despite the importance of these hyperparameters, the relationship between the target system and the hyperparameters remains unexplored.
In a study that was published in Volume 17, Issue 3 of Nonlinear Theory and Its Applications (NOLTA), IEICE on July 01, 2026, a research group led by Professor Tohru Ikeguchi and Assistant Professor Kazuya Sawada from the Faculty of Engineering, Tokyo University of Science, Japan, found that these settings can be optimized by accounting for the time scale of the target system, or how quickly it changes over time.
“ESNs are a type of reservoir computing that has attracted attention in recent years. This method is known to be effective for tasks such as time series prediction, but the prediction accuracy depends on how the parameters are set, so it is necessary to set them properly. The novelty of this paper lies in its focus on the time scale of the time series to be predicted when setting these hyperparameters. Our analysis offers a unique perspective that has not been widely discussed in previous literature,” says Prof. Ikeguchi.
Previous studies have suggested that the optimal hyperparameters depend on the characteristics of the target system, particularly how patterns change over time, but the relationship has remained unclear. The researchers therefore focused on the time scale, which describes how quickly a system changes over time. They investigated whether systems with similar time scales would also show similar patterns in the hyperparameter settings associated with accurate prediction.
To test this idea, the research team studied three chaotic systems: the Lorenz system, the Rössler system, and the Chua circuit. They adjusted the time scales of these systems using decorrelation time, a measure of how quickly a system becomes less related to its past behavior. The researchers then tested how accurately ESNs predicted these systems across many hyperparameter settings. Figure 1 illustrates the procedure used to match the time scales of the different target systems.
The team also compared two training conditions: one using the same number of training data points and another using the same total trajectory length across different time scales. This allowed them to separate the effects of the time scale from the amount of training data.
When the time scales of the different systems were matched, the ranges of hyperparameter settings linked to high prediction accuracy showed similar patterns across all systems. This suggests that prediction performance may depend more on how a system changes over time than on the specific system itself.
In particular, systems with longer time scales achieved better prediction accuracy when larger spectral radius values—a setting that affects how long information remains in the network—were used. The researchers found that the optimal spectral radius increased as the time scale became longer, as shown in Figure 2.
The findings suggest that the time scale of a target time series may provide a practical way to choose ESN hyperparameters by narrowing the range of appropriate hyperparameters, thereby reducing the need for time-consuming trial-and-error searches. The results also suggest that decorrelation time could help estimate suitable settings directly, potentially improving ESNs in applications such as weather forecasting, robotics, and other prediction tasks.
“These results demonstrate the importance of determining the hyperparameters of ESNs based on the time scale of the target time series and provide universal design guidelines for how to set the hyperparameters of ESNs,” concludes Prof. Ikeguchi.
The authors also suggest that these findings require further validation to determine their applicability to various systems beyond chaotic dynamical systems.
