Insider Brief
- New research suggests the design of AI investment systems could influence whether automated investors amplify financial runs, fail to coordinate or support market stability.
- Q-learning agents sometimes withdrew excessively even when conditions favored staying invested, while large language models were less prone to runs but made less predictable collective decisions.
- The simulation does not show that AI will cause a market crash, and researchers called for studies involving live markets, mixed AI systems, human oversight and existing safeguards.
Artificial intelligence may amplify financial runs even when each automated investor is following a seemingly rational strategy.
That is the warning emerging from a series of simulated investment experiments conducted by researchers at Stanford University, the Deutsche Bundesbank, the European Central Bank and the University of Naples Federico II. Their findings suggest that the internal design of an AI system can influence whether automated investors remain calm, rush collectively for the exits or scatter in different directions during periods of financial stress.
The study was conducted in a simplified simulation involving withdrawals from a mutual fund, not in a live market. But it identifies mechanisms through which interacting AI agents could deepen a selloff, fund run or other financial crisis as more investment decisions are delegated to machines.
While the research does not show that AI will cause a stock-market crash, the team found that different kinds of AI could fail in different ways.
For example, AI agents trained through trial and error became highly coordinated but sometimes continued withdrawing money when the fund’s underlying condition no longer justified it. Large language models, or LLMs, were less prone to that excessive withdrawal. They struggled, however, to anticipate one another and reach a common decision under uncertain conditions.
The results suggest that regulators and financial institutions may eventually need to examine not only what an automated investment system is instructed to do, but also how it learns, reasons and responds to other machines.
Two Types of AI, Two Types of Risk
Researchers understand much about how an algorithm behaves alone, but comparatively little about what happens when large populations of autonomous machines make interdependent decisions.
Because financial markets are collective arrangements, an investor’s decision to sell may be reasonable in isolation, but thousands of similar decisions made at once can push down prices, drain liquidity and prompt further selling.
The researchers tested this problem by using a simulated mutual fund holding assets that could not be sold quickly without a cost. Each AI investor had to choose between redeeming its shares early for a smaller but certain payment or remaining invested for a potentially larger future return.
The return from staying depended partly on the fund’s underlying financial health. It also depended on what the other investors did. If enough investors withdrew, the fund had to sell assets, reducing the value available to those who remained.
This is known as a coordination problem — essentially, when the fund was very weak, withdrawing was the rational choice. But, when it was strong, staying was the better option. Between those conditions, either outcome could become self-reinforcing. Investors could remain together and preserve the fund, or they could redeem together and produce a run.
The researchers placed two types of AI into this setting.
The first used Q-learning, a form of reinforcement learning in which an algorithm improves its decisions through repeated trial and error. Such systems associate actions with rewards and gradually favor the choices that have produced better results.
The second used an LLM — identified in a presentation on the work as DeepSeek-R1 — which reached a decision by interpreting written information about the fund, its choices and the likely behavior of other investors.
While both groups received the same basic information and faced the same financial incentives, their behavior differed sharply, according to the researchers.
How AI Learns to Rush for the Exits
The Q-learning investors coordinated closely, but that coordination did not always produce a sound collective decision.
When the simulated fund faced a risk of default, staying invested in the fund could lead to a complete loss. Redeeming produced a smaller but guaranteed payment. After experiencing defaults during training, the algorithms reduced the value they assigned to staying and became more likely to withdraw.
Once an agent chose to redeem, it stopped learning what would have happened had it remained invested. That prevented it from discovering when staying might again have been the better choice.
The researchers described this as the “hot stove effect.” An algorithm that is punished for touching a hot stove may learn never to touch a stove again, including one that is cold.
In the experiments, Q-learning agents eventually reached a point at which they redeemed repeatedly, including under conditions in which staying invested should have been the dominant choice. Because the agents learned in similar ways, they also tended to act together.
The result resembled a financial run with the systems coordinating effectively, but on an outcome that harmed the group.
That behavior differs from algorithmic collusion, in which independent systems learn that cooperating can increase their profits. In the fund simulation, the agents did not collectively withdraw because it benefited them. Their learning process trapped them in what the researchers described as a lose-lose outcome.
The LLM investors generally recognized the fund’s underlying condition and did not redeem when the financial case for staying was strong. In situations where either staying or redeeming could be rational, however, identical LLM agents formed different expectations about what the other agents would do.
Some concluded that the other investors would remain while others expected a run. Those “beliefs” then shaped their decisions, producing a mix of withdrawals and investments rather than a coordinated outcome.
This result may illustrate why individually reasonable AI decisions do not necessarily add up to a stable market. Each model could provide a coherent explanation for its choice while the population as a whole remained unable to coordinate.
It also challenges the assumption that using identical models and prompts will produce uniform behavior. In the experiment, the LLM agents received the same instructions and operated with the model’s temperature set to zero, a setting intended to reduce randomness. They still developed three distinct beliefs about the likely behavior of their peers.
Can Safeguards Prevent an AI-Driven Run?
The researchers also tested several ways to reduce the problems.
Providing the LLM agents with slightly different private signals about the fund helped them coordinate. The added information gave the models a reference point for choosing between competing expectations. Their behavior then closely followed the outcome predicted by economic theory.
In what seems to be a counterintuitive finding, giving every agent the same information might seem more likely to produce agreement, but when that shared information supported more than one possible outcome, the models lacked a basis for choosing among them. Small private signals helped resolve that uncertainty.
Showing the LLM agents a history of earlier decisions also caused their beliefs to converge, but that did not guarantee a beneficial result. The agents coordinated around whichever pattern had emerged previously, meaning that a history of withdrawals could reinforce further withdrawals.
The researchers also attempted to correct the Q-learning agents’ excessive-redemption bias. They showed each agent the hypothetical return it would have received from the decision it did not make. This allowed the system to keep learning about both redeeming and staying invested.
The change eliminated the agents’ permanent rush for the exits, but it created another problem. Instead of settling on a stable choice, the algorithms moved back and forth between the alternatives.
These results indicate another issue in AI trading — correcting one failure may expose another. A system designed to prevent excessive coordination could become unpredictable, while a mechanism that promotes coordination could direct agents toward a harmful outcome.
Possible protections include circuit breakers and trading halts that slow activity when markets move too rapidly. Unlike safeguards designed around a particular algorithm, those controls can operate regardless of which AI architecture is making the decisions.
Financial institutions may also need to determine which AI systems customers or outside asset managers are using, according to the researchers. Traditional risk assessments focus on investors’ finances, experience and tolerance for loss. Widespread autonomous investing could add another variable involving the design and behavior of the technology itself.
What the Study Does Not Show
It’s important to understand the context of the work. The findings, for example, do not establish that current AI trading systems are likely to cause a market crash.
The experiment modeled a specific mutual-fund redemption problem under controlled conditions. It did not reproduce the number of participants, competing strategies, regulations, asset classes and human interventions found in an actual market.
The LLM portion also relied on one model and other commercial and open models could likely interpret the scenario differently. Real financial systems may combine reinforcement learning, language models, conventional trading rules and human judgment rather than rely on one architecture.
The study nevertheless has implications beyond mutual funds. Similar coordination problems occur during bank runs, currency attacks and runs on stablecoins, where confidence in a digital token can collapse if holders expect others to sell.
Further research will need to examine mixed markets in which different AI systems interact, according to Kazinnik’s presentation. Researchers also need to test hybrid agents that combine trial-and-error learning with language-based reasoning, as well as populations containing both people and machines.
Another unresolved question is whether an AI system’s architecture and behavior can be identified in real market data. Controlled experiments can show how a mechanism might work, but detecting it among millions of live transactions would be considerably harder.
Researchers will also need to determine whether existing protections can operate quickly enough when autonomous systems interact at machine speed.
The research team for “Ex Machina: Financial Stability in the Age of Artificial Intelligence”, published in the ECB research bulletin, included Kartik Anand of the Deutsche Bundesbank, Sophia Kazinnik of Stanford, Agnese Leonello of the ECB and Ettore Panetti of the University of Naples Federico II. Kazinnik placed the work within a broader question facing financial markets in a subsequent Economic Data Science Society presentation.
