Insider Brief
- Distributional has raised $11 million in a Seed round led by Andreessen Horowitz to build a platform for AI testing and evaluation, aiming to make AI technologies safe, secure, and reliable across various industries.
- The platform addresses the challenges of AI unpredictability and risk in production by offering a robust testing solution that is applicable to all model types, including machine learning, deep learning, and large language models.
- Founded by CEO Scott Clark and a team with experience in AI system testing at major tech companies, Distributional plans to use the investment to further develop its product and expand its team, with a launch planned for the second half of 2024.
PRESS RELEASE — SAN FRANCISCO/December 14, 2023— (BUSINESS WIRE) — Today, Distributional announced that it has raised $11 million to build the modern enterprise platform for artificial intelligence (AI) testing and evaluation, with the goal of making all forms of AI safe, secure and reliable. The Seed round was led by Andreessen Horowitz with participation from Operator Stack, Point72 Ventures, SV Angel, Two Sigma, Willowtree Investments and dozens of AI leaders as angel investors.
“I directly experienced this testing problem while applying AI at Yelp, optimizing models for customers at SigOpt and running a hundred-person AI & HPC engineering team at Intel,” says Scott Clark, Co-Founder and CEO of Distributional. “I learned that to robustly test AI I needed to evaluate distributions of outcomes and that there is no purpose-built software for this task.”
AI is complex, unpredictable and constantly changing. Whether due to hallucinations, instability, inaccuracy, or dozens of other potential challenges, it can be hard to identify, understand and address AI risk. To meet this challenge, some AI product teams rely on insights gathered during training that rarely translate to model behavior in production. Others rely on monitoring to quickly catch errors in production, but this leaves their customers exposed to potential harm or a poor user experience. And some teams run bespoke tests on their models prior to production, but these tests are inconsistent, incomplete and insufficient.
“Lack of reliability in AI systems is one of the biggest barriers to widespread enterprise adoption,” says Martin Casado, General Partner at Andreessen Horowitz. “We are excited for Distributional to address this problem by building a platform for robust and repeatable AI testing.”
Distributional is working with more than a dozen design partners to build an active testing platform that makes it easy for AI product teams across finance, technology, energy and manufacturing industries to get a complete view of AI risk. The platform will handle all model types, including statistical models, machine learning, deep learning, large language models and other forms of generative AI. With Distributional, AI product teams will continuously catch and address issues before production.
“A number of AI product managers that I have spoken with have told me models are failing in production with increasing regularity,” says Noah Carr, partner at Point72 Ventures. “As a result, I believe generative foundation models are becoming more critical. As demand for implementations grows, so does the potential risk that applications leveraging these models will be pulled offline due to issues related to model shift or exposure to misinformation. We are excited to back Distributional’s efforts to enable these teams to catch such issues before their customers do.”
Distributional was founded by CEO Scott Clark and an 11-person founding team with experience testing complex AI systems at Bloomberg, Google, Meta, Intel, SigOpt, Slack, Stripe, Uber and Yelp. Scott previously co-founded the pioneering AI startup SigOpt, which was funded by Andreessen Horowitz in 2016 and acquired by Intel in 2020.
Distributional is remote first and will use the investment to further develop its product and grow its team. The company plans to launch its enterprise product in the second half of 2024.
For more information, please visit distributional.com.
About Distributional
Distributional is building the modern enterprise platform for AI testing and evaluation to make AI safe, secure and reliable. As the power of AI applications grows, so does the risk of harm. AI product teams use our platform to proactively and continuously identify, understand and address AI risk before it harms their customers in production. Distributional is backed by Andreessen Horowitz, Operator Stack, Point72 Ventures, SV Angel, Two Sigma, Willowtree Investments and dozens of AI leaders. Distributional was founded in September 2023 by a team with experience testing AI systems at Bloomberg, Google, Intel, Meta, SigOpt, Slack, Stripe, Uber and Yelp.
