Insider Brief
- Insilico Medicine launched the Open Drug Discovery and Development Consortium, or O3DC, to improve transparency around how computational drug-discovery benchmarks are evaluated and used.
- O3DC’s Benchmark Index catalogs hundreds of benchmarks across 10 categories while documenting known biases, limitations and criticisms that may affect how performance results are interpreted.
- The free, community-curated initiative includes Insilico’s own benchmarking tools and allows researchers to challenge entries, add benchmarks and contribute to ongoing curation.
- Image: National Cancer Institute
PRESS RELEASE — Insilico Medicine (“Insilico”; HKEX: 3696), a clinical-stage generative artificial intelligence (AI)-driven drug discovery company, today announced the launch of the Open Drug Discovery and Development Consortium (O3DC) – www.o3dc.org, an open initiative addressing how computational drug discovery methods are evaluated, together with its first shared resource: a community-curated index of the benchmarks the field relies on.
The O3DC Benchmark Index catalogues hundreds of benchmarks across 10 categories, associated with 15 consortia and initiatives, spanning molecular property prediction, docking and pose prediction, binding affinity, generative design, retrosynthesis, target identification, clinical prediction, and AI agent evaluation. For each benchmark, it records what is measured, who maintains it, where the code lives, and whether that repository is still active, verified against GitHub each time the page loads.
It also records something no comparable catalogue does: the known caveat. The documented bias, limitation, or published criticism that a benchmark’s own description does not mention.
“Benchmarks define what our field counts as progress,” said Alex Zhavoronkov, PhD, founder and co-CEO of Insilico Medicine. “The irony is that while we are great at uncovering a benchmark’s flaws, we have nowhere to capture those insights. They get buried in the discussion section of a paper you’d already need to know about, all while the benchmark’s citation count keeps climbing. O3DC gives that knowledge a home-right alongside the benchmark itself.”
A standard industry test can easily be aced without ever touching the data it was designed to evaluate. While DUD-E has long served as a go-to benchmark for virtual screening, research reveals a critical bias: models can separate actives from decoys using ligand features alone, without capturing true protein-target interactions. Consequently, top benchmark scores rarely translate to real-world, structure-based screening ability. Thus, while DUD-E is not entirely obsolete, researchers must account for its architectural flaws rather than taking performance at face value. Similarly, PDBbind powers most machine-learned scoring functions while simultaneously underpinning CASF – their primary evaluation benchmark – introducing a data leakage and overlap rarely quantified in published literature. Further complicating model validation, PoseBusters demonstrated that several deep-learning docking tools can outperform classical methods on RMSD metrics while routinely producing poses that violate basic physical laws.
Although these critical limitations are well-documented in the literature and widely accepted, none are flagged when a researcher downloads the data. To bridge this gap, Insilico has launched the Open Drug Discovery and Development Consortium (O3DC), alongside its flagship shared resource: a community-curated index of the field’s core benchmarks.
Radical Transparency: Open to All, Open About Limits
- Community-Driven: Completely free with no registration required; every entry includes a public thread for maintainers and users to contest recorded findings.
- Self-Applied Standards: Insilico lists its own tools (including the newly launched DDD Benchmark as a Service, the Drug Discovery Benchmark leaderboard, Science MMAI Gym, TargetBench, and ChemCensor) alongside every other entry.
- Unfiltered Caveats: The caveat logged against Insilico’s DDD Benchmark explicitly calls out that its reference baselines are proprietary and closed to independent inspection.
“An index like this only carries credibility if it holds its host organization to the exact same standard,” said Alex Zhavoronkov, PhD, founder and co-CEO of Insilico Medicine. “We would rather face criticism in the open than claim a neutrality we cannot prove.”
Since its founding, Insilico has nominated 31 preclinical candidates, received over 10 investigational new drug (IND) clearances, and compressed the timeline to PCC nomination to roughly 12 to 18 months, compared with the 2.5 to 4-plus years typical of traditional drug discovery. Its lead program, Rentosertib (ISM001-055), is a first-in-class, AI-discovered and AI-designed TNIK inhibitor now in Phase III development for IPF.
How to Participate
O3DC is open to researchers across academia, industry, and independent practice with no membership fees or exclusivity. The consortium is actively seeking contributors to update existing entries, catalog new benchmarks, and lead the curation of specific categories.
Access the index at www.o3dc.org. For inquiries and membership requests, contact mmaigym@insilicomedicine.com.
