An alphaXiv-led study runs 936 AI agents from 35 labs as one research community for twelve weeks, and they publish 936 computational studies with code and data
Rehaan Ahmad (alphaXiv), Samuel Schmidgall (UCSF) and colleagues report that 35 human-run labs deployed 936 autonomous agents on a shared preprint server from August to November 2026. The agents published 936 studies with code and data, sent 9,500 messages and started their own peer-review journal. An agent audit traced 23,811 reported numbers to output files: 93.8% matched exactly and the rest differed only in rounding or format. Two LLM judges rated agent papers above matched human arXiv preprints on rigor and below them on novelty.
What happened: Rehaan Ahmad (alphaXiv), Samuel Schmidgall (UCSF) and colleagues report that 35 human-run labs deployed 936 autonomous agents on a shared preprint server from August to November 2026. The agents published 936 studies with code and data, sent 9,500 messages and started their own peer-review journal. An agent audit traced 23,811 reported numbers to output files: 93.8% matched exactly and the rest differed only in rounding or format. Two LLM judges rated agent papers above matched human arXiv preprints on rigor and below them on novelty.
Why it matters: It records hundreds of AI agents doing research as a community over three months, not one agent on one problem, and it separates where their output can be trusted from where it cannot. Numbers checked against code and output files matched; prose did not: 68% of 552 null-result abstracts called a non-significant result proof of no effect. Text-only AI reviewers missed flaws, while turning 156 major objections into scripts run on the data overturned the headline in 77. The authors' lesson for anyone using agents in research: verify against executable artifacts, not text.
Evidence and limits: All studies were computational, with no wet lab; local model experiments mostly used open models of at most 14 billion parameters. Cohort agents ran the audits, which check that numbers match saved outputs, not that experiments were valid; the clinical and solar audits are not yet expert-reviewed. The human comparison used two Gemini models as judges, and the agents' overall edge came entirely from Lab 1. The ban on human coding relied on participants' own disclosure. A preprint.

