AI FOR SCIENCE · INTERVIEWS
Zou says today's models are trained to answer questions, not to ask them, so choosing the question is still a person's job. What his lab is betting on next is more agents working together, and how they behave as a group: who doubts, how work is split, who reviews.
Two results from his lab, given with no paper named and no number: taking the skeptic agent out of a Virtual Lab team gave substantially more mistakes, and on teamwork exercises that organisational research gives people, AI teams often did much worse than human teams. He also reads the gap between three companies' models as reviewers at Agents4Science as default dispositions, and proposes AI review for objective errors only.
A written Q&A, edited with AI, about method and direction rather than one result. He gives no figure for the skeptic or the teamwork finding, and names no system that runs a research loop for months; that, he says, first needs creativity, question setting and coordination.
He says pre-training, post-training and reinforcement learning all reward answering a question well, and none rewards asking a good one, so AI has made much less progress at asking. Deciding what to ask still takes a person's creativity and taste, and researchers differ on what makes a question interesting. He wants people to stay in charge and provide oversight, with a virtual lab behind every researcher: reading the literature, finding ideas, checking that results reproduce and analyses are robust.
“But answering questions is only half of the puzzle in science. The other half is figuring out what questions to ask.”James ZouAsk a model again and again for solutions to a scientific problem and the first one or two are very reasonable; after that it mostly gives small variations of the same standard ideas. He ties this to the training objective, which rewards reproducing existing knowledge. His group's fix is to make the model move explicitly into distant fields (physics, economics, social science), find a problem with the same relational structure, and map that field's existing solution back. He says this gives substantially more diverse solutions, and the ones they implemented gave strong quantitative results.
Diversity and novelty are easy to measure, he says, but they are proxies: of many crazy ideas, some are crazy and good and some are only crazy, and telling them apart takes judgment that current evaluations and models do not capture. Another fix he offers is to change the training objective: reward the model on how good the best of its solutions is, not the average, so many ideas can fail if one is excellent. He says this small change gave better solutions across a range of scientific problems and, in several settings, new state-of-the-art results.
He says scaling used to mean making one model bigger; now the number of agents working together can grow too, from five or ten to thousands and eventually perhaps millions. He lays out his lab's path this way: the Virtual Lab with a few agents, the Virtual Biotech with dozens, and Physics of Agents, which uses statistical physics to predict how large numbers of interacting agents behave as a group. New forms of collective intelligence and creativity could come out of those interactions, he says.
“Rather than only increasing the size of an individual model, we can increase the number of agents collaborating with one another.”James ZouThe interviewer sums up Physics of Agents: repeated interaction makes agents more convinced and pushes them toward consensus, which improves accuracy on maths questions and can amplify bias on subjective political statements. Zou says many good drug programmes are killed for reasons beyond the science: commercial considerations, patents, the competitive landscape, human blind spots. So his lab tested the role of a skeptic: take it out of a Virtual Lab team and the same team makes substantially more mistakes. Frontier models, he adds, are not naturally good at teamwork.
“We found that a team without the skeptic often makes substantially more mistakes than the same team when the skeptic is added back.”James ZouSubmissions have exploded, he says, and the review system is becoming unsustainable. AI is good at technical mistakes, inconsistencies and mismatched numbers; how important a paper is, how creative, whether the direction is worth pursuing, is still where human experts are stronger. At Agents4Science, given the same instructions, models from OpenAI, Anthropic and Google reviewed the same papers: OpenAI's was the most critical and gave the lowest scores, Gemini the most positive, Claude in between. Much AI use in writing and reviewing happens informally and is sometimes hidden, he says, and needs transparent community norms.
“The OpenAI model tended to be the most critical and skeptical, and often gave the lowest scores.”James ZouBefore this conversation the timeline held no entry from Zou's lab: not the Virtual Lab (Nature, July 2025), not Agents4Science, not the two preprints; its first two, Paper2Agent (16 September) and the Virtual Biotech (17 September), came after. The multi-agent research system on the record was Google's co-scientist: previewed on 2025-02-19, in Nature on 2026-05-19.
Zou sets his lab's systems on one line, from five to ten agents to dozens to thousands, and puts the number of collaborating agents forward as a direction to scale alongside model size.
The Physics of Agents preprint (2026-08-17) studied more than 10,000 agent communities and reports that interaction improves collective accuracy on maths questions and drifts group opinion to the right on political statements; it is about how a group converges, and does not test what one role in a team contributes.
Zou gives two results, neither with a number: the Virtual Lab makes substantially more mistakes without its skeptic, and on organisational teamwork exercises AI teams often do much worse than human teams.
The timeline's entry of 25 March has one of three fully AI-written papers from Sakana's AI Scientist passing a workshop's first round of human review. The Agents4Science report (arXiv, November 2025) gives the three model reviewers' average scores on a 1 to 6 scale: GPT-5 2.30, Gemini 2.5 Pro 4.23, Claude Sonnet 4 3.0; GPT-5 and Claude came closest to the human scores.
Zou reads the gap as each model's default disposition, and from it proposes a split in review: AI checks objective errors, people judge significance and creativity; he also asks for AI use in writing and reviewing to be made open.
If removing one skeptic gives substantially more mistakes, which roles a team carries and who can veto need a controlled test as much as the choice of model does. The comparison he cites comes from the Virtual Lab, with a few agents; for the much larger Virtual Biotech, Bravo-Abad, having read the paper, says there is no run with the reviewer agent removed.
In the Agents4Science numbers, the same papers reviewed by another model can average almost two points apart on a 6-point scale; Gemini's average gap from the human scores is 2.73, and the report quotes it praising a paper the human reviewer doubted. So bringing AI into review means first deciding which part it owns: Zou's answer is objective errors, and the report's examples are mismatched numbers and conclusions that contradict each other.
Startups adopt AI faster, he says, but the hardest part of drug discovery is often showing safety and efficacy in people, which takes the experience, resources and scale of large pharmaceutical companies. On his account, agents that propose good molecules more efficiently are still only the start. That is his judgment; the interview gives no figures.
Models are rewarded for answering questions well, not for asking good ones; asking is where human experts still have the advantage.
Guest's judgment
His reading of how models are trained, with no measurement offered.
Asked repeatedly for solutions, a model mostly falls back on small variations of the same standard ideas.
Company-reported
His group's preprint abstract quantifies it as mode collapse, with baselines giving a novel solution as little as 1.6% of the time.
Analogical reasoning gives substantially more diverse solutions, and the ones implemented gave strong quantitative results.
Company-reported
The preprint's abstract: diversity metrics up 90% to 173%, a novel solution more than half the time, four biomedical problems implemented, state of the art on two oligonucleotide datasets. Only the abstract was read; an arXiv preprint.
Diversity and novelty are proxies for creativity; telling good from merely odd takes taste and judgment that current evaluations miss.
Guest's judgment
His own view.
Rewarding the best of a batch of solutions instead of the average gave better solutions across scientific problems and, in several settings, new state-of-the-art results.
Company-reported
He names no paper. The abstract of the TTT-Discover preprint he co-authored (arXiv 2601.16175) describes exactly this objective; its results were not read for this page.
The number of collaborating agents is a new axis to scale beside model size, from five or ten to thousands and eventually millions.
Guest's judgment
Here he gives the Virtual Biotech 'dozens of agents'; the Science paper the timeline records on 17 September assigns up to 37,000, which Bravo-Abad reads as a chief scientific officer, eight AI scientists and four divisions dispatching large numbers of single-task agents.
Taking the skeptic agent out of a Virtual Lab team makes the same team make substantially more mistakes.
Company-reported
No figure is given; only the abstract of the Virtual Lab's Nature paper was read, so whether the paper carries this comparison was not checked.
On the teamwork exercises organisational research and business schools give people, AI teams often did much worse than human teams.
Company-reported
He says only 'some recent work', naming no paper, exercise or score.
AI reviewers are good at technical mistakes, inconsistencies and mismatched numbers, and weaker on significance and creativity.
Guest's judgment
The Agents4Science report gives examples of model reviewers catching a mismatched number and contradictory claims, but no rate of errors caught.
Under the same instructions, OpenAI's model was the most critical and gave the lowest scores, Gemini the most positive, Claude in between.
Company-reported
Matches the Agents4Science report: on a 1 to 6 scale GPT-5 averaged 2.30, Gemini 2.5 Pro 4.23, Claude Sonnet 4 3.0, with average gaps from the human scores of 0.91, 2.73 and 1.09. The report was read in full.
Growth in submissions is already making peer review unsustainable.
Guest's judgment
No submission figures are given.
In outcome-driven science such as a drug, not fully understanding the mechanism, or how the AI reached it, may be acceptable once it is shown to be safe and effective.
Guest's judgment
His view; for curiosity-driven science, he says, the opposite holds and understanding the reasoning is the point.
Startups adopt AI faster, but the hardest part of drug discovery is showing safety and efficacy in people, where large pharmaceutical companies' experience and scale matter more.
Guest's judgment
His judgment, with no figures or examples.
A virtual lab behind every human researcher.
Target
An aim he states, with no date, plan or scale.
A virtual lab that runs a closed research loop for months, not hours or one project.
Not yet closed
In his own account it still needs creativity, choosing the next question, coordination and criticism; he names no system that does it.
Whether a run with the skeptic or reviewer agent removed is published for a large agent system.
Would show up as A paper giving error rates with and without the doubting role, over repeated runs rather than a single case.Still open Bravo-Abad, who read the Virtual Biotech's Science paper and supplement (2026-09-22), says it has no run with the reviewer removed and runs each case study once; this page did not read the paper itself.Whether the comparison of AI and human teams on the same teamwork exercises is published.
Would show up as A paper naming the exercises, the models and both sides' scores.Whether solutions found by analogical reasoning hold up beyond the authors' own benchmarks.
Would show up as A peer-reviewed version, or another group implementing a method found this way and reporting the gain.Whether a journal or conference splits review the way he proposes and publishes how it went.
Would show up as The venue's own report: which errors AI flagged, which decisions human reviewers changed, how many errors were missed.Whether an agent system runs a research loop for weeks or months.
Would show up as A paper stating how long it ran, how the agents split the work and how often a person stepped in.Later evidence that tests or limits what was said, and targets whose date has passed.
Eight days later his group's Paper2Agent appeared in Nature: it turns a paper's code, data and workflows into tools an agent can call, and reproduced AlphaGenome and Scanpy/TISSUE results. It is one concrete form of his virtual lab checking that results reproduce, though only for the few papers it shows.Timeline entry →
The Virtual Biotech appeared in Science: up to 37,000 Claude-based agents annotated 55,984 trials and found drugs against cell-type-specific genes reach market 48% more often. It is his largest case of scaling the number of agents; its lung-cancer proposal was reviewed by experts on paper, not made or tested, which fits his point that the hardest step is in people.Timeline entry →
Bravo-Abad, reading the Virtual Biotech's methods and supplement, notes that its reviewer agent runs on the weakest model in the system (Haiku 4.5) and can read but not re-run analyses, that there is no run with the reviewer removed, and that each case study ran once. So the skeptic comparison Zou cites comes only from the much smaller Virtual Lab.Discovery at Scale (Jorge Bravo-Abad) ↗
Read 2026.11.01 The full written Q&A on Decoding Bio (about 4,400 words, questions by Pablo Lubroth), read in full; the post says the conversation was edited with AI for length and clarity, so quotes are the edited text, not his spoken words. The Agents4Science report it mentions was read in full; the two preprints and the Virtual Lab's Nature paper only in their abstracts. Open the interview ↗