ScientistTwo: From Research Problem to Paper, Fully Autonomously

Researchers at Google Cloud AI Research, working with a collaborator at the University of Waterloo, have introduced ScientistTwo, a multi-agent AI system built to take a research problem and return a working codebase and a complete research paper without human intervention. The study was posted as a preprint on arXiv in September 2026 and has not yet been peer-reviewed.

The team tested the system on 107 machine-learning problems taken from papers accepted at NeurIPS 2025, ICLR 2026 and ICML 2026. ScientistTwo improved on the published human result in 86 of the 107 cases (80.4%). The average relative gain was 25.2% and the median gain was 7.7%.

From limitation to paper: how ScientistTwo works

ScientistTwo connects a series of specialised AI agents, each handling one stage of a typical research project:

• Finding weaknesses: agents identify and verify limitations in the current best method, then generate candidate ideas and check them for novelty.
• Testing ideas: coding agents reproduce the baseline and try each idea on a subset of the data. Only promising ideas move on to the full benchmark.
• Refining and ablating: an “evolver” agent improves ideas using execution traces, and ablation agents test which components actually drive the gain.
• Writing and review: a drafting agent writes the paper and an AI peer reviewer critiques it. Rebuttal agents then run extra experiments, and a meta-review agent decides whether to accept the paper or loop back for another round.

According to the paper, most of the agents run on Gemini 3.6 Flash, while the heavier coding steps use Claude Code with Opus 4.8. The authors also built in integrity checks. In an audit of 49 final outputs, every reported score was verified, no task-specification violations were found, and none of 1,814 references was flagged as hallucinated.

Key terms in plain language

• Multi-agent framework: several AI models, each given a narrow role, that pass work between one another. CBIRT has covered other examples, such as Robin, a multi-agent framework for scientific discovery.
• Human state-of-the-art (SOTA) baseline: the best result reported in the original human-written paper for a task.
• Ablation study: removing parts of a method one at a time to see which parts matter.

How good are the papers?

The authors scored the outputs with two automated reviewers. The first, ScholarPeer, is also used inside ScientistTwo’s own writing loop. The second, the Stanford Agentic Reviewer, was held out from that loop. On ScholarPeer, ScientistTwo papers averaged 7.5 out of 10. On the held-out Stanford reviewer they averaged 5.7, below the 6.1 that human-written ICML 2026 spotlight papers received.

Nine experienced human reviewers also rated 33 of the papers. They judged them broadly on par with human-written work, with a pairwise score of 3.0 on a 5-point scale. On method soundness, however, they slightly favoured the human papers.

None of these papers was submitted to or accepted by a real conference. Where the paper describes outputs as “accepted”, it means they scored well with the AI reviewers.

Limitations the authors acknowledge

• The papers do not yet reach spotlight or oral quality. The gains come from local algorithmic refinements rather than new theory.
• Each task costs roughly US$3,800 and takes 2–3 days, which the authors note may put the system out of reach for many academic labs.
• One case of “reward hacking” (gaming the evaluation metric) was found and filtered out.
• The comparison with a rival system, AutoSOTA, is not head-to-head, because each system was measured against its own reproduced baseline.

What it means for bioinformatics researchers

The benchmark is almost entirely machine learning, and the paper reports no biology or genomics results. A few of the input problems came from life-science-related ML papers, such as protein fold classification, EEG decoding and virtual screening, but the paper does not report how ScientistTwo did on those individually. Practical takeaways:

• AI agents are starting to handle the “improve the baseline” layer of computational research: extending, tuning and ablating existing methods. In biology, similar agent tools are already appearing, from agentic AI for RNA-seq analysis to BiOmics and the NVIDIA BioNeMo Agent Toolkit.
• Verification still matters. The authors themselves rely on reproducible code, honest ablations and independent review to keep outputs trustworthy.
• Cost and compute remain real barriers for now.
• Knowing how to read, run and check code is becoming more valuable, not less. If you’re starting out, our Python for Bioinformatics 30-day roadmap is a practical first step.

For an earlier Google effort in this space, read our coverage of Google’s AI co-scientist.

Would you trust an AI agent to run the next round of experiments on your own pipeline? Tell us in the comments, and follow CBIRT for more research news on AI in biology.

Reference: Nam J, Yoon J, Pan Y, Wang Y, Meng R, Ranganathan P, Pfister T. ScientistTwo: Pioneering the Human Knowledge Frontier with Autonomous AI. arXiv 2609.19644 [cs.AI], 2026 (preprint). https://doi.org/10.48550/arXiv.2609.19644