How this site grades evidence

What the record currently says

616 entries recorded · 403 claims graded · 213 recorded, not graded · 7 carry a contested, corrected or withdrawn mark

Counted from the entries at build time, so this page cannot drift from the data behind it.

The five rungs

Depth is not a score. It records how far down the chain an observation was made, not how good the result is. A failure observed in deployment is deep evidence: of a failure. A brilliant result that exists only in software is depth 2, and that is not an insult.

1. 1
   Exists

   What has to be true: A result, system, dataset or benchmark has been released, with no result reported on it yet. Its existence alone does not establish a claim about the world.

   For example: A public agent platform; a new dataset or benchmark released without scores.

2. 2
   Computed

   What has to be true: The result exists in software: a benchmark score, prediction, proof, generated candidate or simulated outcome. A benchmark that scores models belongs here even when its tasks are built from real measurements: ranking models on existing data is computing, not checking. So is a new benchmark or dataset released with models' scores on it, and a speed or accuracy measured by running software on ordinary computers: running code is computing.

   For example: A database of predicted structures; an Olympiad score.

3. 3
   Checked against existing evidence

   What has to be true: The result is evaluated against measurements, records or operational systems that already exist, under a defined evaluation procedure. What is checked is a particular result, such as a prediction, a design or a forecast, not a ranking of models on a benchmark, which is rung 2. A model's own predictions scored against real measurements or records it was not trained on (held-out experiments, clinical labels, observed weather) are rung 3, even when the release is a database of predictions; the database alone is rung 2. A single worked example that reproduces an already-known result is not a defined procedure.

   For example: A blind structure assessment; beating an operational forecast system.

4. 4
   Tested in the physical world

   What has to be true: A new experiment or physical execution is performed because of the result: something is synthesized, measured, or run on physical hardware beyond ordinary computing, such as a quantum processor, a robot or a lab instrument.

   For example: Designed binders synthesised and assayed; a catalyst made and measured.

5. 5
   Used in practice

   What has to be true: The result moves beyond a research demonstration into sustained scientific, engineering, clinical or commercial use, or into a regulated clinical trial.

   For example: Code merged upstream; a randomised clinical readout; a deployed forecast gain.

Who checked it

• Internal: Evaluated by the originating team.
• External: Evaluated, executed or reviewed by another party.
• Replicated: A separate team independently reproduces the result.

When a claim moves

A claim can be contested, corrected, withdrawn or superseded after it is graded. That mark stays with the entry and does not change its depth: a result can be both deep and wrong.

What is graded, and what is not

The ladder grades claims. A funding round, a spin-out, a hire or a partnership announcement is recorded because it matters and has no claim in it to grade. An entry that looks gradable but is not carries its reason in the data: an institution declining to certify a result, for instance, is a fact about a process rather than evidence about the claim.

Version

Definitions are versioned. A number computed under changed rules is not comparable to one published before it, so the version moves when a definition changes, not when a single entry is regraded.

• v1.3 · 2026-10-21: Three lines that read both ways when a model graded entries from their sources. A dataset or benchmark is rung 1 only until scores on it are reported, then rung 2. Speed or accuracy from running software on ordinary computers is rung 2; rung 4's hardware means physical systems such as quantum processors, robots and lab instruments. A model's own predictions scored against real measurements it was not trained on are rung 3, even in a database release. Four entries were regraded under it (FrontierScience and SoLiD26 to 2, AlphaMissense and AlphaGenome to 3); the rest keep their grades until revisited.
• v1.2 · 2026-10-03: A benchmark that scores models is rung 2, even when its tasks are built from real measurements; rung 3 is for checking a particular result against existing evidence. The record had graded such benchmarks both ways. Entries graded under 1.1 keep their grades until they are revisited.
• v1.1 · 2026-09-29: Wording rewritten; two rungs renamed. Rung 3 is now 'Checked against existing evidence' and asks for a defined evaluation procedure rather than a protocol set before the result. Rung 5 is now 'Used in practice': sustained scientific, engineering, clinical or commercial use, or a regulated clinical trial, which rung 5 already covered. Entries graded under 1.0 keep their grades until they are revisited.
• v1.0 · 2026-09-17: First published. The scale had been applied privately since the timeline began; this fixes the definitions, states what is not graded, and back-fills every claim-bearing entry.

Disagree with a grade?

Say which entry, which rung you would put it on, and what in the public record supports that. Corrections are made in public and the changelog records them. Contact →