Inherent's Faraday: a 27B judgment model that calls a frontier coding agent

Inherent is a London company that came out of stealth in May 2026, founded largely by DeepMind alumni, with a $50m seed round co-led by Index Ventures and Radical Ventures. On August 13, 2026 it released Replica and Faraday.^[S3][S1]

The task is to reproduce one results figure.

Replica generates 310 tasks from 100 well-known machine-learning and AI-for-science papers: 242 for training, 68 for testing. Each hands the agent a paper with one results figure irreversibly redacted, and asks it to produce that figure in 60 minutes on one-seventh of an H200. Where the compute will not stretch to a full replication, the prompt asks for the most faithful scaled-down version. Tasks are generated automatically in three vision-language stages by Gemini 2.5 Pro, then inspected by hand with the poor ones filtered out.^[S1]

The test set changes field, not just papers.

Training tasks come from machine-learning papers published between 1990 and 2026; test tasks come from AI-for-science papers between 2012 and 2026. What is held out is therefore not a handful of unseen papers but an entire research area kept out of Replica post-training. The authors read this as evidence that what it acquired is a transferable way of approaching an underspecified task rather than a memorised procedure. That shows post-training never used these tasks, not that the model never saw the underlying knowledge: the paper's own Appendix D.1 says some of the chosen papers, figures included, are undoubtedly in the pre-training data of frontier multimodal models. The authors' argument is that the process data that produced those figures was never recorded, so no model has been trained on it.^[S1]

Coding agent as a tool

What is new here is not the score. It is the architecture.^[S1]

Faraday does not write the code.

Its base is the open-weight Qwen3.6-27B, post-trained on Replica with a turn-level credit variant of GRPO. It has five tools: apply_patch, read_file, list_dir, grep_files, shell. The code is written by a frontier coding agent: the Codex CLI, wrapped in a script that Faraday invokes through its shell, getting back a transcript of commands, outputs and per-step timings. It can resume the previous session, reset context, or run several in parallel. The paper calls this coding agent as a tool. Its point is explicit: Faraday's scientific capability comes from changing the policy weights, not from complicating the harness. The paper also says Faraday, at 27B parameters, directs a model with an estimated 5T, citing a third-party estimate (Li, 2026); frontier model sizes are not published, so this page says only that a 27B judgment model calls a frontier coding agent, not how much larger that agent is.^[S1]

Swap in a stronger executor and the score follows.

This is the result that makes it a pattern rather than a tuning trick. Faraday is trained initially with GPT-5.4 mini backing the Codex tool. The authors take the last checkpoint trained only against GPT-5.4 mini, swap the tool for GPT-5.5 with no retraining, and scores on the held-out AI-for-science tasks go up. The judgment layer is not welded to the executor it trained against.^[S1]

Prompting does not close the gap.

The authors ran 24 generations of automated prompt optimisation on the Codex baseline: each generation samples 10 training tasks with eight rollouts apiece, and Claude Opus 4.8 rewrites the prompt from the whole history, including the judge's feedback. The optimised prompt correctly identifies the failure modes, but does not perform meaningfully better, and the gap to Faraday is retained. That negative result holds for this setup only: one optimisation method, one baseline, one judge.^[S1]

The default assumption of the past two years has been that a stronger model means a stronger scientist, with the harness and tools covering the rest. This paper is a counterexample: strong execution is not the same as strong research. Adding a layer that has separately learned experiment selection, evidence evaluation and iteration buys capability that the executor does not have, and that layer need not sit inside the largest model.^[S1][S2]

Breaking the score apart makes the claim sharper. Faraday leads on experimental depth, claim reproduction and visual fidelity, and matches Claude on scientific integrity and implementation fidelity. It is not winning by writing better code; it is winning by running the investigation more thoroughly: trying more conditions, actually matching the paper's claim. That is the judgment half, not the execution half.^[S1]

The gain comes from training, not from the tool.

Fig. 2 carries a fourth curve: Qwen3.6-27B without the Replica post-training. The caption states that it runs in the same simple harness as Faraday, with the same Codex GPT-5.5 available as a coding tool, and that the two differ only in Faraday's RL post-training on Replica. Their mean scores are 0.678 against 0.856 on the 242 in-distribution (training-split) tasks and 0.554 against 0.791 on the 68 held-out ones. Handing a frontier coding agent to a small model is not enough on its own; the training is what closes the gap.^[S1]

One judge supplies the reward and scores the final comparison.

The rubric is generated per task by Claude Opus 4.7 from a meta-rubric, and the scoring is done by Codex GPT-5.5. It supplies the reward and the per-turn credit during training, and scores Faraday and both baselines at evaluation. The procedure is documented clearly, but it means "Faraday is better" holds against a ruler the authors built and also trained against.^[S1]

That ruler agrees with people at 0.19.

The authors ran a human study to calibrate it. The rubric judge reaches a Kendall tau of 0.19 against human rankings (a rank correlation, +1 identical and -1 reversed, not a percentage agreement), better than the baseline judge's 0.15, but two humans only agree at 0.30, while two draws of the judge agree at 0.66. Read it this way: the judge is consistent, and it is closer to human preference than the simpler one, yet "agrees with humans" is a low bar here, because replication quality is something people themselves struggle to agree on. The tasks were also chosen to maximise disagreement between the two judges. The authors say plainly that there is room for improvement.^[S1]

Replication is not discovery.

The paper is direct about this: replication is a stepping stone towards innovation, because the skills needed to fill in details a paper leaves vague may be the same ones needed to design your own experiments. But every replication task has a gold plot as its right answer, which is what makes it automatically scorable and trainable at scale, and exactly what open research lacks. The authors also state that Faraday's failures to replicate should not be read as problems with the original work, and note that some papers in the corpus were written by the authors or by people they know.^[S1]

Whether the split survives where there is no right answer.

Faraday already shows two hints: it leads on almost all of 20 counterfactual variants that swap the dataset or change the claim, and it wins five of eight full-scale replications with the compute unlocked. But both still have something to compare against. The real test is how you score anything when there is no gold plot: if research judgment can only be trained on tasks that have a right answer, that is where the ceiling sits.

Whether anyone outside Inherent evaluates on Replica.

The task set, the rubrics, the judge and the conclusion currently all come from one company, and the discussion section invites community engagement. One outside evaluation (or simply someone rerunning the baselines on the same tasks with a different judge) would move this record from a company scoring itself to a claim checked against something external.

Whether others also split judgment from execution.

This is the one worth tracking. If the pattern holds, AI4S system architecture shifts from one general model calling tools to a scientific judgment layer plus swappable specialist executors, with the executor free to be upgraded as frontier models move, and Faraday's tool swap without retraining already shows that is technically workable. The real signal will be a second and third group arriving at the same structure independently.