VISTA trains no new model, only changes how it sees and remembers: Claude Opus 5.0 goes from 40.68 to 100 on ARC-AGI-3's public games

On 1 October 2026, Qiushi Han, Keya Hu and Linlu Qiu (co-first authors), Cathy Wu and Kaiming He of MIT posted the tech report 'VISTA: A Visual Harness for Reasoning in an Interactive World' (arXiv:2610.02200v1, CC BY 4.0). The report notes that an earlier version appeared as a blog post on 5 August 2026 (vista-research.github.io). It has not been peer reviewed; the code is public on GitHub. Kaiming He's own page gives his roles as associate professor with tenure in MIT's Department of EECS and, part-time, Distinguished Scientist at Google DeepMind; the paper lists MIT as the only affiliation.^[S1][S2][S3]

What ARC-AGI-3 tests

ARC-AGI-3 is the ARC Prize Foundation's suite of interactive visual games, each with several levels, whose rules and goals are not given; at each step the agent sees the current frame and the available actions and must infer what objects do, how states change and what the goal is, carrying what it learned into later levels. The paper evaluates on the 25 public games. The score is RHAE (Relative Human Action Efficiency): a cleared level earns 100% if it takes as many actions as first-time human players, up to 115% for fewer and partial credit for more; an unfinished level earns zero; later levels weigh more, and a game's score is capped by its weighted share of levels cleared. The human baseline comes from the organizers' study of 486 members of the public, with ten first-time players per game and, for each level, the upper-median action count among those who cleared it.^[S1]

Three parts: see, keep, revisit

VISTA starts from the observation that today's multimodal models usually encode an image once and discard the original; as the context fills, earlier observations are dropped or replaced with text summaries, and at encoding time the model cannot know which details it will need later. So it does three things. First, the model sees rendered images: the official 64×64 frame upscaled eightfold by nearest neighbour to a 512×512 PNG. Second, lossless visual memory: every frame the environment returns, including the intermediate animation frames an action produces, is archived as it was, indexed by turn and frame, and retrievable whatever the model's current context still holds. Third, inspection on demand: inspect fetches any frame from any turn, crops any rectangle and enlarges it, several frames at once for comparison; read pixels returns exact RGB values. The harness itself is mechanical and contains no trained model; only play advances the game, and every other tool is read-only.^[S1]

In each turn the model sees the current frame and available actions, may read and update two notes (GUIDE.md for rules that carry across levels, WORKING.md as a scratchpad for the current level), reasons in natural language, predicts the outcome, and calls play to take one action. When its context nears the limit it writes a handoff summary and resumes in a fresh context, with its notes, visual memory and action history intact. Every game uses the same interface and the same short prompt. Claude Opus 5.0 runs through the Claude Code CLI at extra-high reasoning effort and GPT-5.6 Sol through the Codex CLI at max; in Claude Code the tools are served by an MCP server with every built-in tool disabled, so the agent has no direct shell or file-system access.^[S1]

Scores and cost

Claude Opus 5.0 with VISTA scores an RHAE of 100.00 in 7,302 actions, 57.4% fewer than the human reference of 17,135; GPT-5.6 Sol with VISTA scores 99.00, and three repeats give 99.00, 98.90 and 99.23 (99.04 ± 0.17). In the paper's Table 1, several concurrent systems that build world models as programs also reach 95.35 to 100 (Tycho scores 100 with both models); VISTA's claim is that, to the authors' knowledge, it is the first to reach a perfect or near-perfect score without program synthesis. In the default setting (200K context), GPT-5.6 Sol uses 30.7 million tokens per game on average; with text-grid input it scores 96.75 with slightly more actions and 71.9 million tokens per game, because a 64×64 grid costs about 4,000 tokens and a 512×512 image about 308. Raising the context to 780K costs 105.7 million tokens per game without a higher score; cutting it to 100K lowers RHAE by only 0.4 and roughly halves the tokens.^[S1]

The paper recasts ARC-AGI-3 from a language problem into a vision problem. Earlier approaches mostly wrote each frame as rows of colour indices and had the model abstract the environment into programs it could execute and check; that works on ARC-AGI-3, but the paper argues that the more visually complex an environment becomes, the harder it is to describe its states, transitions and goals with explicit programs. VISTA has the model reason over the images directly, describing its understanding of the game in natural language rather than code.^[S1]

Against other agents, the change is memory. The game agents the paper reviews (Voyager, JARVIS-1, SIMA and others) typically keep a bounded visual context or select and compress past experience; VISTA keeps the whole visual history losslessly and lets the model return to any frame or region mid-reasoning. The paper calls this an explicit attention over the interaction history and gives two examples: after a board reset, the model pulls four frames from the previous 22 turns in one call to rebuild the map; unable to see three small blocks' orientation, it enlarges them about 13 times to find which face carries a purple marker.^[S1]

For anyone evaluating agents, the most direct lesson is that one model's score can change several-fold with its setup. GPT-5.6 Sol scores 13.33 in the official setup, 47.32 with images instead of text grids, 51.66 with looser limits, 65.82 in one continuous conversation, 70.05 with notes, 94.10 with lossless memory and inspection, and 99.00 with pixel reads. A claim that 'model X scores Z on benchmark Y' is unclear without the harness, the access route and the action limits.^[S1]

For AI for Science, the analogy is a lab's observation stream: microscope time-lapse, robot cameras, instrument screens. The usual practice turns frames into text descriptions or a few metrics and keeps only those; VISTA's results suggest that keeping raw observations, and letting the model go back to fetch, enlarge and compare frames when a question arises, may beat compressing once. That is this site's inference, not the paper's conclusion: the paper tests only games and puzzles, and the authors name embodied tasks and environments closer to the physical world as next steps. The cost counts too: the default setting uses 30.7 million tokens per game, so watching an instrument continuously cannot copy this usage as is; more realistic is cheap low-frequency monitoring that flags moments of doubt, with this kind of look-back called only for those moments.^[S1]

The baselines are counted differently.

Table 1's 40.68 (Opus 5.0, high effort) and 13.33 (GPT-5.6 Sol, max) are the organizers' published results; the 40.68 is the authors' calculation from the official per-game scores on the 25 public games. The official implementation calls the model through the API, uses text grids, keeps up to seven intermediate frames per action, holds a 175K context that drops the oldest turns once exceeded, and caps each section at five times the human action count and 12 hours of runtime; VISTA runs through subscription CLIs with 2,000 actions and 48 hours a game. Appendix A.2 says itself that the official result should be treated as an outside reference and that only configurations (i) to (vi) compare directly. Opus's VISTA score is also at extra-high effort, one step above the baseline's high.^[S1]

The ladder: the largest step is memory and inspection.

Appendix A.2 and Fig. 5 give the results of adding parts step by step (all with GPT-5.6 Sol at max effort and a 200K context). Images instead of text grids take it from 13.33 to 47.32, which the paper attributes partly to images saving tokens and leaving context for reasoning: seven frames cost about 28,000 tokens as text and about 2,200 as images. Looser action and time limits reach 51.66, and the authors saw no clear gain from loosening further; a continuous conversation using the model's own compaction to pass the context limit reaches 65.82; the two notes 70.05; lossless memory with inspect 94.10, with games cleared rising from 76% to 96% and total actions falling from 23,702 to 12,261; read pixels 99.00, every game cleared, in 9,126 actions. On the open-weight GLM-5.3 Flash, removing only inspect and read pixels drops the score from 66.93 to 13.27.^[S1]

Public games only, which the models may have seen.

The paper's limitations section states that the models were released after the public games, so the games may have been in their training data, and that evaluation on ARC-AGI-3's private games would be a stronger test of generalization. The August blog says likewise that the private set remains the real test. The report gives no private-set score. The authors also grant that the models' existing capabilities remain the key factor, with VISTA a simple way to elicit them.^[S1][S2]

The blog and the report differ in places.

The 5 August blog already reported 100 for Opus 5.0 with 56.0% fewer actions than humans (7,542), and 98.27 for GPT-5.6 Sol; its table gave the official minimal interface's Opus baseline as 30.16. The October report gives 7,302 actions, 57.4% fewer, 99.00 for GPT-5.6 Sol, and 40.68 for the Opus baseline (calculated from the official per-game scores on the 25 public games). The ablation, the open-weight model and the three other benchmarks appear only in the October report. Every number in this brief is the report's.^[S1][S2]

3D renderings and three more benchmarks.

Rendering the same games in 3D, VISTA still scores 84.12, though actions rise from 9,126 to 20,640. On three other benchmarks (all with GPT-5.6 Sol, against the same model's official minimal implementation): on GameWorld's 34 browser games and 170 tasks, a 63.3% success rate (official 40.0%, one novice human 55.3%); on AI GameStore's ten games, a geometric mean of 140.3 (human median = 100, official 47.3); on 39 static visual-tracking questions from BabyVision, 63.2% accuracy (official 41.0%, humans 92.3%). The last shows that zooming within a single image helps even with no interaction history.^[S1]

The harness is part of the evidence

The most transferable part of the paper is not a score but its method: on one model and one access route, add one part at a time and see what each step brings. That turns loose claims such as 'images beat text' or 'memory helps' into steps whose sizes can be compared. Any claim that an agent got more capable can be met with one question: did the model change, or the harness?

Compressing early loses details needed later

The paper's core argument is that when a model first sees an image it cannot know which details it will need later; discard the original and keep only an encoding or a summary, and there is no way back when they are needed. It is the same issue as this site's question about the ceiling set by information fidelity: every gate of measurement, extraction and encoding can silently lose information. VISTA is a controlled case: on the same model, adding a way back to the raw observations raised the score by 24 points.

Against the program-writing route

The program-writing systems in Table 1 (Schema, Retrodict, Tycho and others) also reach 95 to 100, so VISTA does not beat them on score; its point is that programs are not needed. The two routes are hard to separate on ARC-AGI-3, but the paper argues that programs get harder to write as scenes get more complex; if that holds, the see-and-revisit route is the more promising one for environments, such as a lab, that resist being written as programs. That has not been tested.