How Does AI Read an Entire Chemical Reaction Route?

Do not ask one large model to do two different jobs

A direct approach asks one VLM to locate every molecule and infer every role. Scene understanding and precise coordinate regression are different tasks; a shifted box can corrupt both identity and relationship.^[S1]

RxnCaption uses an engineering split. MolYOLO first locates molecules and overlays indexed boxes. The VLM no longer predicts coordinates; it describes relationships among numbered objects: 1 and 2 are reactants, 3 is product, and nearby text is a condition.^[S1][S2]

The authors call this BIVP, Bounding-box Index as Visual Prompt. It looks like simply drawing boxes first, but represents a mature scientific-AI architecture: specialist perception followed by foundation-model semantic reasoning.^[S1]

Real reaction diagrams require production-like data

MolDet-33k contains 219,721 manually annotated molecular bounding boxes across 12,209 page-level and 21,155 figure/table-level images after cleaning. The updated RxnCaption data covers 15,128 reaction-parsing training images.^[S1][S2]

Scale is not the only change. A left-to-right single-step reaction is not comparable to a total-synthesis scheme with rows, branches, loops and tree-like dependencies. The data deliberately covers different layouts. As throughout this series, production distribution often matters more than benchmark size.^[S1]

If the paper already says 1a and 2b, why renumber them?

BIVP solves coordinates by modifying the image. RxnID's IdtVP instead uses native compound identifiers as visual-semantic handles. If the paper says 'compound 1a was converted to 2b,' the model can say 1a → 2b rather than remembering 'the third molecule in a blue box.'^[S3]

1a and 2b began as a publishing convention but become natural grounding tokens for a multimodal model. In the paper's zero-shot evaluation, Gemini 3 Pro with IdtVP reports 71.70% Hybrid Match F1 and 88.11% Soft Match F1 on RxnScribe-test, and 86.5% Soft Match F1 on scanned historical literature.^[S3]

These numbers do not mean 86.5% of reaction diagrams are completely correct. Soft and Hybrid Match are task-specific relationship metrics that tolerate some matching differences; they are not the rate of database records requiring no correction.^[S3]

Many real molecules have no identifier

Identifiers are useful but not free ground truth. An intermediate may be unnumbered, the same 7 may reappear in another figure, an identifier may sit far from its structure, or a compound may be named only in prose.^[S3]

RxnID adds Mid-Mapper: molecules with native identifiers are linked directly where possible, while unnumbered molecules receive virtual identifiers. At this point the task moves from reading one image toward understanding a document, and cross-page reference resolution becomes a new bottleneck.^[S3]

Why use verifiable rewards?

Reaction parsing often ends in JSON, but a reaction is not a string with one valid order. Swapping parallel reactants or independent reactions does not change the chemistry. Next-token loss can still penalize a semantically correct answer for a different serialization order.^[S3]

RxnID's Re³-DAPO raises reward to the reaction-set level, judging an output closer to Hybrid/Soft Match rather than demanding token-for-token identity. The interesting point is not merely the use of RL, but aligning the training objective with the scientific object.^[S3]

Verifiable rewards have boundaries. A cropped arrow, illegible condition or broken cross-page identifier can still yield beautiful JSON. Reward can test consistency with annotation rules; it cannot prove that a reaction occurred, its yield is reliable or its conditions reproduce.