Paper & Dataset
GPT-4o reads 83,910 ecology papers and extracts 144,402 species interactions, at 89.5% precision and 70% recall
François Keck, Henry Broadbent and Florian Altermatt of the University of Zurich and Eawag built a workflow: keep the paragraphs that name two species and an interaction word, have GPT-4o extract who does what to whom into a fixed table, then link every name to the NCBI taxonomy and drop what does not link. Checked by hand on 500 random paragraphs, 89.5% of the interactions it reported were right and it found 70% of those present. The run cost about $582. One person did the labelling, and the network carries the literature's own bias.
●Paper + dataset + hand-checked sample
Current boundary: One annotator; three interactions in ten are missed; the check is whether a paper says it, not whether it holds in nature; the model is closed

89.5% / 70.0%reported interactions that are right / real ones found (precision / recall)
144,402interactions among more than 35,000 taxa, from paragraphs of 83,910 papers
Judged against one ecologist's hand annotation of 500 random paragraphs.
- Belief update
- A general-purpose model can do the reading; selecting, name-linking and checking still need purpose-built steps.
- Strongest validation
- 500 random paragraphs and 200 spread evenly across taxa, each annotated by hand; 98.3% of 593 taxonomy links correct.
- Key caveat
- One annotator; three interactions in ten missed; the network leans to the species the literature studies most.
01What happened
- Task
- Turn sentences in papers into records: species A, species B, what A does to B, and the class of interaction.
- Approach
- Keep paragraphs naming two species and an interaction word; GPT-4o fills a fixed table; names are linked to the NCBI taxonomy (taxonomic entity linking), unlinked ones dropped.
- Interaction
- Text only: paragraphs of open-access papers, read through OpenAI's API. Nothing is checked against nature.
- Result
- A network of 144,402 interactions; at species level, 46,467 among 18,589 species, parasitism the largest class.
6.16M → 83,910open-access papers → ecology papers given to the model
545,967paragraphs sent; one person took about 6 days to annotate 500
649,319 → 144,402raw records → interactions after filtering and merging
$582API cost of the whole extraction (batch pricing)
The paper was published online in Ecology and Evolution on 27 October 2026 (volume 16, issue 11, e74626), received 28 May and accepted 29 September, under CC BY 4.0. All three authors list the Department of Evolutionary Biology and Environmental Studies at the University of Zurich and the Department of Aquatic Ecology at Eawag, the Swiss Federal Institute of Aquatic Science and Technology; François Keck is corresponding author and led the conception, data, analysis and software, and Florian Altermatt the funding and supervision. Funding came from the University of Zurich and the Swiss National Science Foundation. Data and code are on GitHub (fkeck/gpt_interactions) and archived on Zenodo.[S1][S2][S3][S4]
How the corpus was narrowed
The start is 6,159,719 papers in the PubMed Central open-access collection. OpenAlex's topic classification picks out the 99,555 tagged with the subfield Ecology, which are split into 3,552,030 paragraphs. Paragraphs, not sentences, because an interaction often takes several sentences to state; not full articles, because of the irrelevant context and the cost. TaxoNERD, a model that recognises scientific and common names of organisms, then finds 8,160,227 names, and a regular-expression search looks for 43 interaction keywords and their variants. Paragraphs with at least two distinct names and one keyword stay: 545,967, from 83,910 papers. The authors say this step errs toward keeping too much, and concede that the keyword list may miss rare phrasings and that TaxoNERD performs unevenly across groups.[S1]
The prompt and the extraction
The model is GPT-4o (version 2024-05-13) at temperature zero with other settings at their defaults, sent in batches of 50,000 requests, none of which returned an error. The prompt asks for step-by-step reasoning, gives one worked example, stresses the points the model tends to miss with capitals and exclamation marks, states the output table's format in TypeScript and tells the model to be brief. The authors say the prompt was developed by reading outputs and revising, not by systematically testing variants. The output has four columns: the two species' names, the interaction as worded in the text, and its class from the list the mangal database uses. The extraction used 414,029,114 prompt tokens and 12,812,317 completion tokens, about US$581.6 at batch pricing as of February 2026.[S1]
Why 649,319 records became 144,402
The model returned 649,319 potential interactions involving 157,354 distinct names, of which 70,841 could be linked to the NCBI taxonomy. The authors kept only records whose two ends both linked, and merged records of the same class between the same pair of nodes. Two kinds of plain error were also removed: in 29 paragraphs the model repeated one word until it hit the output limit, and 4,272 records matched the prompt's worked example exactly, which the authors read as probable copying of the example and not evidence from the paragraph. The final network has 144,402 interactions from 112,647 paragraphs in 36,044 papers. The abstract prints its node count as 36,471 and the results section as 35,471.[S1]

Figure 1: the workflow. 6.16 million open-access papers pass a topic filter, paragraph splitting, name recognition and keyword matching, leaving 545,967 paragraphs; GPT-4o returns 649,319 interactions; taxonomic entity linking leaves 144,402. On the left, the two hand-assessed validation sets. [S1]
02What changed
Capability updateBottleneck updateDatabases were filled by hand or semi-automatically; dictionary tools coped poorly with papers' varied wording.
A general model reads 545,967 paragraphs for under $600; the limit is now what it is shown and what the literature covers.
The authors write in the introduction that an interaction involves two species and a directional relation, described with a large and sometimes ambiguous vocabulary, which limits dictionary-based methods; by their account this complexity had precluded any attempt at automated extraction of biological interactions on a large scale. They frame the paper as a methodological proof of principle. In the discussion they also state that the filtering after the model proved necessary: it removed the large number of false positives the model extracted, such as interactions involving irrelevant or ambiguous entities.[S1]
It misses more than it invents
In the 500 random paragraphs the annotator found 327 interactions. The workflow got 229 right, reported 27 wrongly and missed 98: precision 89.5% (95% confidence interval 83.8% to 94.2%), recall 70.0% (63.1% to 76.8%) and an F1 score of 78.6% (72.6% to 83.5%). Of the 500 paragraphs, 353 held no interaction at all, and in none of them did the workflow report one that clearly was not there. The authors say the misses come mostly from paragraphs where the model returned nothing, so the hard step is deciding whether an interaction is present, not naming or classifying it afterwards.[S1]
A second set of 200 paragraphs was built for balanced representation across taxonomic groups, with no taxon appearing more than once, to capture performance on common and rare taxa alike: precision 85.0% (78.4% to 91.2%), recall 88.5% (84.7% to 92.1%), F1 86.7%. The authors caution that this set was drawn only from paragraphs where the workflow had already returned something, so its recall is performance given detection and not an estimate of overall recall.[S1]
Most names that failed to link were not organisms
The authors checked 1,000 names by hand. 593 were linked to an NCBI taxonomy identifier, 98.31% of them correctly. Of the names that did not link, 63.10% were not organisms at all but genes, environments or biological processes (Glucan, for example); 12.30% were ambiguous (Algal grazer populations); 7.86% carried qualifiers that could not be safely removed (Small bullfrog tadpoles); 5.41% were common names or nouns derived from Latin names (Coleopteran); 5.41% were spelling errors in the source; 3.93% were gaps in the taxonomy itself; 1.97% were unclear. The papers described interactions in 14,767 distinct wordings.[S1]
03Why it matters
A group that keeps an interaction database can run this on its own literature for a few hundred dollars and spend its people on checking.
- Precision beats recall: what it reports is mostly right, but a missing interaction is not evidence of absence.
The authors say the standardised output can be browsed, queried or joined with existing interaction repositories and taxonomies. The repository's README carries GloBI (Global Biotic Interactions) indexing badges, and a query to GloBI's API for globi:fkeck/gpt_interactions on 17 November 2026 returned records from this dataset.[S1][S3][S5]
The authors state two problems with using a closed model for research. First, outputs are not fully reproducible: they re-ran 100 paragraphs 100 times each and found a mean agreement of 82% across 125 interactions at the finest taxonomic level, and 88% across the 22 at species level. They suggest treating an interaction that recurs across runs as a practical sign of confidence. Second, the model is a black box whose training data and method are not public, so validation has to come from checking outputs carefully. They see small open models, fine-tuned and run locally, as a promising direction that could also process papers under stricter licences, but did not try one here.[S1]
04Evidence status and boundaries
Evidence rung 3 / 5 · Checked against existing evidenceEvidence status
Proven
- 89.5% precision, 70.0% recall on 500 random paragraphs
- Nothing invented in the 353 paragraphs with no interaction
Boundaries
Suggested
- Taxon-balanced set: 85.0% / 88.5%, only where it found something
- Repeat runs agree 82% (125 interactions, 100 runs each)
Unknown
- How two annotators would differ: one person labelled it all
- Whether an extracted interaction holds in nature
- Whether another model returns the same network
- What the keyword and name filters left out
ClaimDropping names that do not link removes mostly things that were never organisms.
Main reasons names failed to link to the NCBI taxonomy (1,000 checked by hand; Table 1)
ReadingThe other rows are names too vague, misspelt or missing from the taxonomy to place.
How the evaluation was done.
Paragraph-level hand annotation is compared with the workflow's final output, after taxonomy linking and filtering, so the numbers describe the whole workflow and not the model alone. Two names count as the same species when they resolve to the same NCBI identifier; an interaction's direction has to match the text; the same pair of taxa in the same class within a paragraph counts once; negated interactions do not count; speculative statements were judged case by case and counted only when the paragraph presents the interaction as occurring. Confidence intervals come from 1,000 bootstrap resamples grouped by paragraph. One annotator with a background in ecology did all the labelling, about six days per set; with no second annotator, variation between annotators could not be estimated.[S1]
Whether the model answered from memory.
The prompt tells the model not to assume prior knowledge about the organisms, but the authors concede they cannot guarantee that information from pretraining did not shape individual outputs, most plausibly for common, widely documented interactions. Their argument is that this does not affect the evaluation: an extraction counted as correct only when the supplied paragraph supported it, and every record traces back to its source text. The uncertainty is about how the model reached an answer, not about whether a retained interaction has support in the text.[S1]
The literature's bias.
The species most represented in the results include humans (Homo sapiens), human pathogens (Escherichia coli, Pseudomonas aeruginosa, Staphylococcus aureus) and domestic species (Canis lupus, Sus scrofa, Bos taurus). The authors write that extraction can only show the literature's biases and does not correct them, which has to be kept in mind whenever results like these are read; they add that analysing the biases systematically is itself a way to study research effort and knowledge gaps. They state that the paper does not aim to deliver new ecological conclusions.[S1]

Figure 3: a is the main connected part of the species-level network; b counts interactions by class, parasitism the largest; c shows the 20 most common wordings and the classes they were put in, where one verb such as consume or feed on falls into several; d shows the 20 most connected species, led by Escherichia coli, Homo sapiens and Canis lupus. Grey is Others: symbiosis, amensalism, neutralism, commensalism, scavenging and unspecified. [S1]
05What I learned
- Transferable insight
- Precision owes much to the step after the model: linking names to a taxonomy removed its many false records.
- Design heuristic
- Link every extracted name to a reference database and drop the rest. Check the output for copies of the prompt's example.
- Reading heuristic
- For an LLM-extracted dataset, ask for precision and recall on a random sample, who labelled it, and how much was dropped.
- Watch next
- A second annotator on the same sample; the same corpus through an open model; a food web from it checked against field records.
Sources
Factual claims link to original announcements, project lists, trial records or journal papers where possible. Research plans are kept separate from completed results.
- S1A Scalable Workflow for Extracting Species Interactions From Ecological Literature Using Large Language ModelsEcology and Evolution · Keck, Broadbent & Altermatt · 2026.10.27 · Peer-reviewed paper · CC BY 4.0 · main text read, supporting information not read
- S2Crossref record: 10.1002/ece3.74626Crossref · 2026.10.28 · Publication record (online date and licence)
- S3fkeck/gpt_interactionsGitHub · 2026.11.17 · Code and data · README read
- S4Zenodo archive: 10.5281/zenodo.21397883Zenodo · 2026.11.17 · Data and code archive · link checked only
- S5GloBI API query: globi:fkeck/gpt_interactionsGlobal Biotic Interactions · 2026.11.17 · Database API · returned records from this dataset


