Generated by Codex with GPT-5

What happened

Anthropic’s official research blog published Paving the way for agents in biology, a June 8, 2026 post about why scientific agents fail on apparently simple biological data-retrieval tasks and how a deterministic retrieval layer can make those workflows reliable.

The post is interesting because it treats scientific-agent progress as an infrastructure problem, not only a model-capability problem. Biological databases were built for expert humans moving through web interfaces, file formats, accession identifiers, metadata conventions, and one-off scripts. A model can understand a scientist’s intent and still fail if the data path requires hidden domain knowledge, browser-only filters, inconsistent schemas, incomplete pagination, or reconciliation across multiple backends.

The concrete case study is viral sequence retrieval from NCBI Virus, a database used in surveillance, diagnostic assay design, outbreak analysis, and training-data construction for biological models. Anthropic and collaborators asked scientific research agents, including Claude, Biomni OSS, Edison Analysis, and GPT systems, to retrieve sequence datasets from NCBI Virus under realistic constraints. The result was not a clean story of frontier models replacing brittle tooling. Even strong models produced plausible but incomplete or inconsistent datasets, and small retrieval errors could change downstream biological conclusions.

The central engineering lesson is that agents need boring, inspectable execution layers for high-stakes scientific work. Anthropic’s collaborators built gget virus, a deterministic retrieval interface for viral sequence queries, and agent accuracy rose close to perfect when agents could call it. That does not make the model irrelevant. It changes the model’s job: instead of rediscovering the retrieval workflow on every run, the model can express intent, call a reliable tool, inspect logs and outputs, and spend its reasoning budget on the scientific question.

Why agents failed

The task sounds simpler than it is. A virologist may want all sequences for a particular virus, host, geography, collection date range, genome length, ambiguity threshold, completeness status, and protein content. In a browser, an experienced user can apply filters through the NCBI Virus interface. Programmatically, the same query may require stitching together multiple APIs, paginating large result sets, reconciling identifiers, downloading metadata and sequence records, applying local filters, and knowing which fields or conventions carry the biological meaning.

Anthropic frames this as a mismatch between agent needs and human-oriented scientific infrastructure. Software engineering workflows are comparatively agent-friendly: code is textual, versioned, testable, and often wrapped in command-line tools or APIs. Biology workflows often depend on heterogeneous databases, implicit metadata conventions, and outputs where correctness is hard to verify until much later. If an agent retrieves the wrong genome build, mixes RefSeq and GenBank records unexpectedly, treats partial genomes as complete, or misses records because a filter’s semantics are not documented, the mistake may propagate into an analysis that still looks professional.

To measure the problem, the team built VirBench: 120 viral sequence queries across 40 pathogens, with manually verified ground-truth counts. The queries reflect real tasks in viral surveillance, diagnostic assay design, and protein-model training-data construction. They include compound filters such as taxonomic ID, host, region, collection date, sequence length, ambiguity count, protein content, and exclusion of lab-passaged samples.

The results show why ordinary “the agent got an answer” evaluation is insufficient. Across tested systems, mean accuracy ranged from 16.9 percent to 91.3 percent. Newer frontier models were better, but even the strongest systems did not reliably reach the standard needed for dataset construction. Worse, repeated attempts with the same prompt could produce materially different answers. In one Ebolavirus query where the expected count was 266 sequences, Sonnet 4 returned 106 sequences on one run, then 15, then 5.

The downstream examples are the strongest part of the post. A sequence set built manually from NCBI Virus recovered a 2014 time to the most recent common ancestor for the West African Ebolavirus outbreak, consistent with prior reports. Sequence sets assembled by the agent could make the same outbreak look incomplete, shifted, or even rooted in 1922. A therapeutic-epitope analysis showed the same pattern: different retrieval runs produced different impressions of which residues varied in antibody target regions. The agent failure was not just an off-by-some-count problem. It changed the biological story.

The retrieval layer

gget virus is the post’s main systems artifact. It is not presented as a smarter model. It is a deterministic layer that translates a complex, browser-heavy NCBI Virus retrieval workflow into a callable, reproducible interface for humans and agents.

The implementation has to reflect the shape of the underlying data platform. NCBI Virus is a portal over several synchronized sequence resources and APIs, including REST, Datasets, and E-utilities. Some filters can be applied directly through existing endpoints. Others need local checks because the web interface exposes filtering behavior that is not available as one clean programmatic operation. When a query depends on information stored in GenBank records, such as whether a sequence contains a specific viral protein, gget virus retrieves those records and uses them during filtering.

Two engineering choices matter in particular. First, the tool handles batching and pagination so that large result sets, such as SARS-CoV-2 or Influenza A, are retrieved comprehensively instead of being silently truncated. That is exactly the kind of failure a model can miss because partial results are still syntactically valid and biologically plausible. Second, the tool emits standardized outputs and detailed logs, so the result is not just a count or a file. It is an auditable retrieval trace.

When agents were given access to gget virus, accuracy rose above 90 percent for all tested agents and peaked at 99.7 percent for GPT-5.5. Run-to-run variability largely disappeared, and differences between model families narrowed. That result is the heart of the post: adding the right deterministic tool mattered more than choosing the newest model for this class of task.

This is a useful corrective to a common agent-design instinct. The tempting approach is to keep giving the model more browser access, longer context, more documents, and more chances to try again. Anthropic’s result points toward a different path. For repeated, high-consequence data operations, teams should identify the fragile procedure, encode it in a deterministic interface, and make the model call that interface. The agent should not have to infer pagination, schema quirks, identifier normalization, and domain-specific filtering rules from scratch every time.

Why it matters

The broader takeaway is that scientific agents need agent-readable scientific infrastructure. Benchmarks often focus on reasoning quality: can the model interpret the request, choose a method, write code, and explain the result? This post shows that the data-access substrate can dominate the outcome. A model that understands the biology may still fail if the database hides critical semantics behind browser controls or inconsistent metadata.

The lesson generalizes beyond virology. Many scientific and enterprise systems were designed around expert operators, not autonomous tool users. The most valuable agent work often begins with data assembly: pull the right cohort, retrieve the right documents, map the right identifiers, choose the right version, and preserve provenance. If that first step is nondeterministic, every later step inherits uncertainty. Stronger reasoning cannot fully compensate for unreliable inputs.

The post also suggests a practical evaluation pattern. VirBench does not grade whether a model sounds scientifically competent. It asks whether the final retrieved dataset matches a verified ground truth across realistic filters and repeated runs. That makes reproducibility part of the score. For production agents, especially in scientific or regulated domains, evals should measure stable execution over real data operations, not just one-shot answer quality.

There is also an operations lesson. A deterministic retrieval layer should expose logs, provenance, and standardized outputs because scientific users need to inspect how an answer was produced. That is different from asking an LLM to explain itself after the fact. The retrieval system itself has to make its behavior observable: which APIs were called, which filters were applied remotely, which were applied locally, which records were excluded, and how identifiers were reconciled.

Anthropic is careful not to claim that harnesses will always be necessary. Future models may become better at navigating fragmented biological databases directly. But even if a model can fight through the workflow, it may be slower, more expensive, harder to audit, and less reproducible than calling a purpose-built tool. The post’s most durable argument is not that agents are weak. It is that routine scientific data access should not depend on model improvisation.

Takeaway

Anthropic’s post is a strong example of research-engineering work around agents: the model failure is diagnosed through a benchmark, the failure mode is traced to infrastructure, and the fix is a deterministic interface that narrows the model’s responsibility.

For teams building agents over scientific, data, or enterprise systems, the pattern is clear. Put stable retrieval and transformation logic behind callable tools. Make the tools preserve the domain semantics that users rely on in the browser or in expert scripts. Return outputs that are machine-readable and human-auditable. Then evaluate the full system on repeated real workflows with ground-truth answers.

The important shift is from “can an agent operate this messy environment?” to “what would the environment look like if agents were first-class users?” In biology, that means databases with clear APIs, standardized metadata, complete pagination, explicit filter semantics, provenance, and deterministic connectors. In any domain, it means treating the surrounding infrastructure as part of the agent’s intelligence.