Member of Technical Staff

Benchmarking AI agents on pathogen genomic surveillance. A verifiable benchmark for practical decisions about taxonomy, variants, AMR, source tracking, anomaly detection, and engineered sequences in pathogen genomic surveillance workflows. Jul 09, 2026 LatchBio, Inc. introduce BioSecBench-Surveillance, a verifiable benchmark for testing whether AI agents can make the analytical decisions required in pathogen genomic surveillance. The benchmark contains 100 evaluations spanning seven task categories, six sample types, and both short- and long-read sequencing. Agents receive realistic sequencing data and sparse surveillance context, then must choose the right tools, references, thresholds, and analysis paths. Thanks for reading! Subscribe for free to receive new posts and support my work. Motivation. As sequencing volumes increase, genomic surveillance is increasingly limited by analysis. Public health workflows depend on bespoke and sometimes tacit choices about sequence references, databases, filters, normalization, and thresholds. AI agents are promising because they can inspect files, run tools, and iterate through workflows autonomously. But surveillance is a challenging problem. Agents must chain complex scientific and analysis decisions correctly from messy biological context. Main results. LatchBio, Inc. evaluated sixteen model-harness configurations across roughly 4,800 runs. Pass rates ranged from about 14% to 50%, with most frontier configurations clustered between 38% and 50%. Refusals varied sharply by harness and provider, from zero to nearly one-third of tasks. Performance varied most by task and read technology. Performance varied more by task type and sequencing technology than by sample type or assay. Most task categories landed between 35% and 50%, but anomaly detection fell to 20%, with genetic-engineering characterization next at 35%. Long-read datasets were also harder, scoring 26% versus 41% for short-read datasets. Sample type, nucleic-acid target, and assay type moved performance much less: clinical and isolate samples were handled best, wastewater was somewhat worse, DNA and RNA differed only modestly, and shotgun and targeted assays were nearly identical. Failures demonstrate gaps in scientific judgment. Agents usually found reasonable tools, but struggled with scientific judgment. The failures came from choices around how to invoke those tools in context, e.g. selecting the wrong reference, threshold, normalization method, or final interpretation of biological signal. The hardest tasks were open-ended judgement calls where the agent had to decide what mattered without being told what target to look for. Anomaly detection requires deciding whether a weak signal was real or background. Genetic-engineering characterization required deciding whether a sequence pattern reflected deliberate construction rather than native or homologous biology. AI Agents will be core infrastructure for genomic surveillance. LatchBio, Inc. is building toward a future where agents analyze surveillance data as it arrives, fast enough to shape an outbreak response while it still matters. Today's agents might not be reliable enough to do so, but by measuring their capabilities, LatchBio, Inc. get closer to this future. LatchBio, Inc. regularly update its benchmark family with new models: benchmarks.bio. This was a joint collaboration with Aclid, an automation platform for biosecurity and biosafety. LatchBio, Inc. is grateful to its scientific collaborators: Harmon Bhasin, Kevin Flyangolts, Dianzhuo Wang, Evan Seeyave, Arjun Banerjee, Amanda Darling, Joshua Stallings, David Stern, Shawn Higdon, Claire Duvallet, Bryan Tegomoh. Thanks for reading! Subscribe for free to receive new posts and support my work.

Indeed the year of agents in bio!Latch is at $15M RR, up 5x QoQ. Targeting $130M 2026.Closing a Series B now at $500M. https://t.co/554fQWv8rI— Arfur Rock (@ArfurRock) April 29, 2026