AI agents in biology pose real dual-use risks, but poorly calibrated safeguards are blocking legitimate research. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. 61 Routine tasks, legitimate analyses adapted from the published literature 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. The 107 evals were written by a team of 14 subject-matter experts to cover a range of domains: microbiology, virology, immunology, plant biology, synthetic biology, etc. Each task was annotated by biosafety level, biological agent class, request type and technical domain. Under direct framing, models refuse legitimate research more often than constructed threats Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and 1% to 62% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. For nearly every configuration tested, refusal rates were higher on Routine tasks than on Red-Team tasks. This gap increased with the human-assigned biosafety level of the task from BSL-1 to BSL-3 (the relatively small number of BSL-4 scenarios tested makes comparison at this level inconclusive). Routine and Red-Team refusal rates were tightly correlated across models (Pearson r = 0.91), pointing to a single underlying trigger: surface text. Routine tasks were generally rich in keywords likely to flag a safeguard ("pathogen", "immune evasion"). Red-Team tasks, though written to avoid obvious flag terms, also carried technical language with a dual-use character that a filter might recognize ("DNA assembly", "protein expression"). Agentic meta-evaluations indicate that extended reasoning may improve risk assessment To test whether agentic reasoning can identify complex biosecurity risk, we shifted from a direct framing to a meta-evaluation framing: instead of performing each task, the agent judges whether it should be accepted or refused. In tasks framed as a biosecurity meta-evaluation, the majority of refusals originated in the provider’s API filter, not the model’s own reasoning. For example in the GPT-5.5 x PI configuration, 60% of Routine tasks were refused. Two-thirds of those refusals (40% of all tasks) resulted from an API filter blocking the request before the model could decide. The model’s own refusal accounted for the remaining one-third (20% of all tasks). When agents were allowed to reason, they were occasionally able to recognize threats that were otherwise missed. For example, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks under meta evaluation, versus 13% in the direct framing. These estimates are preliminary, since the high rate of API refusal leaves only a small sample of genuine agentic decisions to evaluate. Refusal is a hard but soluble problem Better biosecurity metrics will help model developers to improve biosecurity performance and deploy agentic tools for biotech R&D with confidence. This is a first step. More progress in benchmark and agent engineering will follow.

FABLE RETURNS! | NEW META CLOUD | ANTHROPIC TALENT WAR x.com/i/broadcasts/1qxoNNrlm…

AI agents in biology pose real dual-use risks, but poorly calibrated safeguards are blocking legitimate research. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. 61 Routine tasks, legitimate analyses adapted from the published literature 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. The 107 evals were written by a team of 14 subject-matter experts to cover a range of domains: microbiology, virology, immunology, plant biology, synthetic biology, etc. Each task was annotated by biosafety level, biological agent class, request type and technical domain. Under direct framing, models refuse legitimate research more often than constructed threats Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and 1% to 62% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. For nearly every configuration tested, refusal rates were higher on Routine tasks than on Red-Team tasks. This gap increased with the human-assigned biosafety level of the task from BSL-1 to BSL-3 (the relatively small number of BSL-4 scenarios tested makes comparison at this level inconclusive). Routine and Red-Team refusal rates were tightly correlated across models (Pearson r = 0.91), pointing to a single underlying trigger: surface text. Routine tasks were generally rich in keywords likely to flag a safeguard ("pathogen", "immune evasion"). Red-Team tasks, though written to avoid obvious flag terms, also carried technical language with a dual-use character that a filter might recognize ("DNA assembly", "protein expression"). Agentic meta-evaluations indicate that extended reasoning may improve risk assessment To test whether agentic reasoning can identify complex biosecurity risk, we shifted from a direct framing to a meta-evaluation framing: instead of performing each task, the agent judges whether it should be accepted or refused. In tasks framed as a biosecurity meta-evaluation, the majority of refusals originated in the provider’s API filter, not the model’s own reasoning. For example in the GPT-5.5 x PI configuration, 60% of Routine tasks were refused. Two-thirds of those refusals (40% of all tasks) resulted from an API filter blocking the request before the model could decide. The model’s own refusal accounted for the remaining one-third (20% of all tasks). When agents were allowed to reason, they were occasionally able to recognize threats that were otherwise missed. For example, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks under meta evaluation, versus 13% in the direct framing. These estimates are preliminary, since the high rate of API refusal leaves only a small sample of genuine agentic decisions to evaluate. Refusal is a hard but soluble problem Better biosecurity metrics will help model developers to improve biosecurity performance and deploy agentic tools for biotech R&D with confidence. This is a first step. More progress in benchmark and agent engineering will follow.

AI agents in biology pose real dual-use risks, but poorly calibrated safeguards are blocking legitimate research. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. 61 Routine tasks, legitimate analyses adapted from the published literature 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. The 107 evals were written by a team of 14 subject-matter experts to cover a range of domains: microbiology, virology, immunology, plant biology, synthetic biology, etc. Each task was annotated by biosafety level, biological agent class, request type and technical domain. Under direct framing, models refuse legitimate research more often than constructed threats Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and 1% to 62% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. For nearly every configuration tested, refusal rates were higher on Routine tasks than on Red-Team tasks. This gap increased with the human-assigned biosafety level of the task from BSL-1 to BSL-3 (the relatively small number of BSL-4 scenarios tested makes comparison at this level inconclusive). Routine and Red-Team refusal rates were tightly correlated across models (Pearson r = 0.91), pointing to a single underlying trigger: surface text. Routine tasks were generally rich in keywords likely to flag a safeguard ("pathogen", "immune evasion"). Red-Team tasks, though written to avoid obvious flag terms, also carried technical language with a dual-use character that a filter might recognize ("DNA assembly", "protein expression"). Agentic meta-evaluations indicate that extended reasoning may improve risk assessment To test whether agentic reasoning can identify complex biosecurity risk, we shifted from a direct framing to a meta-evaluation framing: instead of performing each task, the agent judges whether it should be accepted or refused. In tasks framed as a biosecurity meta-evaluation, the majority of refusals originated in the provider’s API filter, not the model’s own reasoning. For example in the GPT-5.5 x PI configuration, 60% of Routine tasks were refused. Two-thirds of those refusals (40% of all tasks) resulted from an API filter blocking the request before the model could decide. The model’s own refusal accounted for the remaining one-third (20% of all tasks). When agents were allowed to reason, they were occasionally able to recognize threats that were otherwise missed. For example, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks under meta evaluation, versus 13% in the direct framing. These estimates are preliminary, since the high rate of API refusal leaves only a small sample of genuine agentic decisions to evaluate. Refusal is a hard but soluble problem Better biosecurity metrics will help model developers to improve biosecurity performance and deploy agentic tools for biotech R&D with confidence. This is a first step. More progress in benchmark and agent engineering will follow.

AI agents in biology pose real dual-use risks, but poorly calibrated safeguards are blocking legitimate research. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. 61 Routine tasks, legitimate analyses adapted from the published literature 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. The 107 evals were written by a team of 14 subject-matter experts to cover a range of domains: microbiology, virology, immunology, plant biology, synthetic biology, etc. Each task was annotated by biosafety level, biological agent class, request type and technical domain. Under direct framing, models refuse legitimate research more often than constructed threats Across 16 model-harness configurations, refusal rates ranged from 7% to 74% on Routine tasks and 1% to 62% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. For nearly every configuration tested, refusal rates were higher on Routine tasks than on Red-Team tasks. This gap increased with the human-assigned biosafety level of the task from BSL-1 to BSL-3 (the relatively small number of BSL-4 scenarios tested makes comparison at this level inconclusive). Routine and Red-Team refusal rates were tightly correlated across models (Pearson r = 0.91), pointing to a single underlying trigger: surface text. Routine tasks were generally rich in keywords likely to flag a safeguard ("pathogen", "immune evasion"). Red-Team tasks, though written to avoid obvious flag terms, also carried technical language with a dual-use character that a filter might recognize ("DNA assembly", "protein expression"). Agentic meta-evaluations indicate that extended reasoning may improve risk assessment To test whether agentic reasoning can identify complex biosecurity risk, we shifted from a direct framing to a meta-evaluation framing: instead of performing each task, the agent judges whether it should be accepted or refused. In tasks framed as a biosecurity meta-evaluation, the majority of refusals originated in the provider’s API filter, not the model’s own reasoning. For example in the GPT-5.5 x PI configuration, 60% of Routine tasks were refused. Two-thirds of those refusals (40% of all tasks) resulted from an API filter blocking the request before the model could decide. The model’s own refusal accounted for the remaining one-third (20% of all tasks). When agents were allowed to reason, they were occasionally able to recognize threats that were otherwise missed. For example, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks under meta evaluation, versus 13% in the direct framing. These estimates are preliminary, since the high rate of API refusal leaves only a small sample of genuine agentic decisions to evaluate. Refusal is a hard but soluble problem Better biosecurity metrics will help model developers to improve biosecurity performance and deploy agentic tools for biotech R&D with confidence. This is a first step. More progress in benchmark and agent engineering will follow.

Wake up everyone, Anthropic is making MEDICINES! Improving the human condition is the most valuable opportunity that has ever existed and will ever exist. The industry is about to reflect that. Cool to see @ManifoldBio @LatchBio @boltz_bio here - models, data, workflows are key

We’re launching the Latch MCP and announcing its availability within Claude Science, Anthropic’s new AI workbench for scientists. AI for biology requires agent-native infrastructure: systems where agents can store, process, and visualize large molecular datasets from the interfaces scientists already use. Biological analysis workflows often require substantial compute. Retries or incorrect long-running tool calls can quickly inflate workflow time and cost, especially when analyses take hours or days to complete. These tools should also be curated with appropriate parameters and agent-readable documentation, ideally provided by the original assay developer, to support correct use across many complex scientific contexts. At Latch, we’ve seen customers of our Solution Provider partners, including TakaraBio, Vizgen, and AtlasXOmics, use both the Latch Agent and external harnesses like Claude Code and Cursor to accelerate analysis of their data. The Latch MCP is a remote MCP server that securely connects agent harnesses to the Latch platform, giving agents access to verified bioinformatics tools built and maintained by kit and instrument providers. Agents can navigate data on Latch and launch existing deployments of validated bioinformatics workflows to analyze that data.

Introducing Claude Science, a new app designed with every stage of research in mind. Artifacts traced to their code, environments managed on demand, and 60 optional scientific databases that you can connect. Available now in beta.

We are excited to announce that LatchBio has acquired TwentyTwo, a YC S25 team building AI infrastructure for biosecurity. As AI makes biology easier to engineer, the safeguards around these models stop being an afterthought and become core infrastructure. Security has to improve as fast as capabilities to allow science to progress. Within our lifetimes, AI will put the ability to engineer a pandemic within reach of a single bad actor. What once took a nation-state and years of work is collapsing toward an undergraduate and a few weeks in the lab. That world is arriving faster than most AI labs are prepared for. However, in the right hands, these models will accelerate science at a pace never seen before. Scientists will use these models to cure diseases that families have fought for generations and catch outbreaks before they spread. We cannot wall them off from the tools that augment their work. The problem is that the line between the two is hard to draw. Studying SARS-CoV-2 or Ebola to prepare for the next pandemic is very similar to what a bad actor would do to cause one. TwentyTwo builds intelligence defense systems that distinguish this line and apply safeguards or grant capabilities accordingly. Harmon Bhasin, Evan Seeyave, and John Wang are exceptional researchers who care deeply about building the future of life sciences responsibly. The three of them will join Latch as Members of Technical Staff and lead Latch Biosecurity. I am grateful to be working with them. Expect many more announcements from them over the coming days. We are hiring biosecurity engineers. If this is the kind of work you want to do, reach out.