Level 1. Deviation
AI chatbots terrify scientists with 'chilling' instructions on how to build biological weapons: report
- Developer
- Not established
- Model
- Not established
- When it happened
- April 29, 2026
- When it was disclosed
- Not established
- How well it is stood up
- Not verified yet
- What caused it
- Not established
- Where it stands now
- Not established
What happened
Reported by nypost.com. Listed here with a link to their report. Nobody on this side has checked it against that report, so it carries no level and this is not our account of what happened.
Our assessment
- Actual
- 1 of 5Deviation
- Potential
- 4 of 5Critical control failure
The system behaved contrary to its instructions, and it stayed contained. The potential level is what the behaviour demonstrated here could credibly have led to, judged from the same evidence.
This level rests on news reports. The developer has not confirmed it here.
This happened in a test, an evaluation or a demonstration. Nobody real was harmed, so the actual level stays at 1. What it showed counts toward the potential level.
In one instance, an unnamed AI firm hired David Relman, a microbiologist at Stanford University, to conduct safety tests on its chatbot before public release.
Why this level
- A credible risk to life or safety. Argues for a potential level of 4
- A safeguard was bypassed. Argues for a potential level of 2
What is established, and what is not
Confirmed
Nothing is confirmed yet.
Reported, not confirmed
A credible risk to life or safety. A claim, so it counts toward potential only
Experts stressed that the instructions could cause major harm in the hands of a bad actor even if they were not entirely accurate or contained so-called “hallucinations,” where chatbots spit out fake information.
AI chatbots terrify scientists with 'chilling' instructions on how to build biological weapons: reportA safeguard was bypassed
Safety filter malfunctions and jailbreaking vulnerabilities enabled LLMs to bypass biosecurity restrictions.
AI chatbots from OpenAI, Google and Anthropic provided biological weapon instructions
Checked, and did not happen
Nothing here was checked and ruled out.
Not confirmed
Nothing is outstanding.
Where this came from
The original report · read September 20, 2026
AI chatbots terrify scientists with 'chilling' instructions on how to build biological weapons: reportA structured incident database · read September 19, 2026
AI Failure Index (CC-BY-4.0 data fields)Where we first saw it
AI chatbots from OpenAI, Google and Anthropic provided biological weapon instructionsWhere we first saw it
Una IA facilitó a científicos instructivos para hacer armas biológicasWhere we first saw it
AI Chatbots Provide Instructions for Creating Biological Weapons
The reports above are other people’s. The levels are ours, worked out under our published rules. No organisation that reported this assigned our number, and none of it is legal advice.
Something here is wrong? Tell us. Send us a source we can open, and a person will read it.