Level 1. Deviation
ChatGPT produced graphic violent images that shocked researchers
- Developer
- OpenAI
- Model
- Not established
- When it happened
- June 22, 2026
- When it was disclosed
- Not established
- How well it is stood up
- Not verified yet
- What caused it
- Not established
- Where it stands now
- Not established
What happened
Reported by malwarebytes.com. Listed here with a link to their report. Nobody on this side has checked it against that report, so it carries no level and this is not our account of what happened.
Our assessment
- Actual
- 1 of 5Deviation
- Potential
- 3 of 5Significant control incident
The system behaved contrary to its instructions, and it stayed contained. The potential level is what the behaviour demonstrated here could credibly have led to, judged from the same evidence.
This level rests on news reports. The developer has not confirmed it here.
This happened in a test, an evaluation or a demonstration. Nobody real was harmed, so the actual level stays at 1. What it showed counts toward the potential level.
Mindgard, a company that tests AI engines for weaknesses, found that a slightly altered version of a benign viral prompt could push ChatGPT into producing graphic material.
Why this level
- The behaviour repeated rather than stopping. Argues for a potential level of 3
- A safeguard was bypassed. Argues for a potential level of 2
What is established, and what is not
Confirmed
Nothing is confirmed yet.
Reported, not confirmed
The behaviour repeated rather than stopping
When OpenAI responded to that prompting hack to say that it had fixed the problem, Mindgard tweaked the same prompt and continued to be able to produce concerning output.
ChatGPT produced graphic violent images that shocked researchersA safeguard was bypassed
The content filters completely fell away, and the researcher was shown the dark side of what sits underneath.
Researchers bypassed ChatGPT's image filters with a 'restore this image' trick
Checked, and did not happen
Nothing here was checked and ruled out.
Not confirmed
Nothing is outstanding.
Where this came from
The original report
ChatGPT produced graphic violent images that shocked researchersA structured incident database · read September 18, 2026
AI Failure Index (CC-BY-4.0 data fields)Where we first saw it · read September 19, 2026
Researchers bypassed ChatGPT's image filters with a 'restore this image' trickWhere we first saw it
ChatGPT produced graphic violent images that shocked researchers
The reports above are other people’s. The levels are ours, worked out under our published rules. No organisation that reported this assigned our number, and none of it is legal advice.
Something here is wrong? Tell us. Send us a source we can open, and a person will read it.