Level 3. Significant control incident
Discord admits AI moderation bug wrongfully banned users over harmless images
- Developer
- Not established
- Model
- Not established
- When it happened
- July 7, 2026
- When it was disclosed
- Not established
- How well it is stood up
- Not verified yet
- What caused it
- Not established
- Where it stands now
- Not established
What happened
Reported by techcrunch.com. Listed here with a link to their report. Nobody on this side has checked it against that report, so it carries no level and this is not our account of what happened.
Our assessment
- Actual
- 3 of 5Significant control incident
- Potential
- 3 of 5Significant control incident
An important technical, security or permission boundary was crossed, or there were real consequences. The potential level is what the behaviour demonstrated here could credibly have led to, judged from the same evidence.
This level rests on news reports. The developer has not confirmed it here.
Why this level
- An external system was affected. Argues for level 3
- A safeguard was bypassed. Argues for level 2
- The system acted without authorisation. Argues for level 2
- The behaviour repeated rather than stopping. Argues for a potential level of 3
What is established, and what is not
Confirmed
Nothing is confirmed yet.
Reported, not confirmed
An external system was affected
Discord has acknowledged that a bug in its AI moderation system mistakenly banned more than 8,000 users over the past two months, after harmless images—including spreadsheets, chessboards, game textures, as well as white and gray transparent backgrounds—were incorrectly flagged as harmful content.
Discord admits AI moderation bug wrongfully banned users over harmless imagesThe behaviour repeated rather than stopping
The company confirmed that the issue had been affecting accounts since May, with an additional 200 users banned over the weekend before its team identified and fixed the problem.
Discord admits AI moderation bug wrongfully banned users over harmless imagesA safeguard was bypassed
The intended workflow routed flagged content to a human Trust and Safety reviewer before any ban, but the bug bypassed that step and issued instant bans.
Discord's AI moderation wrongly banned more than 8,000 users after a bug skipped human reviewThe system acted without authorisation
A human moderator reviews the content, but a bug caused the system to immediately ban affected accounts.
Discord admits AI moderation bug wrongfully banned users over harmless images
Checked, and did not happen
Nothing here was checked and ruled out.
Not confirmed
Nothing is outstanding.
Where this came from
The original report
Discord admits AI moderation bug wrongfully banned users over harmless imagesA structured incident database · read September 18, 2026
AI Failure Index (CC-BY-4.0 data fields)Where we first saw it · read September 19, 2026
Discord's AI moderation wrongly banned more than 8,000 users after a bug skipped human reviewWhere we first saw it
Discord admits AI moderation bug wrongfully banned users over harmless imagesWhere we first saw it
Discord's AI moderation bug wrongfully banned 8,000 users over two months
The reports above are other people’s. The levels are ours, worked out under our published rules. No organisation that reported this assigned our number, and none of it is legal advice.
Something here is wrong? Tell us. Send us a source we can open, and a person will read it.