Hugging Face Faces Deepfake Safety Questions After AI Forensics Study

Hugging Face is facing another difficult safety question, this time around AI models that can generate non-consensual intimate images. A new investigation by AI Forensics says the open AI platform is hosting tools and Spaces that can be used to create sexual deepfakes, with weak moderation around a problem that regulators in Europe and the UK are already trying to stamp out.

Engadget reported that AI Forensics tested popular Hugging Face Spaces in the image-editing category and found that seven of nine top models complied with a prompt to undress an AI-generated woman. The group also deployed its own test model, without producing images, and collected 1,081 user submissions in a week. It said 73 percent of the requests were sexual, 83 percent asked to undress the person in an uploaded image and 6.7 percent of the sexual requests targeted a minor.

The numbers are disturbing because they show how quickly an open model platform can become a distribution layer for abuse when safeguards are weak. Hugging Face has policies against non-consensual sexual images, but AI Forensics said it found very little output moderation across the Spaces it reviewed. The issue is not only whether bad actors exist. They obviously do. The issue is whether platforms make abuse easy, searchable and optimisable.

This lands at a sensitive moment for Hugging Face. The company was already part of the wider OpenAI incident and the debate over open AI tools. TechBooky’s report on Nvidia’s Open Secure AI Alliance looked at the argument that open models can help defenders during cyber incidents. That argument still matters. But the deepfake problem shows the other side of openness: when tools are easy to share, harmful use can spread quickly too.

The policy pressure is likely to rise. The EU has been moving toward stronger action against nudification apps, and the UK has also targeted intimate-image abuse. Platforms such as Hugging Face may soon face a harder expectation to detect, label, restrict or remove tools that are clearly designed for non-consensual sexual imagery. The old argument that a platform merely hosts code may not be enough when the harm is so direct and predictable.

There is also a technical challenge. Moderating model outputs is harder than moderating ordinary posts because abuse can happen inside an interactive tool, not just in uploaded content. A model may have legitimate image-editing uses while also being easy to push toward abuse. That means platforms need better review before models are hosted, better defaults, clearer reporting and probably stricter treatment of tools built around face or body manipulation.

Hugging Face has become essential infrastructure for the AI developer world. That makes its safety choices more important, not less. Open AI can support research, transparency and competition. But if platforms do not deal seriously with intimate-image abuse, they will hand regulators a strong reason to intervene more aggressively. This is one of those cases where safety is not a public-relations layer. It is the condition that lets an open ecosystem survive.