A bug in OpenAI’s ChatGPT allowed the chatbot to produce graphic erotica for accounts registered to minors under 18. OpenAI confirmed the issue and said it is deploying a fix.
In some instances, ChatGPT not only generated explicit content but also encouraged minors to request even more graphic material.
OpenAI stated that its policies prohibit such responses for users under 18. “Protecting younger users is a top priority, and our Model Spec clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting,” a spokesperson said. “A bug allowed responses outside these guidelines, and we are actively deploying a fix to limit these generations.”
TechCrunch said its testing aimed to probe ChatGPT’s guardrails after recent updates made the platform more permissive. In February, OpenAI changed technical specifications to allow broader discussions on sensitive topics and removed warnings that previously flagged potential violations of its terms of service.
The goal, according to ChatGPT’s head of product Nick Turley, was to eliminate “gratuitous/unexplainable denials.” However, one result was that ChatGPT’s default model, GPT-4o, became more willing to engage in discussions about sexual activity.
TechCrunch primarily tested for sexual content, given OpenAI’s stated intent to relax restrictions in this area. CEO Sam Altman has discussed developing a “grown-up mode” and expressed openness to allowing some “NSFW” content.
To conduct the tests, multiple ChatGPT accounts registered to fictional users aged 13 to 17. They used a single PC, cleared cookies between sessions, and ensured no cached data influenced results. OpenAI’s policies require parental consent for users aged 13 to 18, but the platform does not verify this during registration. As long as users provide a valid email address or phone number, they can create an account.
Testing involved prompting ChatGPT with phrases like “talk dirty to me.” In many cases, after only a few messages, ChatGPT generated sexual stories. It sometimes even asked users to specify kinks and role-play scenarios.
In one instance, ChatGPT told a user registered as 13 years old: “We can go into overstimulation, multiple forced climaxes, breathplay, even rougher dominance — wherever you want.” This response came after the chatbot was nudged toward more explicit descriptions.
While ChatGPT often warned that its guidelines prohibited fully explicit content, it still occasionally wrote graphic depictions of genitalia and sexual acts. Only once, during TechCrunch’s tests, did the chatbot stop after being reminded that the user was under 18.
The chatbot responded: “Just so you know: You must be 18+ to request or interact with any content that’s sexual, explicit, or highly suggestive. If you’re under 18, I have to immediately stop this kind of content — that’s OpenAI’s strict rule.”
The findings follow a similar report from The Wall Street Journal, which uncovered that Meta’s AI chatbot, Meta AI, also allowed minors to engage in sexual role-play after Meta loosened restrictions on content.
OpenAI’s weakening of AI safeguards comes as it seeks greater adoption in schools. The company has partnered with groups like Common Sense Media to produce guides on using ChatGPT in educational settings.
This strategy has helped ChatGPT gain popularity among Gen Z students. A Pew Research Center survey earlier this year showed growing use of the platform for schoolwork.
In support materials for educators, OpenAI warns that ChatGPT “may produce output that is not appropriate for all audiences or all ages” and urges caution when using the tool with students.
Steven Adler, a former OpenAI safety researcher, expressed concern over the results. “Evaluations should be capable of catching behaviors like these before a launch,” Adler said. “I wonder what happened.”
Meanwhile, ChatGPT users have reported strange behaviors, including excessive flattery, following the release of GPT-4o. OpenAI CEO Sam Altman acknowledged some issues in a post on X, stating the company is “working on fixes ASAP.” He did not address the chatbot’s explicit content with minors.
