Oh deer AnonymousCountry code: blank.gif, country type: blank, valid: No.20750
(Bloomberg) -- Elon Musk’s artificial intelligence chatbot Grok has created sexualized images of people including minors on the social media platform X in response to user prompts in recent days, drawing rebuke from officials such as the French government.
Grok created and published images of minors in minimal clothing, in apparent violation of its own acceptable use policy, which prohibits the sexualization of children. Some of the offending images were later taken down.
Most Read from Bloomberg
New York City’s Hunts Point Produce Market to Be Rebuilt
Sinkholes and Quake-Hit Roads Expose Japan’s Infrastructure Emergency
The French government accused Grok on Friday of generating “clearly illegal” sexual content on X without people’s consent, flagging the matter as potentially violating the European Union’s Digital Services Act. The regulation requires large platforms to mitigate the risk of illegal content spreading, the country said in a statement.
Representatives for xAI, the company that develops Grok and runs X, didn’t respond to requests for comment. The chatbot Grok generated a post on X in response to users’ questions on Friday that it had identified “lapses in safeguards” that were being “urgently” fixed. It echoed xAI employee Parsa Tajik who earlier posted that “the team is looking into further tightening” its guardrails.
X users can interact with Grok directly on the platform by tagging its account in posts and prompting the chatbot to respond. Grok generates text and images that appear as posts on the social network.
The rise of AI tools that can generate realistic pictures of undressed minors highlights the challenges of content moderation and safety systems built into image-generating large language models. Tools that claim to have guardrails can be manipulated, allowing for the proliferation of material that has alarmed child safety advocates. The Internet Watch Foundation, a nonprofit that identifies child sexual abuse material online, reported a 400% increase in such AI-generated imagery in the first six months of 2025.
“AI products must be tested rigorously before they go to market to ensure they do not have the capability to generate this material,” the foundation’s Chief Executive Officer Kerry Smith said in a statement Friday.
XAI has positioned Grok as more permissive than other mainstream AI models, and last summer introduced a feature called “Spicy Mode” that permits partial adult nudity and sexually suggestive content. The service prohibits pornography involving real people’s likenesses and sexual content involving minors, which is illegal to create or distribute.
Still, users have prompted Grok to digitally remove clothing from photos — mostly of women — so the subjects appeared to be wearing only underwear or bikinis. This prompted India’s IT ministry to demand a comprehensive review of Grok’s safety features, according to a copy of the complaint posted Friday by Indian Member of Parliament Priyanka Chaturvedi.
In its statement, the French government said it had reported the sexual content created by Grok to the country’s public prosecutor to get them immediately removed.
As AI image generation has become more popular, the leading companies behind the tools have released policies about the depictions of minors. OpenAI prohibits any material that sexualizes children under 18 and bans any users who attempt to generate or upload such material. Google has similar policies that forbid “any modified imagery of an identifiable minor engaging in sexually explicit conduct.” Black Forest Labs, an AI startup that has previously worked with X, is among the many generative AI companies that say they filter child abuse and exploitation imagery from the datasets used to train AI models.
In 2023, researchers found that a massive public dataset used to build popular AI image-generators contained at least 1,008 instances of child sexual abuse material.
Many companies have faced criticism for failing to protect minors from sexual content. Meta Platforms Inc. said over the summer that it was updating its policies after a Reuters report found that the company’s internal rules let its chatbot hold romantic and sensual conversations with children.
