Grok produces sexualized photos of women and minors for users on X – a legal scholar explains why it’s happening and what can be done

Since the end of December, 2025, X’s artificial intelligence chatbot, Grok, has responded to many users’ requests to undress real people by turning photos of the people into sexually explicit material. After people began using the feature, the social platform company faced global scrutiny for enabling users to generate nonconsensual sexually explicit depictions of real people.

The Grok account has posted thousands of “nudified” and sexually suggestive images per hour. Even more disturbing, Grok has generated sexualized images and sexually explicit material of minors.

X’s response: Blame the platform’s users, not us. The company issued a statement on Jan. 3, 2026, saying that “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.” It’s not clear what action, if any, X has taken against any users.

As a legal scholar who studies the intersection of law and emerging technologies, I see this flurry of nonconsensual imagery as a predictable outcome of the combination of X’s lax content moderation policies and the accessibility of powerful generative AI tools.

Targeting users

The rapid rise in generative AI has led to countless websites, apps and chatbots that allow users to produce sexually explicit material, including “nudification” of real children’s images. But these apps and websites are not as widely known or used as any of the major social media platforms, like X.

State legislatures and Congress were somewhat quick to respond. In May 2025, Congress enacted the Take It Down Act, which makes it a criminal offense to publish nonconsensual sexually explicit material of real people. The Take It Down Act criminalizes both the nonconsensual publication of “intimate visual depictions” of identifiable people and AI- or otherwise computer-generated depictions of identifiable people.

Those criminal provisions apply only to any individuals who post the sexually explicit content, not to the platforms that distribute the content, such as social media websites.

Other provisions of the Take It Down Act, however, require platforms to establish a process for the people depicted to request the removal of the imagery. Once a “Take It Down Request” is submitted, a platform must remove the sexually explicit depiction within 48 hours. But these requirements do not take effect until May 19, 2026.

Problems with platforms

Meanwhile, user requests to take down the sexually explicit imagery produced by Grok have apparently gone unanswered. Even the mother of one of Elon Musk’s children, Ashley St. Clair, has not been able to get X to remove the fake sexualized images of her that Musk’s fans produced using Grok. The Guardian reports that St. Clair said her “complaints to X staff went nowhere.”

This does not surprise me because Musk gutted then-Twitter’s Trust and Safety advisory group shortly after he acquired the platform and fired 80% of the company’s engineers dedicated to trust and safety. Trust and safety teams are typically responsible for content moderation and initiatives to prevent abuse at tech companies.

Publicly, it appears that Musk has dismissed the seriousness of the situation. Musk has reportedly posted laugh-cry emojis in response to some of the images, and X responded to a Reuters reporter’s inquiry with the auto-reply “Legacy Media Lies.”

Limits of lawsuits

Civil lawsuits like the one filed by the parents of Adam Raine, a teenager who committed suicide in April 2025 after interacting with OpenAI’s ChatGPT, are one way to hold platforms accountable. But lawsuits face an uphill battle in the United States given Section 230 of the Communications Decency Act, which generally immunizes social media platforms from legal liability for the content that users post on their platforms.

Supreme Court Justice Clarence Thomas and many legal scholars, however, have argued that Section 230 has been applied too broadly by courts. I generally agree that Section 230 immunity needs to be narrowed because immunizing tech companies and their platforms for their deliberate design choices – how their software is built, how the software operates and what the software produces – falls outside the scope of Section 230’s protections.

In this case, X has either knowingly or negligently failed to deploy safeguards and controls in Grok to prevent users from generating sexually explicit imagery of identifiable people. Even if Musk and X believe that users should have the ability to generate sexually explicit images of adults using Grok, I believe that in no world should X escape accountability for building a product that generates sexually explicit material of real-life children.

Regulatory guardrails

If people cannot hold platforms like X accountable via civil lawsuits, then it falls to the federal government to investigate and regulate them. The Federal Trade Commission, the Department of Justice or Congress, for example, could investigate X for Grok’s generation of nonconsensual sexually explicit material. But with Musk’s renewed political ties to President Donald Trump, I do not expect any serious investigations and accountability anytime soon.

For now, international regulators have launched investigations against X and Grok. French authorities have commenced investigations into “the proliferation of sexually explicit deepfakes” from Grok, and the Irish Council for Civil Liberties and Digital Rights Ireland have strongly urged Ireland’s national police to investigate the “mass undressing spree.” The U.K. regulatory agency Office of Communications said it is investigating the matter, and regulators in the European Commission, India and Malaysia are reportedly investigating X as well.

In the United States, perhaps the best course of action until the Take It Down Act goes into effect in May is for people to demand action from elected officials.

Wayne Unger does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.