US law enforcement targets Grok for “pornography”: Musk’s xAI under investigation in California

Elon Musk’s AI chatbot Grok is facing escalating controversy over generating non-consensual explicit images, as U.S. law enforcement agencies have officially intervened. On Wednesday the 14th (ET), California Attorney General Rob Bonta announced the launch of an investigation into xAI and its AI model Grok, marking the transition of the controversy from global public condemnation to concrete legal action.

According to the official website of the California Department of Justice, the investigation focuses on Grok’s “large-scale production” of non-consensual deepfake intimate images used to harass women and girls online. The investigation will determine whether, and how, xAI has violated the law. California Governor Gavin Newsom also expressed support for the investigation, calling xAI’s actions “disgusting.”

This is the latest regulatory challenge Grok faces globally. Previously, governments in the EU, UK, France, Malaysia, Indonesia, India, and Brazil had taken action or issued warnings about the tool. In response, Musk said this week he was unaware of the situation and emphasized that Grok has been programmed to reject illegal requests.

This incident coincides with the U.S. Department of Defense announcing the deployment of Grok on both classified and unclassified networks, highlighting the urgency of AI technology governance amid regulatory and practical contradictions.

California Officially Launches Investigation, Accusing xAI of Abetting Image Abuse

California Attorney General Rob Bonta said in an official statement on Wednesday: “The past few weeks' reports about xAI producing and posting non-consensual explicit content online are shocking.” He emphasized that these materials, depicting women and children in nude or explicit sexual scenarios, have been used to harass victims online.

According to the Department of Justice announcement, the investigation targets xAI for “appearing to enable large-scale production of deepfake, non-consensual intimate images being used to harass women and girls online, including through the social media platform X.” In his statement, Bonta made it clear:

“We have zero tolerance for AI-generated non-consensual intimate images or child sexual abuse material.”

The announcement detailed Grok’s issues. It mentioned that, in the past few weeks, multiple news reports indicated that Grok users fetch ordinary photos of women and children from the internet and use Grok to depict them in suggestive or explicit scenarios, even “undressing” them—all without the subjects’ knowledge or consent.

The announcement states:

“Even more shocking, news reports say people are using Grok to alter children’s photos, putting them in revealing clothing and sexual scenes. It’s reported that xAI has generated realistic images of children engaging in sexual acts. These uses of Grok appear to be happening on a wide scale. According to one analysis, more than half of the 20,000 images generated by xAI between Christmas and New Year depicted scantily clad individuals, with some appearing to be children.”

xAI’s "Spicy Mode" Becomes a Marketing Highlight

The Department of Justice’s investigation focuses on xAI’s product design choices. The announcement noted that xAI developed Grok’s image generation model to include a so-called "spicy mode," which generates explicit content. “The company made this mode a marketing highlight, unsurprisingly resulting in a surge of non-consensual sexualized content.”

This issue emerged after Grok Imagine was launched last year. The AI image generator allows users to input text prompts to create videos and images, including adult content through "spicy mode." Problems worsened sharply late last month when Grok, hosted on the X platform, apparently started approving large numbers of user requests to modify images posted by others, with requests such as “make her wear a transparent bikini.”

Musk has positioned his chatbot as a competitor with fewer safeguards, and Grok’s images are publicly viewable and easily spread, amplifying the severity of the issue.

Restrictions Failing to Fully Work

Amid the controversy, Grok claims to have limited image generation and editing functions to paid subscribers. Last Friday, the chatbot responded to image editing requests with: “Image generation and editing are currently limited to paid subscribers. You can subscribe to unlock these features.” It only grants publicly visible request permissions to X users with a blue verification mark, who pay $8 a month for premium features.

However, according to the AP on Tuesday, free users can still use image editing tools by clicking the “edit image” button on each image displayed on the platform, instead of marking the request as coming from Grok. Independent Grok websites and apps were also still approving image editing requests as of Tuesday.

When asked for comment, Musk’s xAI company has repeatedly replied with automated messages claiming “mainstream media lies.” X previously said it would delete illegal content, including child sexual abuse material, permanently suspend accounts, “and cooperate with local government and law enforcement when needed.” Musk warned that "anyone using Grok to create illegal content will face the same consequences as those uploading illegal content."

Global Regulatory Pressure Intensifies

Prior to California's investigation, Grok had already faced regulatory action in multiple jurisdictions worldwide. The UK media regulator Ofcom launched an investigation on Monday, examining whether Grok violated obligations to protect people from illegal content. Indonesia temporarily blocked access to Grok last Saturday, while Malaysia said on Tuesday it would take legal action against X and xAI.

The EU’s top technology official warned X must resolve the problem, citing the Digital Services Act: “If not, we will not hesitate to fully use the DSA to protect EU citizens.” The Paris prosecutor’s office said its investigation into X is expanding to include sexually explicit deepfake content.

According to CCTV News, Musk responded on Wednesday saying he was unaware of Grok generating “minor nude images,” stressing Grok’s settings reject illegal requests and must comply with the laws of each country. Governments and women’s and children’s rights organizations in several countries have called on Apple and Google to remove the app from their stores.

Risk Reminder and DisclaimerThe market has risks; invest with caution. This article does not constitute personal investment advice and does not take into account any user’s specific investment objectives, financial situation, or needs. Users should consider whether any opinion, view, or conclusion in this article suits their particular circumstances. Investing accordingly is at your own risk.