The online platform X has caused significant outrage after individuals used its AI chatbot Grok to alter pictures of females by removing their garments.
Numerous examples have emerged where the technology has been employed to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to put them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the organization behind Grok, offered no comment to questions, other than an standard message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to outlaw image stripping software. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a jail term and significant fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "assess the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' prompts when they mention it in a message.
It is commonly used to give commentary or further background to other posters' comments, but people on X are also can edit an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The system has been condemned for permitting users to create images and clips with explicit content and sexualised content, and it was earlier accused of generating graphic content of public figures.
"The platform has been permitting the production and distribution of these pictures for a long time without taking any measures and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said services such as X were obligated to take "necessary steps" to "minimize the risk" of users coming across unlawful content on their services and to take down it promptly when they learn of it.
