The online platform X has caused widespread anger after individuals employed its AI chatbot Grok to change photos of women by removing their clothing.
Numerous instances have been seen where the technology has been used to remove clothing from women to make them appear in swimwear without consent, as well as to insert them in sexualized contexts.
Developer Response and Legal Position
XAI, the organization responsible for Grok, remained silent to inquiries, except for an standard message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to outlaw nudification software. Under a proposed law, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"The platform could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their platforms, but did not confirm whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' prompts when they tag it in a message.
It is often used to give commentary or further background to other posters' comments, but people on X are also capable of edit an uploaded picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been criticised for enabling users to create pictures and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in creating inappropriate content of celebrities.
"X has been allowing the creation and sharing of these images for months without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of persons in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were required to take "suitable steps" to "reduce the danger" of users encountering illegal material on their services and to delete it swiftly when they learn of it.
