Elon Musk's online network X has sparked significant backlash after individuals used its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Numerous instances have surfaced where the system has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to put them in sexualized situations.
Developer Response and Regulatory Position
XAI, the firm behind Grok, remained silent to inquiries, except for an standard message stating "legacy media lies".
A government representative stated that legislation is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who provided such technology would "receive a jail term and significant fines".
"X or Grok is capable of halting these forms of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the potential" of people accessing illegal content on their networks, but did not specify whether it was presently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which answers X users' prompts when they mention it in a message.
It is often used to provide reactions or additional context to other posters' remarks, but people on X are also capable of alter an uploaded image through its AI image editing function.
Background of Controversy and Policy
The system has been condemned for enabling users to create images and videos with explicit content and sexually suggestive material, and it was in the past accused of generating sexually explicit content of celebrities.
"X has been enabling the generation and spread of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of individuals in a graphic way".
In a statement, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or illegal content" and stated this included sexual deepfakes created with AI.
It said networks such as X were obligated to take "suitable actions" to "lessen the risk" of users viewing unlawful content on their services and to delete it promptly when they learn of it.
