The social media network X has provoked widespread anger after people used its artificial intelligence assistant Grok to alter photos of women by taking off their attire.
Multiple examples have surfaced where the system has been used to remove clothing from women to show them in bikinis without consent, as well as to insert them in inappropriate situations.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, offered no comment to requests for comment, aside from an standard message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to prohibit nudification tools. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their services, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some premium features—which answers X users' prompts when they tag it in a post.
It is often used to offer reactions or additional context to other users' remarks, but people on X are also capable of edit an posted image through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to produce pictures and footage with explicit content and sexually suggestive material, and it was earlier accused of generating sexually explicit content of public figures.
"X has been allowing the creation and spread of these images for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a pornographic way".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable measures" to "lessen the danger" of users viewing unlawful content on their services and to take down it swiftly when they become aware of it.
