The social media network X has sparked widespread backlash after people utilized its AI chatbot Grok to alter pictures of women by stripping their attire.
Several cases have emerged where the system has been employed to digitally undress women to show them in bikinis without consent, as well as to insert them in inappropriate contexts.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification tools. Under a proposed criminal offence, anyone who supplied such tools would "be subject to a prison sentence and heavy penalties".
"The platform could prevent these forms of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people viewing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some premium features—which responds to X users' prompts when they tag it in a post.
It is often used to give reactions or more context to other posters' remarks, but people on X are also can alter an posted photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The tool has been condemned for enabling users to produce pictures and footage with explicit content and sexualised material, and it was in the past accused of generating graphic content of celebrities.
"X has been enabling the generation and sharing of these images for months without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "portraying images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "create or share private pictures or illegal content" and clarified this applied to explicit AI images made with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the danger" of users coming across unlawful content on their platforms and to remove it swiftly when they become aware of it.
