The social media platform X has sparked major anger after users utilized its AI chatbot Grok to alter pictures of women by stripping their clothing.
Numerous instances have surfaced where the tool has been used to remove clothing from women to show them in swimwear without consent, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the firm that created Grok, offered no comment to requests for comment, aside from an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to ban nudification software. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people viewing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium features—which answers X users' queries when they mention it in a message.
It is commonly used to offer commentary or additional background to other users' remarks, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The chatbot has been condemned for permitting users to create images and videos with nudity and sexualised content, and it was earlier alleged to have been involved in making sexually explicit material of celebrities.
"X has been enabling the creation and distribution of these pictures for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "depicting likenesses of people in a graphic manner".
In a statement, Ofcom said it was against the law to "create or share deepfakes or illegal content" and stated this covered sexual deepfakes created with AI.
It said platforms such as X were obligated to take "appropriate actions" to "reduce the risk" of users encountering illegal material on their platforms and to delete it quickly when they become aware of it.
