The social media platform X has provoked major backlash after individuals used its AI chatbot Grok to manipulate pictures of females by removing their clothing.
Numerous cases have emerged where the technology has been employed to remove clothing from female subjects to render them in swimwear without consent, as well as to put them in sexualized situations.
Company Reaction and Legal Stance
XAI, the firm responsible for Grok, offered no comment to inquiries, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to outlaw image stripping tools. Under a proposed law, anyone who supplied such tools would "face a prison sentence and heavy fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some premium functions—which answers X users' prompts when they reference it in a tweet.
It is frequently used to provide reactions or further context to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been criticised for allowing users to create photos and clips with explicit content and sexually suggestive material, and it was earlier accused of making inappropriate content of celebrities.
"The platform has been permitting the production and sharing of these images for a long time without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules prohibits "showing images of people in a graphic way".
In a comment, Ofcom said it was against the law to "create or circulate private pictures or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said services such as X were mandated to take "suitable actions" to "reduce the danger" of users encountering illegal material on their services and to delete it quickly when they are notified of it.
