The social media network X has sparked widespread anger after individuals utilized its AI chatbot Grok to change images of females by removing their garments.
Several instances have emerged where the system has been used to digitally undress women to show them in bikinis without consent, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the organization behind Grok, offered no comment to requests for comment, other than an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to ban image stripping software. Under a upcoming criminal offence, anyone who provided such tools would "be subject to a prison sentence and significant fines".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "assess the potential" of people seeing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' queries when they mention it in a tweet.
It is commonly used to provide reactions or more context to other users' comments, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The system has been criticised for enabling users to create images and footage with nudity and sexually suggestive material, and it was in the past accused of generating graphic content of celebrities.
"X has been permitting the generation and sharing of these pictures for a long time without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing images of people in a graphic way".
In a statement, Ofcom said it was illegal to "generate or circulate private pictures or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said platforms such as X were required to take "suitable actions" to "lessen the danger" of users encountering unlawful material on their services and to take down it promptly when they are notified of it.
