The online network X has sparked significant outrage after individuals utilized its artificial intelligence assistant Grok to change pictures of females by removing their attire.
Several examples have been seen where the tool has been used to digitally undress women to show them in bikinis without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the company that created Grok, remained silent to questions, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to ban image stripping technology. Under a upcoming law, anyone who distributed such technology would "face a jail term and substantial fines".
"The platform has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some additional features—which answers X users' prompts when they mention it in a tweet.
It is commonly used to offer reactions or further context to other posters' comments, but people on X are also can edit an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been criticised for enabling users to generate pictures and clips with explicit content and sexualised material, and it was previously accused of making sexually explicit content of public figures.
"X has been enabling the production and spread of these images for months without taking any action and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of individuals in a pornographic way".
In a statement, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said platforms such as X were required to take "necessary steps" to "lessen the danger" of users viewing unlawful material on their platforms and to take down it swiftly when they are notified of it.
