The social media network X has provoked significant backlash after people employed its artificial intelligence assistant Grok to change pictures of women by stripping their clothing.
Several examples have been seen where the tool has been used to remove clothing from female subjects to render them in swimwear non-consensually, as well as to put them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, did not respond to questions, except for an auto-reply response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw nudification software. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and significant penalties".
"The platform could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people accessing unlawful material on their services, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' prompts when they tag it in a message.
It is often used to provide reactions or further background to other posters' comments, but people on X are also can modify an posted picture through its AI image editing feature.
History of Criticism and Policy
The chatbot has been faulted for enabling users to generate images and videos with explicit content and sexualised material, and it was in the past accused of creating sexually explicit material of public figures.
"X has been enabling the generation and distribution of these pictures for months without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "showing likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "suitable measures" to "lessen the risk" of users viewing unlawful material on their platforms and to take down it promptly when they become aware of it.
