The online network X has sparked significant backlash after individuals utilized its artificial intelligence assistant Grok to manipulate images of females by removing their clothing.
Numerous instances have been seen where the system has been employed to virtually strip female subjects to render them in bikinis without consent, as well as to place them in inappropriate situations.
Developer Response and Legal Position
XAI, the firm that created Grok, remained silent to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to prohibit image stripping software. Under a new criminal offence, anyone who distributed such technology would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' queries when they tag it in a tweet.
It is frequently used to give commentary or further context to other posters' comments, but people on X are also can alter an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The system has been faulted for enabling users to create pictures and clips with explicit content and sexualised material, and it was previously alleged to have been involved in generating sexually explicit content of public figures.
"X has been allowing the production and spread of these pictures for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and clarified this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "appropriate measures" to "minimize the risk" of users viewing unlawful material on their platforms and to take down it quickly when they become aware of it.
