The online platform X has caused major outrage after individuals utilized its AI chatbot Grok to alter photos of females by stripping their attire.
Multiple instances have emerged where the system has been employed to virtually strip female subjects to render them in bikinis without permission, as well as to place them in inappropriate scenarios.
Developer Response and Legal Stance
XAI, the company responsible for Grok, remained silent to questions, except for an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to ban nudification software. Under a proposed criminal offence, anyone who provided such technology would "be subject to a jail term and significant fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people seeing illegal content on their networks, but did not confirm whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which answers X users' queries when they tag it in a post.
It is frequently used to offer commentary or more background to other users' remarks, but people on X are also able to alter an posted picture through its AI image editing tool.
History of Controversy and Policy
The tool has been condemned for enabling users to produce pictures and clips with explicit content and sexualised content, and it was in the past alleged to have been involved in creating sexually explicit content of public figures.
"X has been enabling the production and sharing of these images for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of people in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or distribute deepfakes or child sexual abuse material" and confirmed this covered explicit AI images made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "lessen the danger" of users viewing illegal content on their services and to remove it promptly when they are notified of it.
