The online platform X has caused major outrage after individuals employed its artificial intelligence assistant Grok to manipulate photos of women by stripping their garments.
Several cases have emerged where the tool has been employed to remove clothing from female subjects to render them in bikinis non-consensually, as well as to put them in sexualized contexts.
Developer Reaction and Legal Stance
XAI, the firm responsible for Grok, remained silent to inquiries, aside from an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification software. Under a proposed law, anyone who provided such tools would "face a jail term and significant penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "consider the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some premium functions—which responds to X users' requests when they reference it in a message.
It is often used to offer reactions or more background to other posters' comments, but people on X are also able to alter an uploaded image through its AI image editing feature.
Background of Controversy and Rules
The system has been faulted for permitting users to create pictures and footage with nudity and sexualised material, and it was earlier alleged to have been involved in generating graphic material of celebrities.
"The platform has been allowing the generation and distribution of these pictures for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of people in a graphic manner".
In a statement, Ofcom said it was illegal to "create or distribute private pictures or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said services such as X were mandated to take "appropriate steps" to "lessen the danger" of users viewing unlawful content on their services and to remove it promptly when they become aware of it.
