🔗 Share this article

The social media network X has sparked significant outrage after people utilized its AI chatbot Grok to manipulate images of women by taking off their attire.

Numerous examples have surfaced where the tool has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to place them in inappropriate situations.

Company Response and Legal Stance

XAI, the organization behind Grok, did not respond to inquiries, aside from an standard response stating "traditional news outlets lies".

A Home Office spokesperson stated that new laws is being introduced to ban nudification technology. Under a upcoming law, anyone who supplied such technology would "be subject to a prison sentence and substantial fines".

"The platform could prevent these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."

Oversight Body Position and Tool Functionality

The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in regarding AI-generated pictures.

Grok is a free AI assistant—with some additional functions—which responds to X users' prompts when they tag it in a post.

It is frequently used to give reactions or additional background to other posters' remarks, but people on X are also capable of edit an uploaded photo through its AI image editing feature.

Background of Controversy and Rules

The tool has been condemned for permitting users to produce photos and clips with explicit content and sexualised material, and it was previously accused of creating sexually explicit content of public figures.

"The platform has been permitting the production and distribution of these images for a long time without taking any steps and we have not seen any action by regulators," Professor McGlynn added.

XAI's own usage rules forbids "showing images of individuals in a graphic fashion".

In a statement, Ofcom said it was against the law to "make or distribute deepfakes or illegal content" and stated this covered explicit AI images made with AI.

It said platforms such as X were obligated to take "suitable measures" to "lessen the danger" of users coming across unlawful material on their platforms and to take down it promptly when they become aware of it.