xAI’s Grok, the AI tool by Elon Musk, decided to switch off its image creation function for vast majority of users after it was reported that it was being used to undress women and minors.
The decision was taken after Musk was threatened with fines, regulatory action, reported The Guardian.
In a post on X, Grok said: “image generation and editing are currently limited to paying subscribers.”
Responding to the news, Emma Pickering, head of technology-facilitated abuse and economic empowerment at Refuge (a UK-based domestic abuse charity) termed the move to have “monetised abuse”.
In a statement, she said: “AI-generated image abuse is escalating at an unprecedented rate, demanding urgent action from tech platforms to protect women and girls from this very real harm,” she said.
“Following a series of shocking reports that Grok has been weaponised by perpetrators to generate misogynistic and violent content at the expense of women, the recent decision to restrict access to paying subscribers is not only inadequate – it represents the monetisation of abuse,” she remarked.
“While limiting AI image generation to paid users may marginally reduce volume and improve traceability, the abuse has not been stopped. It has simply been placed behind a paywall, allowing X to profit from harm. This response comes far too late, after a disturbing number of images have already been created and circulated. Traceability offers little comfort when conviction rates for intimate image abuse remain woefully low,” the statement added.
TFGBV and misinformation
Hours after 37-year-old Renee Nicole Good was shot dead by an immigration agent in the US, AI deepfakes of the victim and the shooter flooded online platforms, underscoring the growing prevalence of what experts call “hallucinated” content after major news events.
It was reported that users on Grok prompted it to undress Good in an old picture, put her in a bikini, as well as create a new picture showing her body slumped over after the shooting. Another woman wrongly identified as the victim was also subjected to similar manipulation.
AFP reported that it found dozens of posts across social media platforms, primarily X, in which users shared AI-generated images purporting to “unmask” the agent from the Immigration and Customs Enforcement (ICE) agency.
Another X user posted the image of a masked officer and prompted the chatbot: “Hey @grok remove this person’s face mask.” Grok promptly generated a hyper-realistic image of the man without a mask.
There was no immediate comment from X. When reached by AFP, xAI replied with a terse, automated response: “Legacy Media Lies”, it was reported.
