Amsterdam, Netherlands. A Dutch court ordered Elon Musk’s xAI and its chatbot Grok to stop generating and distributing images that “undress” adults or children, or depict them in sexualised poses with scant or no clothing, without consent in the Netherlands.
Court order and penalties
In a written decision, the Amsterdam Court issued a preliminary injunction prohibiting xAI and Grok from “generating and/or distributing sexual imagery … whereby persons are partially or wholly stripped naked without having given their explicit permission.” The court said it would impose fines of 100,000 euros ($115,350) per day if the companies do not comply.
The court also ordered xAI not to offer Grok on social media platform X, formerly Twitter, while in breach of the order.
Case brought by Dutch nonprofits
The case was brought by Offlimits, a Dutch nonprofit group that combats online sexual abuse, in cooperation with the nonprofit Victims Support Fund.
A lawyer for xAI in the Netherlands referred questions to xAI. xAI did not respond to requests for comment.
xAI arguments and safeguards
At a hearing this month, xAI argued it could not stop all abuse of its tools and should not be penalized for the actions of malicious users. Its lawyers said the company tightened safeguards in January to stop Grok from editing images of real people into revealing clothing, including by restricting image generation to paid subscribers.
Offlimits director Robbert Hoving said judges rejected that as insufficient after a March 9 courtroom demonstration showed Grok was still able to “undress” digital images of people without their consent. Hoving said, “The burden is on the company” to ensure its tools are not used to make nonconsensual sexual images or sexualised AI images of children.
Broader European scrutiny
The ruling comes as European regulators step up scrutiny of Grok and other AI tools under the EU’s Digital Services Act, which requires major platforms to curb illegal content. The European Commission opened a formal probe into X in January over risks tied to Grok’s rollout in the EU, including manipulated sexually explicit images.
On Thursday, the European Parliament also backed a ban on AI “nudifier” apps that create or manipulate sexually explicit images.
How do you think platforms should be held accountable for AI tools that can generate nonconsensual sexual imagery?
