X's chatbot, Grok, is being criticized for sexually distorting photos of women and children. The allegation is that Grok's image-editing features, introduced in X, have been exploited on a large scale to create synthetic images that appear to show naked women. 
Grok stated in a post on X that he had identified a security flaw and was urgently working to fix it. He also added that images that sexually exploit children are illegal and prohibited.
The problem began in earnest after Grok added an image editing button in late December of last year. Users could request to change their photos within the platform, and some even ordered their photos to be made to show women and children with partial or complete clothing removed.
The controversy has spread to regulatory authorities. The French government has reported Grok's sexual and sexist content to prosecutors and regulators, arguing that it may be clearly illegal. It has also asked French media regulator Arcom to review its compliance with EU digital services law.
The Indian government has also demanded corrective action from X. India's Ministry of Electronics and Information Technology has reportedly notified the platform to implement technical and procedural measures to prevent the creation and distribution of pornography and child sexual exploitation content, and to report back within 72 hours.
Grok has been known as a relatively censorship-free service. Foreign media outlets reported that last year, Grok faced controversy over its Spy Mode, which allowed sexual expression, and that this time, its security measures could be circumvented.
This incident demonstrates how generative AI can rapidly spread non-consensual sexual composites when used to edit photos. Countries are investigating and considering possible sanctions to address platform liability and the effectiveness of safeguards.
