\n
![](\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2024\/08\/GettyImages-1167582826-e1723851708639.jpg?w=2048\")
<\/p>\n [ad_1]
\n<\/p>\n
Nearly a year after AI-generated nude images of high school girls upended a community in southern Spain, a juvenile court this summer sentenced 15 of their classmates to a year of probation.<\/p>\n
But the artificial intelligence tool used to create the harmful deepfakes is still easily accessible on the internet, promising to \u201cundress any photo\u201d uploaded to the website within seconds.<\/p>\n
Now a new effort to shut down the app and others like it is being pursued in California, where San Francisco this week filed a first-of-its-kind lawsuit that experts say could set a precedent but will also face many hurdles.<\/p>\n
\u201cThe proliferation of these images has exploited a shocking number of women and girls across the globe,\u201d said David Chiu, the elected city attorney of San Francisco who brought the case against a group of widely visited websites based in Estonia, Serbia, the United Kingdom and elsewhere.<\/p>\n
\u201cThese images are used to bully, humiliate and threaten women and girls,\u201d he said in an interview with The Associated Press. \u201cAnd the impact on the victims has been devastating on their reputation, mental health, loss of autonomy, and in some instances, causing some to become suicidal.\u201d<\/p>\n
The lawsuit brought on behalf of the people of California alleges that the services broke numerous state laws against fraudulent business practices, nonconsensual pornography and the sexual abuse of children. But it can be hard to determine who runs the apps, which are unavailable in phone app stores but still easily found on the internet.<\/p>\n
Contacted late last year by the AP, one service claimed by email that its \u201cCEO is based and moves throughout the USA\u201d but declined to provide any evidence or answer other questions. The AP is not naming the specific apps being sued in order to not promote them.<\/p>\n
\u201cThere are a number of sites where we don\u2019t know at this moment exactly who these operators are and where they\u2019re operating from, but we have investigative tools and subpoena authority to dig into that,\u201d Chiu said. \u201cAnd we will certainly utilize our powers in the course of this litigation.\u201d<\/p>\n
Many of the tools are being used to create realistic fakes that \u201cnudify\u201d photos of clothed adult women, including celebrities, without their consent. But they\u2019ve also popped up in schools around the world, from Australia to Beverly Hills in California, typically with boys creating the images of female classmates that then circulate widely through social media.<\/p>\n
In one of the first widely publicized cases last September in Almendralejo, Spain, a physician whose daughter was among a group of girls victimized last year and helped bring it to the public\u2019s attention said she\u2019s satisfied by the severity of the sentence their classmates are facing after a court decision earlier this summer.<\/p>\n
But it is \u201cnot only the responsibility of society, of education, of parents and schools, but also the responsibility of the digital giants that profit from all this garbage,\u201d Dr. Miriam al Adib Mendiri said in an interview Friday.<\/p>\n
She applauded San Francisco\u2019s action but said more efforts are needed, including from bigger companies like\u00a0California-based Meta Platforms<\/a>\u00a0and its subsidiary WhatsApp, which was used to circulate the images in Spain.<\/p>\n While schools and law enforcement agencies have sought to punish those who make and share the deepfakes, authorities have struggled with what to do about the tools themselves.<\/p>\n In January, the executive branch of the European Union explained in a letter to a Spanish member of the European Parliament that the app used in Almendralejo \u201cdoes not appear\u201d to fall under the bloc\u2019s sweeping\u00a0new rules for bolstering online safety<\/a>\u00a0because it\u2019s not a big enough platform.<\/p>\n Organizations that have been tracking the growth of AI-generated child sexual abuse material will be closely following the San Francisco case.<\/p>\n The lawsuit \u201chas the potential to set legal precedent in this area,\u201d said Emily Slifer, the director of policy at Thorn, an organization that works to combat the sexual exploitation of children.<\/p>\n A researcher at Stanford University said that because so many of the defendants are based outside the U.S., it will be harder to bring them to justice.<\/p>\n Chiu \u201chas an uphill battle with this case, but may be able to get some of the sites taken offline if the defendants running them ignore the lawsuit,\u201d said Stanford\u2019s Riana Pfefferkorn.<\/p>\n She said that could happen if the city wins by default in their absence and obtains orders affecting domain-name registrars, web hosts and payment processors \u201cthat would effectively shutter those sites even if their owners never appear in the litigation.\u201d<\/p>\n<\/div>\nRecommended Newsletter:<\/strong> The Fortune Next to Lead newsletter is a must-read for the next generation of C-suite leaders. Every Monday, the newsletter provides the strategies, resources, and expert insight needed to claim the most coveted positions in business. Subscribe now<\/a>.<\/div>\n