10대 남학생들이 소셜 미디어 사진을 악용해 여학생의 가짜 성적 합성물을 만드는 딥페이크 범죄가 전 세계 학교로 확산되고 있습니다. 이는 미성년자가 포함된 아동 성학대 물질(CSAM)에 해당하며, 피해자들에게 막대한 정신적 피해를 주고 있습니다. 생성형 AI 기술의 접근성이 높아짐에 따라 학교와 당국의 실효성 있는 대응과 피해자 보호가 시급한 과제로 떠올랐습니다.
번역된 본문
모든 것은 대개 소셜 미디어에서 다운로드한 한 장의 사진에서 시작됩니다. 전 세계의 10대 남학생들은 학교에서 아는 여학생들의 인스타그램이나 스냅챗 이미지를 저장하고, 유해한 '옷 벗기기(nudify)' 앱을 사용해 그들의 가짜 나체 사진이나 영상을 제작하고 있습니다. 이러한 딥페이크는 순식간에 학교 전체에 퍼질 수 있으며, 피해자들은 수치심, 모욕감, 절망감을 느끼고 그 이미지가 평생 자신을 괴롭힐까 두려워합니다.
학교를 강타한 딥페이크 위기는 몇 년 전부터 서서히 시작되었지만, 음란 이미지를 생성하는 기술에 쉽게 접근할 수 있게 되면서 그 규모가 상당히 커졌습니다. 디지털 기만 및 허위 정보에 초점을 맞춘 매체인 WIRED와 Indicator의 공개 보고 사례 검토에 따르면, 딥페이크 성범죄 사건은 전 세계 약 90개 학교를 강타했으며 600명 이상의 학생에게 영향을 미친 것으로 나타났습니다. 조사 결과, 2023년 이후 최소 28개국의 학생(대부분 고등학교 남학생)이 생성형 AI를 사용하여 급우들을 상대로 성적 딥페이크를 표적으로 삼은 혐의를 받고 있습니다. 미성년자가 포함된 이러한 음란 이미지는 아동 성학대 물질(CSAM)로 간주됩니다.
이 분석은 전 세계 학교에서 발생하는 AI 딥페이크 남용의 실제 사례를 검토한 최초의 연구로 여겨집니다. 분석 전반을 살펴보면, 유해한 AI 옷 벗기기 기술의 전 세계적 확산을 보여줍니다. 이 기술은 개발자들에게 매년 수백만 달러의 수익을 안겨주며, 많은 사건에서 학교와 법 집행 기관이 이러한 심각한 성범죄 사건에 제대로 대처하지 못하고 있음을 드러냅니다.
북미 전역에서는 2023년 이후 30건에 가까운 딥페이크 성범죄 사건이 보고되었습니다. 여기에는 60명 이상의 피해자가 발생한 사건, 피해자가 오히려 학교에서 일시 퇴학당한 사건, 여러 학교 학생들이 동시에 표적이 된 사건 등이 포함됩니다. 남미에서는 10건 이상, 유럽에서는 20건 이상, 호주와 동아시아를 합쳐서 12건 이상의 사건이 공개적으로 보고되었습니다.
학교 내에서 발생하는 딥페이크 성범죄의 실제 규모는 이보다 훨씬 클 가능성이 높습니다. 유엔 아동 기금(UNICEF)의 설문 조사에 따르면 작년에 120만 명의 아동이 자신의 성적 딥페이크 피해를 입은 것으로 추정됩니다. 스페인의 청소년 5명 중 1명은 '세이브 더 칠드런' 연구진에게 자신의 딥페이크 나체 사진이 만들어졌다고 답했습니다. 아동 보호 단체 '쏜(Thorn)'은 10대 8명 중 1명이 표적이 된 사람을 알고 있다고 밝혔으며, 2024년 민주주의와 기술 센터(Center for Democracy and Technology)의 조사에서는 응답한 학생의 15%가 자신의 학교와 관련된 AI 생성 딥페이크에 대해 알고 있다고 답했습니다.
캐나다 아동 보호 센터의 기술 책임자인 로이드 리처드슨(Lloyd Richardson)은 "이 영향을 받지 않은 학교를 찾기란 매우 힘들 것"이라며, "가장 중요한 것은 이런 일이 발생했을 때 우리가 피해자들을 어떻게 도울 수 있느냐는 것입니다. 왜냐하면 그 여파는 엄청날 수 있기 때문입니다."라고 말했습니다.
WIRED와 Indicator의 분석은 학교 위치 및 잠재적 피해자 수와 같이 구체적인 세부 사항이 공개적으로 보고된 사건들을 조사했습니다. 주로 영어권 보도 자료를 중심으로 이루어졌으며, 많은 국가의 경우 데이터가 부족한 실정입니다. 많은 사건이 언론에 보도되지 않거나, 보되더라도 구체적인 내용이 포함되지 않은 채 학교와 법 집행 기관에 의해 비공개로 처리됩니다. 그럼에도 불구하고 명확한 패턴이 나타납니다.
거의 모든 사건에서 10대 남학생이 이미지나 영상 제작의 가해자로 지목되고 있습니다. 이들은 주로 소셜 미디어 앱이나 인스턴트 메신저를 통해 급우들과 해당 자료를 공유합니다. 그리고 이는 피해자에게 막대한 해를 끼칩니다. 올해 초 아이오와주의 한 피해자는 "그들이 나를 볼 때마다 그 사진들을 떠올릴까 봐 걱정된다"고 말했습니다. 또 다른 피해자의 가족은 "그 아이는 울기만 하고 아무것도 먹지 않는다"고 전했습니다. 여러 사례에서 피해자들은 학교에 가기를 원하지 않거나, 자신들의 음란 이미지나 영상을 만든 사람들을 마주치는 것을 꺼립니다. "그녀는 상"
Save Story Save this story Save Story Save this story It usually starts with a photo downloaded from social media. Around the world, teenage boys are saving Instagram and Snapchat images of girls they know from school and using harmful “nudify” apps to create fake nude photos or videos of them. These deepfakes can quickly be shared across whole schools, leaving victims feeling humiliated, violated, hopeless, and scared the images will haunt them forever. The deepfake crisis hitting schools started slowly a couple of years ago, but it has since grown considerably as the technology used to create the explicit imagery has become more accessible. Deepfake sexual abuse incidents have hit around 90 schools globally and have impacted more than 600 pupils, according to a review of publicly reported incidents by WIRED and Indicator , a publication focusing on digital deception and misinformation. The findings show that since 2023, schoolchildren—most often boys in high schools—in at least 28 countries have been accused of using generative AI to target their classmates with sexualized deepfakes. The explicit imagery, containing minors, is considered to be child sexual abuse material (CSAM). This analysis is believed to be the first to review real-world cases of AI deepfake abuse taking place at schools globally. As a whole, the analysis shows the worldwide reach of harmful AI nudification technology, which can earn their creators millions of dollars per year, and shows that in many incidents, schools and law enforcement officials are often not prepared to respond to the serious sexual abuse incidents. Across North America, there have been nearly 30 reported deepfake sexual abuse cases since 2023—including one with more than 60 alleged victims , one where the victim was temporarily expelled from school , and others where pupils at multiple schools have allegedly been targeted simultaneously . More than 10 cases have been publicly reported in South America, more than 20 across Europe, and another dozen in Australia and East Asia combined. The true scale of deepfake sexual abuse taking place in schools is likely much higher. One survey by United Nations children’s agency Unicef estimates that 1.2 million children had sexual deepfakes created of them last year. One in five young people in Spain told Save the Children researchers that deepfake nudes had been created of them. Child protection group Thorn found one in eight teens know someone targeted, and in 2024, 15 percent of students surveyed by the Center for Democracy and Technology said they knew about AI-generated deepfakes linked to their school. “I think you’d be hard-pressed to find a school that has not been affected by this,” says Lloyd Richardson, director of technology at the Canadian Centre for Child Protection. “The most important thing is how we’re able to help the victims when this happens, because the effects of this can be massive.” WIRED and Indicator’s analysis looked at incidents that have been publicly reported with specific details, such as locations of schools and potential victim counts. Mostly these are English-language reporting, with a lack of data being available for many countries. Many incidents are never reported in the press, may not include specific details if they are, and instead can be handled privately by schools and law enforcement officials. Nevertheless, there are clear patterns that appear. In nearly all cases, teenage boys are allegedly responsible for the creation of the images or videos. They are often shared in social media apps or via instant messaging with classmates. And they are hugely harmful to the victims. “I’m worried that every time they see me, they see those photos,” one victim in Iowa said earlier this year. “She's been crying. She hasn't been eating,” another’s family said . In multiple instances, victims often do not want to attend school or be faced with seeing those who created explicit images or videos of them. “She feels hopeless because she knows that these images will likely make it onto the internet and reach pedophiles,” says lawyer Shane Vogt, and three Yale Law School students, Catharine Strong, Tony Sjodin, and Suzanne Castillo, who are representing one unnamed New Jersey teenager in legal action against a nudifying service. “She is severely distressed by the knowledge that these images are out there, and she will have to monitor the internet for the rest of her life to keep them from spreading.” In South Korea and Australia , schools have given pupils the option not to have their photos in yearbooks or stopped posting images of students on their official social media accounts, citing their use for potential deepfake abuse. “Around the world, there have been cases where school images were taken from public social media pages, altered using AI, and turned into harmful deepfakes,” one school in Australia said . “Imagery will instead feature side profiles, silhouettes, backs of heads, distant group shots, creative filters, or approved stock photography.” Sexual deepfakes created using AI have existed since around the end of 2017 ; however, as generative AI systems have emerged and become more powerful, they have led to a shadowy ecosystem of “nudification” or “undress” technologies. Dozens of apps, bots, and websites allow anyone to create sexualized images and videos of others with just a couple of clicks, often with no technical knowledge . “What AI changes is scale, speed, and accessibility,” says Siddharth Pillai, cofounder and director of the RATI Foundation, a Mumbai-based organization working to prevent violence against women and children. “The technical barrier has dropped significantly, which means more people, including adolescents, can produce more convincing outputs with minimal effort. As with many AI-enabled harms, this results in a glut of content.” Amanda Goharian, the director of research and insights at child safety group Thorn, says its research indicates that there are different motivations involved in teenagers creating deepfake abuse, ranging from sexual motivations, curiosity, revenge, or even teens daring each other to create the imagery. Studies involving adults who have created deepfake sexual abuse similarly show a host of different reasons why the images may be created. “The goal is not always sexual gratification,” Pillai says. “Increasingly, the intent is humiliation, denigration, and social control.” “It’s not just about the tech,” says Tanya Horeck, a feminist media studies professor and researcher focusing on gender-based violence who has looked at sexualized deepfakes in UK schools at Anglia Ruskin University. “It's about the long-standing gender dynamics that facilitate these crimes.” As the number of deepfake incidents at schools has increased in recent years, the ways that they are handled by schools and law enforcement agencies can vary wildly. Parents have complained that not enough action has been taken by officials. In one case, it reportedly took three days for a school to report an incident to police; in another, a victim claimed there have been no immediate consequences for the individuals allegedly responsible. Sometimes students face charges for creating and possessing CSAM, while others face different criminal charges or suspensions from school. In March, two students in Pennsylvania admitted guilt in juvenile court and were later sentenced to 60 hours of community service on CSAM-related felony charges for creating images and videos of 60 girls. In multiple cases, teenage girls and their families are the ones who have ended up fighting back against the creation of deepfake sexual abuse, often moving faster than politicians, who generally have been slow to act. Teenagers have walked out of class to show support to victims, protested against alleged perpetrators, been involved in the creation of online training courses , and changed laws, including contributing to the creation of the T