Deepfake porn is a new form of digital violence. It consists of creating fake images or videos of sexual, erotic, or intimate content using Artificial Intelligence. To create this content, a real image of a person is used, for example, a photo of their face or body. Then, Artificial Intelligence modifies that image and creates content that looks real, but is actually fake. It can change the body, the location, the movements, or other elements to make it seem like the person is participating in a sexual or intimate situation that never happened. This practice, which denigrates and violates the dignity of women, emerged in June of this year.
In a WhatsApp group, where there were around 50 adolescents from two pre-university schools in CABA, AI-generated pornographic images and videos were exchanged and commercialized, featuring adolescents whose images had been stolen and altered, i.e., deepfake porn. The case quickly went viral and opened a debate about what measures should be taken against those responsible. However, the debate focused mainly on punishing the aggressors and relegated a deeper question to the background: what is behind this new form of gender violence and what conditions allow it to reproduce? Deepfake porn does not appear in isolation, but in a context where discourses that question women's rights and reinforce their objectification are normalized. At the same time, these new technologies open up a market where large technology companies can obtain significant profits, thus becoming a business. However, it is not the first time such an event has occurred.
In 2003, when Mark Zuckerberg was still a student at Harvard University. That year he created FaceMash, a website that allowed users to compare photographs of two female students and vote on which one they considered more attractive. To create FaceMash, Zuckerberg used photographs of Harvard students without permission for users to compare and vote on which one they considered more attractive. The page reached more than 22,000 visits in four hours, but was shut down by the university due to the violation of the students' privacy. A few months later, Zuckerberg launched TheFacebook, a platform that allowed students to create profiles and share photographs. Over time, that project would become Facebook, one of the most important social networks in the world.
Questions to the experts
Experts agree on a fundamental idea: technology is not neutral. In many cases, it can be used to perpetrate violence against women, girls, and other vulnerable groups, and these aggressions can also generate economic benefits. According to the State of Deepfakes report, it alleges that between 2022 and 2023, the number of pornographic videos created with Artificial Intelligence increased by 464%, from 3,725 to 21,019 videos. Furthermore, 99% of the people who appear in these videos are women. Cristina Pujol, professor of Information and Communication Sciences at the UOC, explains that: "It's not new, the first deepfakes started with celebrities, faces of Jennifer Lawrence or Scarlett Johansson were cut out and embedded in pornographic scenes that were then uploaded to adult websites."
The harm caused by this content can be very real. Rocío Pina, professor of Psychology, Education Sciences, and Criminology at the UOC, explains that sexual deepfakes can be considered a form of violence against women because fake images are used to manipulate and damage a person's reputation. Even if the video is fake, the victim can suffer personal, social, and emotional problems. Also, there are legal consequences. Eduard Blasi, professor of Law and Political Science at the UOC and privacy expert, points out that this content can violate rights such as privacy, honor, and one's own image. This happens because, although the sexual situation that appears in the video is fake, a person's real image, such as their face or voice, is used without their permission. Furthermore, the problem increases when these images are shared and reach other people.
First conviction in Spain for undressing women with artificial intelligence
Three men have been sentenced in Seville to nine months in prison each for using artificial intelligence to manipulate photographs of three former schoolmates and simulate that they were naked. This is the first conviction in the city for this type of act, according to the Superior Court of Justice of Andalusia. The convicted individuals obtained the images from the young women's public Instagram profiles and modified them using an artificial intelligence application on Telegram.
Afterward, they shared the fake photographs in WhatsApp and Discord groups, from where they spread beyond those spaces. One of the victims became aware of what happened when another person forwarded the images to her. The judge has imposed a ban on using Telegram, WhatsApp, Instagram, and Discord for three years and nine months, in addition to obliging them to complete a sexual education program. The sentence applies the gender aggravating circumstance, considering that the accused acted with a sexist attitude that objectified women. Although the prison sentence has been suspended due to the absence of prior offenses and the reparation of damages, they must comply with the conditions set by the court.
Is the punishment sufficient? It is not normal for three men to use artificial intelligence to manipulate images of three women, simulate that they are naked, and disseminate them without their consent, and for the judicial response to include a temporary ban on using certain social networks. What happens when that ban ends? What guarantees do the victims have that they will not suffer a similar situation again? Although the resolution contemplates educational measures and other obligations, it is worth asking whether they are sufficient in the face of behavior that violates the privacy, dignity, and image of women. When the ban ends, the convicted individuals will be able to use these tools again, unless there is some other legal restriction. Artificial intelligence is not the problem by itself: it is using it to humiliate, sexualize, and harm other people.
Therefore, the judicial response must take into account not only the damage caused, but also the prevention of new behaviors and the protection of victims. The freedom to use social networks cannot become an excuse to violate the privacy of other people without assuming responsibility.
Deepfake pornography survey: knowledge, attitudes and impact. Source: Security Hero, State of Deepfakes.
The EU tightens AI control and Sánchez calls for greater regulation
The European Union is taking new measures to combat sexual deepfakes and protect people whose images are used without permission. One of the most important measures is the prohibition of Artificial Intelligence tools that allow the creation of fake sexual images or videos of real people without their consent. This measure also affects applications that can use a person's photograph to create an image in which they appear naked.
In addition, the EU wants tech companies to be more responsible for the tools they create. Action can be taken not only against the person who uses these applications to cause harm, but also against companies that allow their systems to be used to create this type of content. Companies will have to include security measures to prevent such uses. The new rules consider it a crime to create or share child sexual abuse material using Artificial Intelligence or deepfakes. With these measures, Brussels seeks to adapt laws to the advancement of a technology that allows the creation of increasingly realistic fake content that can cause real harm to affected individuals.
Pedro Sánchez argues that Artificial Intelligence should not remain under the control of tech companies themselves, but that there must be public regulation and supervision. In a recent speech, he criticized the idea of leaving regulation in the hands of companies and stated that "it is a mistake to put the fox in charge of the henhouse." In the specific case of sexual deepfakes, Sánchez has argued that the response must involve creating laws that prohibit such content when it is made without the consent of the affected person.
