In late 2017, a Reddit user operating under the name deepfakes posted videos that superimposed celebrity faces — including Gal Gadot, Scarlett Johansson, and Taylor Swift — onto pornographic content. Within weeks, the technique had spread across multiple platforms. The user published open-source code. Within months, dedicated apps had automated the process so thoroughly that anyone with a photograph could generate non-consensual explicit content in minutes.
This was not a fictional scenario. It was the documented origin point of an industrial-scale harm that researchers at Sensity AI would later quantify: by 2019, 96% of all deepfake videos online were non-consensual pornography, and 99% of targets were women.
The 2017 Reddit incident triggered a cascade that proved extraordinarily difficult to contain. Platforms including Reddit, Twitter, and Pornhub eventually banned deepfake pornography, but enforcement remained inconsistent. By 2023, the AI image generator landscape had expanded so dramatically that dedicated deepfake pornography sites were generating hundreds of thousands of images per month, according to research published by the Stanford Internet Observatory.
In 2023, teenage girls at Westfield High School in New Jersey discovered that AI-generated explicit images of them had been circulating among classmates. Similar incidents were documented in schools across New Jersey, Washington state, and internationally. In Spain, a group of girls aged 11–17 in the town of Almendralejo became victims of explicit AI-generated images created by male classmates using an app called Cloth Off. Spanish prosecutors opened criminal investigations in October 2023.
The harm is not abstract. Psychologists who have studied victims document trauma profiles comparable to those of survivors of actual sexual assault: intrusive thoughts, hypervigilance, social withdrawal, and severe anxiety about being photographed. Many victims report feeling that their bodies have been violated even though no physical contact occurred.
In a 2018 interview with Polygon, Scarlett Johansson — one of the earliest high-profile targets — stated: "The fact is that trying to protect yourself from the internet and its depravity is basically a lost cause... the internet is a vast wormhole of darkness that has no bottom." She noted that despite having resources to pursue legal action, there was effectively nothing to be done. Most victims have no such resources.
The legal landscape evolved slowly. The United Kingdom's Online Safety Act 2023 made sharing non-consensual intimate deepfakes a criminal offense. In the United States, as of 2024, no comprehensive federal law existed, though over 20 states had passed their own legislation with varying definitions and penalties. Georgia and Virginia were among the first states to explicitly criminalize deepfake pornography.
The DEFIANCE Act, introduced in the U.S. Senate in 2024 following high-profile deepfake images of Taylor Swift that spread across X (formerly Twitter), would create a federal civil cause of action for victims. The Swift incident — explicit AI-generated images that received tens of millions of views before platform removal — galvanized congressional attention in a way that thousands of less-famous victims had not.
Platform responses remained reactive. Meta, Google, and Microsoft all added filtering systems, but researchers consistently demonstrated that these filters could be bypassed with minor prompt modifications.
The defining characteristic of synthetic NCII is that it requires only a face. Unlike traditional non-consensual intimate imagery, which required either covert photography or theft of existing images, synthetic NCII can be created from any publicly available photograph — a LinkedIn profile, a school yearbook photo, a social media post. This transforms every public image of a person into a potential vector for abuse.
Researchers at the Revenge Porn Helpline (UK) reported a 400% increase in reports of synthetic NCII between 2022 and 2023. Yet advocates believe actual prevalence is vastly underreported. Many victims do not report because they do not believe law enforcement can help, because they fear secondary victimization, or because they are unaware that what happened to them may constitute a crime.
For minors, the reporting gap is particularly acute. Parents and school administrators often lack frameworks for understanding synthetic NCII, and the stigma that attaches to the victim — rather than the perpetrator — mirrors patterns documented in traditional sexual abuse cases for decades.
In this lab you will explore the documented psychological, legal, and social harms caused by non-consensual intimate deepfakes. Discuss real cases, the reporting gap, and why traditional legal frameworks often fail victims.
On January 1, 2019, Gabonese state television broadcast a video of President Ali Bongo Ondimba delivering a New Year's address. Bongo had not been seen publicly for months following a stroke. Within days, opposition figures and international media were debating whether the video was a deepfake — fabricated to conceal the president's incapacity or death. On January 7, elements of the military launched a coup attempt, citing the video as evidence of a government cover-up.
Investigators later concluded that the video was almost certainly genuine. But the damage was done: the mere possibility of a deepfake had been weaponized to justify an attempted seizure of power. Political scientists would later call this the liar's dividend — the way deepfake technology benefits those who wish to deny authentic evidence, not just those who create fake evidence.
Days before Slovakia's September 2023 parliamentary election, an audio recording spread on social media appearing to feature Michal Šimečka, leader of the liberal Progressive Slovakia party, discussing plans to buy votes. The recording was almost certainly an AI-generated fake, according to digital forensics specialists. Slovak law imposed a 48-hour media blackout before elections, preventing news organizations from reporting on or debunking the recording before polls opened.
Progressive Slovakia narrowly lost. The party that won, SMER, led by Robert Fico, had campaigned in part on a pro-Russia, anti-Ukraine platform. Whether the audio deepfake swayed the outcome cannot be proven. What can be documented is that a fabricated recording circumvented media norms precisely calibrated to prevent last-minute character assassination.
The incident was not isolated. Researchers at Oxford Internet Institute documented AI-generated political audio and video in elections across at least 16 countries between 2022 and 2024, including Bangladesh, India, Indonesia, Pakistan, and the United States.
In January 2024, New Hampshire voters received automated calls featuring a voice cloned to sound like President Joe Biden urging Democrats not to vote in the state's primary. "Don't vote on Tuesday," the voice said. "Save your vote for the November election." A political consultant named Steve Kramer later claimed responsibility as a demonstration of AI misuse risks. The FCC responded by clarifying that AI-generated voice calls without disclosure violated existing robocall regulations.
Legal scholar Robert Chesney and political scientist Danielle Citron coined the term "liar's dividend" in a 2019 paper to describe a counterintuitive consequence of deepfake technology: even if no deepfake is created, the knowledge that deepfakes exist allows bad actors to deny authentic evidence. A politician caught on a genuine recording of corrupt behavior can now claim the recording is AI-generated. A journalist with verified footage faces skepticism that did not exist before.
This phenomenon appeared in practice in 2023 when elected officials in multiple countries dismissed unfavorable video evidence as potential deepfakes without providing technical evidence for that claim. The burden of proof shifted from the accused to the accuser — a reversal of evidential norms that had taken centuries to establish.
In Nigeria's 2023 presidential election, a video circulated showing a candidate making allegedly incriminating statements. Both sides claimed the video's authenticity or fakery depending on which narrative served them. No definitive forensic conclusion was reached before election day.
A 2024 report by the AI company ElevenLabs — itself a major provider of voice-cloning technology — found that political deepfakes had been deployed in elections in every major democratic region. The report noted that the marginal cost of producing convincing synthetic political speech had fallen from tens of thousands of dollars to effectively zero between 2020 and 2024.
Meta announced in 2024 that it would require political advertisers to disclose when AI had been used to generate or substantially alter content. Google imposed similar requirements for election-related YouTube ads. But organic content — shared user-to-user rather than paid — remained largely unaddressed by these policies.
The EU AI Act (2024) required that synthetic media depicting real people carry persistent, machine-readable watermarks. Whether those watermarks would survive social media compression and sharing remained an open technical question at time of publication.
In this lab, you will analyze real documented cases of political deepfakes and discuss the systemic risks they pose to democratic institutions. Consider the liar's dividend, election interference mechanisms, and the adequacy of existing regulatory responses.
A finance worker at a multinational firm received a message from someone claiming to be the company's Chief Financial Officer, requesting a transfer for a confidential transaction. The employee was skeptical — until invited to a video conference call that appeared to show the CFO and other colleagues. The employee transferred HK$200 million (approximately US$25.6 million) to five different bank accounts.
Hong Kong police later confirmed that every participant on the video call — except the targeted employee — had been a deepfake. Fraudsters had apparently gathered enough video and audio of the actual executives from publicly available sources to reconstruct convincing real-time synthetic versions. It was one of the largest documented deepfake financial fraud cases in history.
The Hong Kong case represented the high end of deepfake financial fraud. At the mass-consumer level, voice cloning had by 2023 become an industrial fraud tool. The Federal Trade Commission (FTC) reported that Americans lost over $2.7 billion to impostor scams in 2023, with AI voice cloning increasingly implicated in a subset of these cases.
The most documented variant was the "grandparent scam" — a long-running fraud in which callers impersonate a grandchild in trouble, typically claiming to have been in a car accident or arrested. AI voice cloning transformed this scam: instead of merely sounding approximately like a young person in distress, fraudsters could now clone an actual grandchild's voice from social media videos. In 2023, Ruth Card, a 73-year-old Canadian woman, nearly wired $9,000 after receiving a call from a voice indistinguishable from her grandson's, saying he was in jail after a car accident.
The Wall Street Journal and Washington Post documented multiple cases in 2023 where cloned voices of real family members were used in emergency scam calls. In each case, the victim had never encountered a voice clone before and had no framework for doubting what their ears told them.
In one of the earliest documented deepfake audio fraud cases, the CEO of a UK-based energy company received a phone call from what he believed to be his parent company's German CEO. The voice instructed him to wire €220,000 to a Hungarian supplier immediately. The voice — later analyzed by cybersecurity firm Symantec as likely AI-generated — replicated the German executive's accent and speech patterns. The transfer was made. The money was never recovered.
Security researchers at companies including Pindrop and Resemble AI documented a taxonomy of corporate deepfake fraud vectors by 2024:
Executive impersonation: Cloned audio or video of C-suite figures used to authorize fraudulent transfers or disclose sensitive information. The Hong Kong case was the most dramatic example, but dozens of smaller-scale incidents were reported to cybersecurity firms annually.
Interview fraud: North Korean state actors were documented by the FBI and multiple tech companies as using AI-generated faces and voices to pass job interviews at technology companies, gaining employment and potentially insider access. The FBI issued a public warning about this vector in 2022, updated in 2024.
Synthetic reference checks: Fraudulent job applicants using voice cloning to impersonate references who could be called for verification. Multiple HR platforms documented this emerging pattern.
Market manipulation: Fake video statements attributed to executives causing stock price movements before correction. Securities regulators in the United States and EU flagged this as an emerging enforcement priority in 2024.
Corporate security protocols typically rely on voice recognition ("I recognize your voice") or video presence ("I can see you on screen") as authentication factors. Deepfake technology invalidates both. The Hong Kong case demonstrated that even trained, skeptical employees in high-value environments can be deceived when multiple simultaneous synthetic participants reinforce each other's apparent legitimacy. Security experts now recommend that large wire transfers require out-of-band verification — a separate, pre-established communication channel that cannot be faked in real time.
Cyber insurance providers began explicitly addressing deepfake fraud in policy language by 2023. Lloyd's of London market participants reported requests to clarify whether synthetic media-enabled fraud was covered under existing cyber or crime policies — a question with significant legal ambiguity. The answer varied by policy, by the nature of the deception, and by the degree of employee culpability.
Courts in multiple jurisdictions were beginning to grapple with whether employees who authorized transfers following deepfake deception could be held liable for negligence, or whether the technology had created a new category of criminal victimization that existing negligence frameworks did not anticipate.
In this lab you will analyze the mechanisms of deepfake financial fraud and develop practical organizational defenses. Consider the Hong Kong video call case, voice cloning scams, and the failure modes of traditional verification procedures.
During the October 2023 Hamas attack on Israel and the subsequent Israeli military operation in Gaza, social media platforms were flooded with video content. Fact-checkers at organizations including BBC Verify, AFP Fact Check, and Bellingcat documented dozens of videos that were either deepfakes, misattributed real footage from other conflicts, or AI-generated imagery. Some videos of alleged atrocities that circulated widely were later debunked. Others were real footage dismissed as fakes.
The result was something more disorienting than simply encountering misinformation: neither side could agree on which documented events had happened. Researchers noted that audiences increasingly sorted themselves into communities with entirely incompatible accounts of verifiable events. The technology had not merely created false content — it had destroyed the shared evidentiary baseline that even adversarial parties need to engage in meaningful disagreement.
Political philosophers and information scientists use the term epistemic infrastructure to describe the institutions, practices, and technologies that allow societies to establish shared facts. This includes journalism, courts of law, academic peer review, regulatory agencies, and the evidentiary standards embedded in each. Democratic governance depends on this infrastructure: elections require shared facts about who voted and for whom, courts require shared facts about what occurred, policy requires shared facts about the world.
Synthetic media does not merely introduce false facts into this infrastructure. More fundamentally, researchers at the Oxford Internet Institute and the Shorenstein Center at Harvard argue that it raises the cost of establishing any fact. Every piece of video evidence now requires authentication. Every audio recording requires forensic analysis. The marginal cost of doubt has dropped to zero while the marginal cost of certainty has risen dramatically.
Human rights organization Witness documented a new challenge in 2024: the use of deepfake allegations to prevent accountability for real atrocities. When documented evidence of human rights violations is dismissed as potentially AI-generated — without specific technical evidence — perpetrators gain a new legal and public relations tool. Witness's "Deepfakes and Human Rights" project noted that this represented a fundamental threat to evidence-based accountability mechanisms that had been central to international law since Nuremberg.
Researchers at MIT Media Lab and the Reuters Institute for the Study of Journalism documented measurable shifts in media consumption behavior attributable in part to deepfake awareness:
Healthy skepticism becoming pathological doubt: A 2023 Reuters Institute Digital News Report found that 56% of respondents in surveyed countries worried about distinguishing real from fake online news — the highest level recorded. Critically, this worry did not translate into better fact-checking; it translated into disengagement. Many people simply stopped believing news media entirely.
A 2024 study published in the journal Nature Human Behaviour found that simply warning people that deepfakes exist caused them to doubt authentic video evidence — even when no deepfake was present in the content shown to them. This "implied deepfake effect" suggested that the mere knowledge of the technology undermined trust in all video evidence.
Journalism's documentary function under strain: Photojournalists and documentary filmmakers at organizations including World Press Photo reported that their work was increasingly met with deepfake allegations even when images were certified authentic. The organization updated its rules in 2023 to require disclosure of AI use and created new verification frameworks — but noted that these frameworks could not address audiences who had already decided to disbelieve.
Creating a convincing deepfake takes seconds. Debunking it requires forensic analysis that may take hours or days, specialist expertise, access to original footage for comparison, and a platform willing to amplify the correction. The correction almost never reaches as many people as the original fabrication. Information scientists call this "correction asymmetry," and research consistently shows that corrections — even when successful — fail to fully reverse belief change caused by misinformation. Deepfakes dramatically widen this asymmetry.
Multiple response categories have emerged, each with documented limitations:
Technical detection: AI detection tools exist but suffer from an adversarial dynamic — detection improves, generation improves to defeat detection, repeat. Researchers at University of Southern California and DARPA's MediFor program have shown that current detectors perform well on known deepfake types but fail at rates that make them unreliable as standalone verification tools.
Provenance and watermarking: The Content Authenticity Initiative (CAI), backed by Adobe, Microsoft, and major news organizations, developed the C2PA standard — a cryptographic provenance system that embeds creation metadata into media files. When functional, this system can verify that an image or video was captured by a specific device and has not been altered. Limitations: metadata can be stripped by social media platforms during compression, and the system only verifies provenance for content created within it.
Media literacy: Studies show media literacy education improves critical thinking about media claims, but the implied deepfake effect suggests that some deepfake awareness may paradoxically increase rather than decrease overall media distrust — producing disengagement rather than discernment.
Regulation: The EU AI Act, UK Online Safety Act, and various U.S. state laws created obligations around disclosure and removal. Enforcement against offshore creators and distributors remains a fundamental challenge. Compliance obligations fall on platforms while creators often operate anonymously across jurisdictions.
In this lab you will explore how synthetic media creates epistemic crises — not merely by introducing false content, but by destroying shared mechanisms for establishing truth. Discuss the implied deepfake effect, correction asymmetry, and the limits of technical, legal, and educational responses.