In April 2023, voice-cloning startup ElevenLabs found its platform weaponized within weeks of launch. Users uploaded brief audio samples of real individuals β including a cloned voice of Emma Watson reading Adolf Hitler's Mein Kampf β without any consent from the subjects. The company had not yet implemented consent verification. The episode forced an immediate policy overhaul and became a canonical example of what happens when synthetic media tools launch without consent infrastructure.
Consent in synthetic media is not the same as consent in photography or even conventional acting. When a person's face, voice, or body is used to generate synthetic content, the result can be made to say or do anything β in perpetuity, at scale, with near-zero marginal cost of reproduction. This asymmetry makes informed, specific, and revocable consent the central ethical requirement.
The three components matter equally. Informed means the subject understands what the synthetic output will be used for, not merely that their data is being collected. Specific means consent to one use-case does not automatically extend to others β an actor who licenses their voice for a video game has not consented to political advertising. Revocable means the subject retains the ongoing right to withdraw, and that withdrawal must propagate through systems that have already generated derivative content.
The 2023 SAG-AFTRA strike β the longest Hollywood work stoppage in decades β placed AI likeness consent at its center. Studios had proposed contract language allowing them to scan background actors' likenesses for a single day's pay and then use those digital replicas indefinitely without further compensation or consent. The union's negotiators called this "digital slavery." The eventual November 2023 deal included explicit provisions requiring informed consent for each distinct use, compensation for AI-generated work equivalent to the work it replaces, and the right to decline AI use entirely.
The SAG-AFTRA outcome became a reference point for employment law discussions globally, demonstrating that consent frameworks must address not just initial capture but ongoing, use-specific authorization.
The consent principles codified in the SAG-AFTRA deal apply equally outside Hollywood. Any organization training synthetic voice or face models on employee data, any app capturing user likenesses for avatar generation, and any researcher building datasets from public figures' images faces the same ethical obligations: specificity, transparency, and reversibility.
A common misunderstanding holds that public figures β politicians, celebrities, executives β forfeit consent rights by entering public life. This is legally and ethically incorrect. Public figures accept reduced privacy expectations regarding their public conduct, but they retain the right to control synthetic representations of themselves in commercial contexts, sexual contexts, and contexts that imply endorsement of views they do not hold.
When the Indiana Pacers used AI to generate a fake post-game interview with a player who had actually declined to speak with media in 2024, the team faced significant backlash precisely because the player had actively withheld consent β and the synthetic version overrode that decision. Being a public figure does not constitute blanket consent to synthetic impersonation.
Ethical practitioners treat consent not as a legal checkbox but as a design constraint embedded in the creation process itself. This means:
Consent is not a one-time event. It is an ongoing relationship between creator and subject that must be actively maintained throughout the lifecycle of synthetic content β from generation through distribution to archival or deletion.
You are advising a documentary production company that wants to use AI voice synthesis to recreate the voices of deceased historical figures for an educational film. The director wants to know what consent framework they should build β and whether this use-case is even ethically permissible.
Discuss consent design, edge cases, and the ethical distinctions with your AI lab assistant. There are no simple right answers β work through the reasoning.
In March 2024, a synthetic audio clip of President Joe Biden's voice instructing New Hampshire Democrats not to vote in the primary was traced to a political consultant named Steve Kramer, working for a rival candidate. The robocall reached approximately 25,000 voters. New Hampshire's Attorney General launched an investigation, and the FCC subsequently ruled that AI-generated voices in robocalls require disclosure under the Telephone Consumer Protection Act. Kramer had made no attempt to label the content as synthetic.
Disclosure β the act of clearly identifying synthetic media as AI-generated β serves a fundamentally different purpose than consent. Consent protects the subject of the synthetic content. Disclosure protects the audience. Both obligations exist independently; satisfying one does not satisfy the other.
The core ethical case for disclosure is epistemic: audiences have a right to know what kind of evidence they are evaluating. A video that looks like documentary footage activates different cognitive processes than one labeled as AI-generated speculation. When that distinction is hidden, the creator is exploiting a cognitive vulnerability β the audience's evolved tendency to trust audiovisual evidence β in a way the audience has not agreed to.
Disclosure requirements are moving rapidly from ethical best practice to legal mandate. As of 2024, the regulatory landscape includes:
Requires providers of AI systems that generate or manipulate image, audio, or video content to label outputs as AI-generated. Deep synthesis systems must watermark outputs in a machine-readable format.
Following the Biden robocall incident, the FCC ruled that AI-generated voices in robocalls are illegal under the Telephone Consumer Protection Act β criminalizing undisclosed synthetic voice impersonation at the federal level.
Requires large online platforms to label or remove "materially deceptive" AI-generated content related to elections during the 120 days before an election.
China's Cyberspace Administration regulations require deepfake content to carry a clear and conspicuous label, and prohibit using deepfakes to impersonate news organizations or government bodies.
Political satire has always placed fabricated words in real people's mouths. Saturday Night Live impersonators, editorial cartoons, and satirical news sites operate under this tradition. The question is whether deepfakes qualify for the same protection β and the answer is conditional.
Satire enjoys First Amendment and equivalent protections when a reasonable person would understand the content is not genuine. The critical variable is whether the synthetic content is realistic enough to deceive. A cartoon drawing of a politician saying something absurd is clearly satirical. A photorealistic video of the same politician that is indistinguishable from real footage β even if labeled satirical β operates in ethically distinct territory, because not all viewers will see the label, and the realistic footage will circulate independently of it.
The deepfake of Ukrainian President Volodymyr Zelensky appearing to order Ukrainian troops to surrender, which circulated on social media in March 2022, illustrates where the satire defense breaks down entirely: the content was not labeled as satire, was not obviously satirical in tone or content, and was designed to be mistaken for genuine news footage during an active military conflict.
The video was distributed on Ukrainian social media platforms and briefly appeared on a Ukrainian news website that had been hacked. Zelensky quickly responded with a genuine video from Kyiv to counter the disinformation. Meta, YouTube, and Twitter removed it. The incident demonstrated that deepfakes deployed in active conflict zones can constitute information warfare β and that disclosure labels are functionally useless once decontextualized clips begin spreading.
Ethical creators should implement disclosure at multiple layers β not just a label in the video itself, but structural mechanisms that persist even as content travels:
Transparency is not adequately achieved by a label that theoretically exists somewhere. It requires that disclosure be impossible to separate from the content β structurally, technically, and contextually β so that audiences who encounter the content in any context can identify its synthetic nature.
A civic tech nonprofit wants to create educational videos showing AI-generated versions of historical presidential speeches β using voice synthesis to make famous speeches more accessible to younger audiences. They plan to post on YouTube and TikTok.
Design a disclosure strategy that is robust across both platforms and all the ways content might be re-shared. Consider what mechanisms to use and what failure modes remain.
In October 2019, the AI Foundation published research showing that deepfake pornography constituted approximately 96% of all deepfake video detected online at that time. The vast majority depicted female celebrities and public figures who had not consented. The same year, a 15-year-old girl in Virginia had her face deepfaked onto pornographic footage by a classmate β a case that contributed to Virginia becoming the first U.S. state to criminalize non-consensual deepfake pornography. By 2024, at least 14 U.S. states had enacted similar statutes.
Ethical frameworks for synthetic media recognize certain categories of use that are prohibited absolutely β not merely regulated, not merely subject to disclosure requirements, but impermissible regardless of consent status, creative framing, or claimed purpose. These categorical prohibitions exist because some harms are so severe, so reliably produced by the use-case, and so structurally impossible to mitigate through procedural safeguards that no framework can make the activity ethical.
Non-consensual intimate imagery (NCII) using synthetic faces is categorically prohibited because the harm β to the subject's dignity, psychological wellbeing, professional life, and relationships β is severe and irreversible regardless of the technical label of "synthetic." Studies by the Cyber Civil Rights Initiative found that victims of NCII (synthetic or real) suffer rates of anxiety, depression, and suicidality comparable to survivors of sexual assault.
Synthetic child sexual abuse material (CSAM), even when entirely AI-generated without any real child involved, is prohibited under federal law in the United States (PROTECT Act, as extended) and equivalent legislation in most jurisdictions, on the grounds that it normalizes abuse, may be used in grooming, and is intrinsically harmful regardless of the absence of a direct victim in production.
The convergence of synthetic media capabilities and democratic elections represents one of the most significant harm vectors in the technology's history. The 2024 global "super cycle" of elections β with major votes in the United States, India, the United Kingdom, the European Parliament, Mexico, Taiwan, Indonesia, and dozens of other countries β produced an unprecedented volume of documented synthetic media incidents.
In Slovakia's September 2023 elections, an audio deepfake of liberal party leader Michal Ε imeΔka appeared to discuss buying votes, released two days before the election when Slovak law prohibits campaign activity β and thus prohibits rebuttals. Fact-checkers identified it as synthetic, but the timing was designed to exploit the legally mandated silence period.
In Bangladesh's January 2024 elections, multiple deepfake videos of opposition politicians making inflammatory statements were distributed on Facebook. The platform later acknowledged delayed response to removal requests.
These cases share a structure: synthetic content deployed at a moment of maximum harm (immediately before voting) with minimum opportunity for effective rebuttal. This is not incidental β it is a designed weaponization of the gap between synthetic media creation speed and institutional response speed.
The strategic deployment of the Ε imeΔka deepfake during Slovakia's mandatory pre-election media blackout period illustrates why harm prevention cannot rely solely on the targeted politician's ability to rebut. Effective governance requires platform-level rapid-response capabilities that operate independently of the subject's ability to defend themselves.
Many synthetic media techniques exist on a spectrum where the same capability enables both clearly beneficial and clearly harmful applications. Face-swapping technology enables privacy protection for documentary subjects and harassment campaigns. Voice synthesis enables accessibility tools for people with speech impairments and telephone fraud. This dual-use character means harm prevention cannot simply prohibit technologies β it must address the specific harmful applications.
The ethical obligation for creators working with dual-use tools is harm forecasting: before creating and releasing synthetic media, systematically considering how the specific output could be misused, what the realistic harm scenarios are, and whether the safeguards in place are sufficient to reduce those harms to acceptable levels. This is not a theoretical exercise β it is a professional responsibility analogous to the "foreseeable misuse" standards applied in product liability law.
When a synthetic media use-case has a high probability of producing severe harm that cannot be adequately mitigated β regardless of the creator's intent β the ethical obligation is to not create the content. Intention does not determine harm. The question is not "did I mean to cause harm?" but "is this content likely to cause harm in the hands of anyone who encounters it?"
A startup has built a real-time face-swapping tool designed for the entertainment industry β allowing actors to try on different appearances for costume and makeup previsualization. The tool works in real-time via webcam and produces high-quality results. They are considering a consumer app launch.
Conduct a harm forecast for this product launch. Identify the realistic misuse scenarios, assess their severity, and recommend which safeguards β if any β would make the consumer release ethical.
In May 2023, the Content Authenticity Initiative β founded by Adobe, Twitter, and The New York Times in 2019 and expanded to over 2,000 member organizations β released version 2.0 of the C2PA specification, adding explicit support for AI-generated content provenance. Members including Microsoft, Google, Sony, Nikon, and the BBC committed to implementing the standard in their creation and distribution tools. Adobe began embedding C2PA credentials in all content produced with its generative AI features in Photoshop and Firefly. For the first time, the creative tools industry had a technical infrastructure for provenance at scale.
Responsible innovation in synthetic media is not primarily about saying no. It is about building the technical, procedural, and cultural infrastructure that makes ethical use the default β easier than unethical use, not harder. This requires action at multiple levels simultaneously: individual creators, development companies, platforms, and industry coalitions.
Individual creators adopt consent documentation practices, implement multi-layer disclosure, conduct harm forecasts before release, and participate in professional communities that develop and enforce norms.
Companies building synthetic media tools integrate consent verification, embed C2PA metadata, build revocation mechanisms, and implement safeguards against categorical prohibited uses at the model level.
Distribution platforms implement AI-content labeling, establish rapid-response deepfake removal protocols, surface C2PA provenance data to users, and develop appeals processes for wrongful removal.
Cross-industry bodies like C2PA and PAI (Partnership on AI) develop shared technical standards, conduct research on harm mitigation, and create accountability mechanisms that operate across competitive boundaries.
In 2023, the Partnership on AI β a nonprofit coalition including Amazon, Apple, Google, Meta, Microsoft, and civil society organizations β published its Responsible Practices for Synthetic Media framework. The document identified four core obligations for actors in the synthetic media ecosystem:
Adobe's implementation of C2PA through its "Content Credentials" system provides a concrete example of responsible innovation in commercial tools. When a user generates an image using Adobe Firefly, the system automatically attaches a signed, tamper-evident credential to the file recording: the creation date, the tool used, whether AI generation was involved, and the specific model. This credential remains attached when the file is exported, and can be verified at contentcredentials.org even after the image has been shared online.
Adobe faced an internal tension: adding Content Credentials added friction to the creative workflow and potentially disadvantaged Firefly in competition with tools that didn't implement similar safeguards. The company publicly committed to the standard anyway, arguing that the long-term value of trustworthy creative AI outweighed short-term competitive costs. Whether that commitment holds under continued competitive pressure remains an open question β but it represents a documented case of a major company accepting competitive costs for ethical infrastructure.
Responsible innovation faces a structural challenge: companies that implement ethical safeguards (consent verification, harm filtering, provenance tracking) bear real costs that less scrupulous competitors avoid. This creates a "race to the bottom" dynamic unless industry standards are enforced broadly enough to prevent evasion. C2PA and similar coalitions exist partly to solve this coordination problem β making ethical implementation the industry norm rather than a competitive disadvantage.
For individual practitioners, responsible innovation translates into a set of specific professional commitments. These are not abstract principles but concrete practices that can be built into any synthetic media workflow:
The ethical frameworks covered in this module β consent, disclosure, harm prevention, and responsible innovation β are not static. They are being actively developed by practitioners, policymakers, researchers, and affected communities in real time. The most important commitment any synthetic media creator can make is to engage with that ongoing development rather than treating any current framework as final. Ethical practice in synthetic media is a process, not a checklist.
You are a freelance video producer who has been approached by a client to create a promotional video using AI voice synthesis to recreate the voice of a recently deceased founder of their company. The client says they have "family permission." The video will be used for internal corporate events and potentially investor presentations.
Design your complete ethical review protocol for deciding whether and how to take this project. Apply the consent, disclosure, harm prevention, and responsible innovation principles from this module.