When Rogue One: A Star Wars Story opened in December 2016, audiences gasped at what they saw: Grand Moff Tarkin, played convincingly by a CGI recreation of Peter Cushing, who had died in 1994. The face was his. The posture, the cadence — all reconstructed from archival footage. ILM's visual effects team called the technique "digital human reconstruction." Cushing's estate had approved the use. Critics called it uncanny. Ethicists called it a threshold crossed.
Digital doubles began modestly. In the late 1990s and early 2000s, visual effects studios created brief stand-in renders of actors for dangerous shots — a face briefly visible during a car crash, a body tumbling off a building. These were functional fictions: good enough for a half-second, never meant to carry emotional weight.
The technology's evolution accelerated dramatically. The Lord of the Rings trilogy (2001–2003) pioneered large-scale digital crowd simulation and motion capture for characters like Gollum, but the faces of human actors remained practically off-limits for extended screen time. By the mid-2000s, projects like The Polar Express (2004) and Beowulf (2007) attempted performance capture for entire casts, with famously unsettling results — critics coined the term "uncanny valley" to describe the eerie wrongness of near-human faces.
The inflection point came with deep learning. Around 2017–2020, neural rendering and generative adversarial networks began producing photorealistic human faces that could be animated with high fidelity. What had required armies of artists and months of manual refinement could now be approximated in days, then hours. The digital double transformed from a costly visual effects specialty into a scalable production tool.
For Rogue One (2016), ILM spent approximately a year reconstructing Peter Cushing's likeness from archival footage and reference photographs. The team used machine learning-assisted texture mapping combined with traditional animation. Despite approval from the Cushing estate, the result ignited industry-wide debate about whether consent from an estate is ethically equivalent to consent from the living person.
Alongside posthumous reconstruction, a parallel technology emerged: de-aging living performers. Martin Scorsese's The Irishman (2019) used Industrial Light & Magic's Fusion Camera System to digitally subtract decades from the faces of Robert De Niro, Al Pacino, and Joe Pesci. The approach required the actors to perform without the usual prosthetic age-reversal makeup — the AI system tracked their facial geometry and retextured it in real time during post-production.
Marvel Studios had already pioneered similar techniques: Captain America: Civil War (2016) de-aged Robert Downey Jr. for a brief flashback, and Avengers: Endgame (2019) used neural rendering to restore Samuel L. Jackson's 1990s appearance across extended scenes. The practical effect on casting was immediate — studios began discussing whether actors could now "perform" across any era of a franchise's timeline, collapsing the relationship between biological age and on-screen age.
The 2023 SAG-AFTRA strike made these questions legally urgent. Among the union's core demands was protection against studios using a single day of performance capture to generate an unlimited synthetic version of an actor's likeness, deployable across productions and timelines without additional compensation.
Several companies now offer industrial-grade digital human pipelines. Metaphysic, the company behind hyperrealistic deepfake demonstrations of Tom Hanks and others, competed on America's Got Talent in 2022 using real-time AI face-swap technology, signaling that the tooling was no longer confined to major film studios. Epic Games' MetaHuman Creator provides a free browser-based tool that generates photorealistic 3D human heads in minutes, exportable to Unreal Engine — democratizing what once cost millions.
The implications for theater and live performance are beginning to materialize. In 2023, a touring production of Back to the Future: The Musical used projection mapping and digital composite techniques to integrate archival footage of original film actors into live scenes — a hybrid of recorded and live performance that blurs the boundary between the stage and the screen in unprecedented ways.
The digital double represents a fundamental shift in what "performance" means legally and economically. A performance is no longer a bounded event — a night's work, a day on set. It becomes raw material that can be extracted, scaled, and redeployed indefinitely. The actor's body is no longer the irreducible unit of production.
You are a studio ethics consultant reviewing a proposal to digitally recreate a recently deceased actor for three scenes in a sequel. Explore the ethical, legal, and craft considerations with the AI advisor below.
When The Two Towers opened in December 2002, Andy Serkis's performance as Gollum sparked an argument that has never fully resolved. His body movements and vocal work were recorded via optical motion capture markers, fed into Weta Digital's animation pipeline, and used as the foundation for a fully digital character. The craft was his. The screen credit was "Visual Effects." Serkis received no Academy Award nomination. Some called this an oversight. Others said the rules were clear: the Academy did not then recognize motion capture as a performance category.
Modern motion capture systems typically combine optical tracking (reflective markers on a bodysuit tracked by infrared cameras), inertial measurement units (IMUs) embedded in the suit, and facial performance capture (a helmet-mounted camera recording subtle muscle movements). The resulting data — a skeleton in space, with facial blendshapes — drives a rig: a digital puppet whose surface is rendered separately.
The gap between "performance data" and "finished character" is where the interpretive work happens, and that gap has been a source of ongoing labor disputes. Directors, animators, and motion capture actors each contribute meaningfully to the final result, and the existing guild structures were not designed to account for the entanglement of their contributions.
For Avatar (2009), James Cameron developed a new approach called "performance capture" — a term he preferred to motion capture — combining the body performance with on-set emotional capture using head-mounted rigs. The resulting Na'vi characters retained far more of the actors' original facial expressivity, reducing the animators' interpretive role and increasing the actor's direct authorship of the character.
The Screen Actors Guild has repeatedly debated whether motion capture performers should receive the same protections as on-screen actors. Historically, mocap performers sometimes worked under stunt contracts rather than SAG-AFTRA principal performer contracts, receiving lower minimums and fewer residual protections — despite delivering what were, in effect, lead performances.
The Academy of Motion Picture Arts and Sciences did not create a category for motion capture performance as of this writing. Andy Serkis's work as Gollum in the Lord of the Rings trilogy, Caesar in the Planet of the Apes reboot series, and Snoke in the Star Wars sequels has prompted recurring annual campaigns for recognition — none successful within the existing system. The AMPAS's position, historically, is that the performance cannot be evaluated independently of the visual effects work, making a separate acting nomination impossible under current rules.
Serkis himself has been the most prominent advocate for recognition, arguing in interviews and industry forums that the digital skin layered over a mocap performance is no different from the prosthetic makeup applied to traditional character actors — and that no one would deny Charlize Theron an acting nomination because she wore a prosthetic nose for Monster (2003).
The counterargument, raised by animators and their unions, is that the Weta or ILM team's interpretive work in translating raw capture data into expressive animation constitutes a genuine artistic contribution that is inappropriately erased when acting credit is assigned. The 2023 VFX workers' unionization efforts, led in part by organizers at major Marvel contractors, brought this tension into sharper relief.
Until recently, translating raw mocap data into final character animation required substantial manual cleanup — trained animators correcting foot sliding, interpenetration, and missing data. AI systems have increasingly automated this cleanup, compressing timelines and reducing crew sizes. NVIDIA's Omniverse platform includes AI-driven retargeting tools that can apply a captured human performance to a non-human rig automatically, a process that previously required weeks of manual work per shot.
The downstream effect on performer relationships is significant. If the animator's interpretive role is reduced — because AI is handling what they used to do — then the motion capture actor's original data becomes even more directly the source of the final performance. This strengthens the acting community's argument that mocap work deserves full performer recognition. It also intensifies concerns about how that original capture data might be reused without the performer's knowledge or consent.
Motion capture sits at the intersection of three competing claims of authorship: the actor who generated the movement, the animator who refined it, and the AI system that increasingly bridges them. None of the existing legal or guild frameworks was designed for this triangulation — which is precisely why it has generated so much unresolved conflict.
You're on an awards committee deciding whether to create a new category for motion capture performance, or to fold it into existing acting categories. Debate the merits with the AI mediator, who will steelman both sides.
Val Kilmer lost much of his natural speaking voice to throat cancer surgery in 2015. When Tony Scott's sequel Top Gun: Maverick (2022) called for Kilmer to reprise his role as Iceman, the production faced a dilemma. Kilmer agreed to participate. Sonantic, a UK AI voice company, trained a custom voice model on archival recordings of Kilmer's voice — interviews, film dialogue, recordings made before his surgery. The result was used for his scenes in Maverick. Kilmer himself called the experience emotional and deeply personal. He retained full consent and oversight throughout the process. It was, in the view of many commentators, a rare example of the technology genuinely serving a performer's needs rather than a studio's cost reduction goals.
Modern voice cloning systems use deep neural networks — typically a combination of a speaker encoder, a synthesizer, and a vocoder — trained first on large datasets of human speech, then fine-tuned on recordings of the target voice. Companies like ElevenLabs, Resemble AI, and Respeecher have built commercial pipelines that can produce convincing voice clones from as little as a few minutes of clean audio, though more data generally produces more accurate results.
Respeecher, a Ukrainian AI voice company, has been involved in several high-profile entertainment productions. Their technology was used to recreate a young Luke Skywalker's voice for The Mandalorian Season 2 (2020) and The Book of Boba Fett (2022), restoring Mark Hamill's vocal quality from the original Star Wars trilogy. Hamill's involvement and approval were secured for these projects, and he reportedly worked closely with the team to ensure the result matched his own standards.
The technology has also been used more controversially. In 2023, an AI-generated song featuring cloned voices of Drake and The Weeknd — titled "Heart on My Sleeve" — went viral with millions of plays before being taken down. The creator, a producer who released it under the name Ghostwriter977, argued it was a creative experiment. Universal Music Group, whose artists were cloned, argued it was copyright infringement. No legal resolution had been reached as of this writing.
The Val Kilmer and Mark Hamill cases represent one pole of voice cloning practice: the performer is alive, fully informed, and has meaningful agency over the result. The "Heart on My Sleeve" case represents the opposite pole: no consent, no compensation, and no clear legal remedy under existing law. Most real-world cases fall somewhere between these extremes.
The most legally fraught applications involve recreating the voices of deceased performers. In 2023, the Beatles released "Now and Then," promoted as the last Beatles song, using AI audio restoration tools developed by Peter Jackson's WingNut Films — the same technology used for the Get Back documentary — to isolate John Lennon's voice from a 1978 demo cassette tape. Paul McCartney described the process in a BBC interview, noting they used AI to separate Lennon's voice from background noise, not to fabricate new vocal lines. The distinction mattered enormously to the band and to critics evaluating the ethical status of the release.
The legal landscape governing posthumous voice use varies dramatically by jurisdiction. In California, Civil Code Section 3344.1 (the Astaire Celebrity Rights Act) extends rights of publicity to deceased persons for 70 years after death — one of the most protective statutes in the world. Tennessee's ELVIS Act (2024) extended similar protections specifically to AI-generated voice replicas, the first state law in the US to directly address synthetic voice cloning of performers' likenesses.
At the federal level, no comprehensive right of publicity law exists, creating a patchwork of state-level protections that leaves performers and estates in different legal positions depending on where a production is based or distributed.
For working voice actors — the audiobook narrators, video game characters, commercial voices, and animation performers who make up a large and relatively economically precarious sector of the performing arts — AI voice synthesis represents a direct structural threat. Voice work does not require physical presence, making it particularly vulnerable to automation. A publisher can generate an audiobook narration from an AI model trained on a prolific narrator's existing work, at a fraction of the cost of hiring that narrator for new sessions.
Findaway (an audiobook platform owned by Spotify) launched an AI-narrated audiobook initiative in 2023, generating significant controversy within the voice acting community. The Authors Guild and various voice actor unions pushed back. Some individual authors chose to opt out of the AI narration program. The episode illustrated how quickly AI voice technology can move from experimental to industrial deployment, and how inadequate existing contracts are at protecting performers from that transition.
The voice is simultaneously the most intimate signature of a performer and, from a data perspective, the easiest to capture and reproduce. A few hours of clean audio is all a modern system requires. This asymmetry — between the depth of a voice's meaning to its owner and the ease with which it can be synthesized — is at the heart of why voice cloning has become the most contested frontier in AI and performing arts.
You're advising a streaming platform that wants to implement an AI voice narration program for audiobooks. Work with the AI advisor to design a consent, compensation, and transparency framework that could satisfy both the platform's business needs and performers' rights advocates.
For the first time in 63 years, both the Writers Guild of America and the Screen Actors Guild went on strike simultaneously. The WGA had walked out in May; SAG-AFTRA followed in July. Together they shut down the majority of major Hollywood production. AI was not the only issue — streaming residuals, minimum guarantees, and basic compensation were all in dispute. But AI provisions dominated the public narrative, and for good reason: the studios' initial proposals included language that would have allowed them to scan background performers for a single day's pay and use those scans in perpetuity, across any project, without further consent or compensation.
The WGA reached a deal with the Alliance of Motion Picture and Television Producers (AMPTP) in late September 2023. The agreement included key provisions on AI: studios cannot use AI to write or rewrite literary material, AI-generated text cannot be used to undermine writers' residuals, and if studios use AI tools to generate material, that material is not considered "written by" a human writer under the agreement — protecting minimums and credits.
SAG-AFTRA reached its agreement in November 2023 after a 118-day strike. On AI specifically, the deal required: informed consent for the creation of a digital likeness; compensation for the creation and use of that likeness; and the right for performers to negotiate terms before any AI replication. The agreement created a distinction between "synthetic performers" — entirely AI-generated characters with no human model — and digital replicas of real performers, with stronger protections applying to the latter.
Crucially, the SAG-AFTRA agreement applied only to the major studios and streaming platforms that are signatories to the deal. Independent productions, international co-productions, and companies outside the studio system are not covered. This leaves a substantial portion of the industry — particularly the video game sector and non-union productions — operating under no binding AI consent framework.
One of SAG-AFTRA's most publicized demands was protecting background performers — extras — from having their scanned likenesses reused indefinitely without pay. The final agreement requires that studios obtain consent and provide compensation each time a background performer's digital scan is used in a new production, closing a loophole the studios had sought to exploit.
SAG-AFTRA went on strike against major video game companies in July 2024, specifically over AI provisions. The dispute centered on voice and motion capture actors' demands for protections equivalent to those won in the film and television deal. Striking companies included Activision, Electronic Arts, Epic Games, and Take-Two Interactive. The strike — SAG-AFTRA's second game industry strike, following one in 2016–2017 — was still ongoing as of late 2024, with AI consent and compensation the core unresolved issue.
The video game industry presents particular complications for performer protection. A character voice recorded for a game title may be reused across sequels, DLC expansions, and marketing materials over many years. Mocap data recorded for a character may be applied to entirely new characters through AI retargeting. The original performer may have no contractual visibility into these downstream uses under current game industry contracts.
Fran Drescher, SAG-AFTRA president, framed the video game dispute in explicitly existential terms in a July 2024 statement: "If we don't win this AI battle, there will be no industry for any of us."
The debates that convulsed film and television in 2023 are beginning to reach live performance, though more slowly and with less institutional infrastructure. Theater, with its strong emphasis on the live contract between performer and audience, has historically been more resistant to technologically mediated substitution. But several trends are challenging that insularity.
Digital projection and spatial audio systems have become common in large-scale theater productions, enabling hybrid performances that blend recorded and live elements. The use of holograms and high-definition projection to create "ghost appearances" of deceased performers at live concerts — ABBA Voyage at the ABBA Arena in London (2022–present) being the most sophisticated example — has created a new category of "live performance" that does not require the original performers to be physically present.
ABBA Voyage is instructive as a consent model: all four original ABBA members were alive at launch, all consented extensively to the creation of their "ABBAtars," and all participated in the motion capture sessions. The production was built from the beginning as a collaborative project rather than a top-down extraction of performer data. Whether such a model can or will be replicated more broadly in the industry remains to be seen.
For dance, AI tools have begun to emerge that can generate movement sequences in the style of specific choreographers, raising parallel questions about authorship and the ownership of movement vocabulary. The choreographic community has not yet reached the same level of collective action as actors' unions, but organizations including the International Association of Dance Medicine and Science have begun developing guidance on AI and movement data.
The 2023 strikes produced the first legally binding industry agreements on AI performer rights in history — a genuine threshold moment. But the gaps are substantial: independent productions, international markets, video games, theater, and dance are all largely outside the frameworks that now protect major-studio film and TV performers. The battles won in 2023 are a beginning, not a resolution.
You're advising a major performing arts organization — opera, ballet, or theater — on developing an AI policy that covers digital doubles, voice cloning, motion capture data, and posthumous performance. The board wants both creative possibilities and ethical guardrails.