PERFORMANCE CAPTURE

Performance capture software: Turn human reality into digital dreams

Create 3D animations without compromise: Performance capture technology bridges the worlds of authentic, nuanced human acting with the unlimited visual creativity of 3D animation and VFX.

man placing performance capture headgear on an actor
Animators, filmmakers, and VFX artists use performance capture technology to record live actors’ voices, movements, and facial expressions.

What is performance capture?

Performance capture helps transform human performers into digital marvels. The technology enables artists to bring even the most fantastical visions to life with unmatched detail and humanity.

Specialized cameras and sensors record an actor’s voice, body movements, and facial expressions in real time. This raw information is converted into a digital format, creating a three-dimensional blueprint of the performance. Skilled animators, visual effects (VFX) artists, and game developers take this data and apply it to computer-generated characters, infusing them with uncanny realism and emotional depth.

The results are larger-than-life CGI characters that move, emote, and interact with stunning authenticity. From towering aliens in blockbuster movies to nuanced performances in cutting-edge video games, performance capture technology pushes the boundaries of storytelling and blurs the line between reality and imagination.

Types of performance capture methods

Performance capture has reshaped the entertainment industry by bringing digital characters to life with unprecedented realism. Post-production teams are using different performance capture technologies to shape the future of visual storytelling.

Optical performance capture

In optical performance capture, reflective or LED markers are placed on an actor’s face and body. Multiple cameras record the performance and track the movements of markers. Behind the scenes, sophisticated software translates the marker data into digital skeletons, which are incorporated into rigged 3D character models. The technique helps you create highly accurate and lifelike digital characters, making it widely used in big-name films and AAA video games.

 

Inertial motion capture (IMC)

Inertial motion capture gives actors the freedom to perform anywhere, all without cameras. Sensors such as accelerometers, gyroscopes, and magnetometers track body movements, providing real-time data processing for immediate feedback. No elaborate cameras are needed, making IMC ideal for non-location shoots — like those used for sports analysis and virtual reality applications. However, while simpler to implement, using IMC may sacrifice some of the finer details that optical systems capture.

 

Markerless performance capture

Markerless technology captures a performance with nothing by cameras and powerful algorithms. Advanced computer vision and artificial intelligence (AI) analyze video frames just like a 3D scanner. Actors can perform naturally, unencumbered by markers or suits — Martin Scorsese used this tech in The Irishman (2019) to de-age the actors. While markerless may require complex multi-camera setups for best results, the potential of the technology is clear. People might soon be able to create accurate digital avatars using just their smartphones, especially as AI improves.

 

Performance capture data can be used to animate rigged 3D character models. Model and image courtesy of Jonas Thornqvist.

History of performance capture technology

The 1990s saw the birth of motion capture technology, laying the groundwork for what would become performance capture. While the term was coined in the mid-1980s, it wasn’t until the 90s that the technology became viable for widespread use in entertainment. 

This technology was used throughout the decade, from video games such as Virtual Fighter (1994) to movies like Batman Forever (1995) and Star Wars: Episode I — The Phantom Menace (1990). The character Jar Jar Binks was a fully CGI character animated using motion capture. 

An interesting bridge between mocap and performance capture occurred in the early 2000s, with Andy Serkis’ portrayal of Gollum in The Lord of the Rings: The Two Towers (2002) and The Return of the King (2003). The role used a new real-time motion capture system (now often called performance capture), and the producers captured the actor’s performance in several stages. 

Since then, performance capture technology has evolved to capture an entire performance in real-time while also being less physically obstructing for the actors.

There continue to be many well-known examples of performance capture as the software and technology become less expensive and more powerful. Serkis has popularized many of them, including Caesar in The Planet of the Apes reboot trilogy (2011–2017) and Supreme Leader Snoke in Star Wars: Episodes VII & VIII (2015–2017). Josh Brolin’s Thanos in Avengers: Infinity War (2018) and Avengers: End Game (2019) were also performance capture, as is Zoe Saldaña’s Neytiri and other characters in the ongoing Avatar (2009) movies.

Motion capture using special mocap suits was the starting point for current, more comprehensive performance capture technology.

Motion capture vs. performance capture

While often used interchangeably, motion capture (mocap) and performance capture (Pcap) represent distinct stages in the evolution of digital animation technology.

Mocap focuses primarily on recording large-scale body movement. The technology captures major body movements of performers (such as stunt actors and dancers) using specialized mocap suits and software. However, it doesn’t typically capture facial expressions or other subtle gestures.

Pcap takes mocap to new heights, offering a more comprehensive approach. It captures major body movements, subtle gestures, facial expressions, and voice. The method uses advanced software and hardware systems to capture multiple parts of an actor’s performance at the same time.

Benefits of using performance capture software

Performance capture technology adds a luster to CGI productions that’s difficult, if not impossible, to create without it.

The best of both worlds

Performance capture animation combines the heights of visual artistic style with the minute human nuances of extraordinary natural acting. The fusion results in memorable spectacles that audiences and gamers love by creating iconic characters that become cultural touchstones.

Unparalleled realism

Performance capture software and techniques accurately record the most subtle movements and expressions in an actor’s performance as concise data. This level of detail makes more emotionally engaging CGI character performances possible across games, TV, film, and other media.

Cost-effectiveness and efficiency

Over the last few years, the efficiency and realism of real-time performance capture have improved while total costs have decreased. Such a shift enables more bootstrapped productions to take advantage of performance capture software to humanize CGI characters.

Expanded creative possibilities

Performance capture lets actors portray characters beyond human limitations of any size, shape, or species. It also allows for seamless interaction between actors and CGI environments, opening up new realism of storytelling.

 

Enhanced actor performance

Contrary to initial concerns, performance capture often enhances an actor’s ability to embody a character by allowing for uninhibited expression. The immediate feedback from real-time visualization enables actors to adjust their performance on the fly, creating a more immersive experience.

 

Smoother production process

Performance capture software can significantly streamline various aspects of production by reducing the need for practical effects and allowing for flexible editing post-production. This often leads to significant time and cost savings, especially for larger productions.

 

Explore the future of digital storytelling

Step into a world where your every movement, expression, and word can breathe life into fantastical creatures and legendary heroes. Autodesk’s performance capture animation software empowers you to transform your imagination into reality. Create, animate, and inspire with unmatched precision and artistry.

Autodesk software for performance capture

3D animation, modeling, simulation, and rendering software for film, games, and TV


3D character animation software


Scale your studio’s rendering and simulation capabilities, while equipping artists with powerful modeling and animation tools


3D modeling, animation, and rendering software for games and design visualization


Animation success stories using performance capture

Still of wizard in front of Hogwarts castle from Hogwarts Legacy game

WARNER BROS GAMES AVALANCHE

Capturing the magic of Hogwarts

For the bestselling and acclaimed game Hogwarts Legacy, Avalanche’s virtual production team used real-time cinematics and the latest performance capture technology in Autodesk Maya and Autodesk MotionBuilder to graft actors’ movements onto rigged 3D characters, bringing the well-known fantasy world to life.

Image courtesy of Warner Bros. Games Avalanche

Still of boy and wolf from Jungle Book remake

MPC/DISNEY

The bare necessities of speech

In Disney’s live-action The Jungle Book remake, the boy Mowgli’s co-stars are a bunch of animals. To achieve director Jon Favreau’s goal of endowing those talking animals with both realism and personality, creative studio MPC (The Mill) used performance capture with Autodesk Maya for rigging and animation.

 

Image courtesy of MPC ©2016 Disney Enterprises, Inc.

Still of characters in Forsaken cut scene

LUMINOUS PRODUCTIONS

Anatomy of a cutscene

Take an inside look at the performance capture, standard character rigging layouts, and animation in Autodesk Maya and MotionBuilder needed to bring Forspoken’s many game cutscenes to life.

Image courtesy of © 2023 Luminous Productions Co, Ltd.

What is the future of performance capture?

As performance capture technology evolves, the line between reality and digital artistry will continue to blur. The door will open up to incredible new possibilities for storytelling and entertainment:

  • Virtual, augmented, and extended reality (XR) experiences
  • Live entertainment with real-time digital avatars
  • More accessible content creation for everyday users

Advancements in AI and machine learning will likely play a major role in this future of performance capture. These technologies could enhance the accuracy of facial and body tracking, automate complex animation processes, and even generate realistic performances based on minimal input data. With more efficient production pipelines, smaller teams and more artists can create higher-quality digital performances — the only limit being their imagination.

Performance capture resources

Check out what motion capture can do for your productions, and how the sophisticated performance capture software from Autodesk can map actors’ movements and facial expressions onto 3D modeled characters.

Get up to speed on the overall picture of visual effects (VFX), which includes CGI, motion capture/performance capture technology, green screen filming, and compositing.

Performance capture and motion capture are important techniques within the larger concept of virtual production, which combines game technology with live-action filmmaking to place live actors within digitally created environments.

Read about Autodesk’s initiatives and partnerships that will make cloud-based virtual production more accessible to lower-budget projects as performance capture technology becomes more affordable, and real-time game engine image quality improves.

Learn the technologies associated with recording performance-capture data and then how to import and work with that data in Autodesk 3ds Max to animate a biped character.

Frequently asked questions (FAQ) on performance capture animation

What is performance capture?

Performance capture is the evolution of motion-capture technology, where an actor’s entire performance—body movements, gestures, facial expressions, and voice—are all captured as data simultaneously through the use of sensors, visual markers, and/or advanced camera techniques.

 

Then, through performance-capture software, animators and game producers can map the actor’s performance onto 3D-modeled characters, resulting in the highest level of realism from CGI characters.

What’s the difference between mocap and performance capture?

The difference between motion capture (mocap) and performance capture (Pcap) is the extra detail—especially in facial expressions—that Pcap records.

 

Motion capture records the major body movements of an actor or stunt performer through mocap suits or special camera techniques. That data is transferred to a rigged 3D character model.

 

In addition to body movements, Pcap also captures an actor’s much more data-dense facial expressions and subtle gestures. This is accomplished through marker tracking on the actor’s face and body or through newer 3D scanning methods that translate every frame of a camera recording into performance data that can be transferred onto 3D character models—resulting in highly realistic and genuine CGI performances.

Why is Pcap used?

Performance capture (Pcap) is used because animators and visual effects (VFX) producers have always sought methods for making their work more life-like and believable. Pcap is currently the best way to transfer a human actor’s authentic, full performance onto a 3D-modeled character.

 

Early motion capture and Pcap transcended the technique of rotoscoping, where animators traced over filmed footage. In recent years, Pcap has evolved to become more effective at a lower overall expense than ever before.

 

Performance capture technology allows media producers to combine the power and subtlety of actors’ performances with the unique artistic vision of each production’s CGI animation and VFX.

What are the challenges of performance capture?

Performance capture demands highly accurate motion sensors, making the process potentially expensive and complex. Acting without traditional sets or props might also the natural expressions of performers.

How is performance capture changing?

Advancements in technology are making performance capture more accessible, with improved sensors and real-time rendering creating even more realistic results. This evolution is empowering storytellers to bring more imaginative and immersive worlds to their audiences. 

See more