SAN FRANCISCO, CA – During its Game Developers Conference (GDC) keynote address, Epic Games unveiled “Siren,” a high-fidelity, full-performance character driven in real time.
Re-creating the subtle intricacies of movement can be the difference between a realistic digital re-creation and a trip into the uncanny valley, so Epic enlisted the real-time capture abilities of its partner Vicon, an industry leader in motion capture, to bring this project to life.
The real-time presentation displayed during Epic’s keynote was previously recorded on Vicon’s capture stage at its headquarters in Oxford, England. To create the video, actress Alexa Lee wore a full-body motion capture suit with a head-mounted camera. Using Vicon’s new Shōgun 1.2 software, her body and finger movements were captured on one screen while the data was streamed into Unreal Engine using Vicon’s new Live Link plug-in. On a second screen, the Siren character – created using the likeness of Chinese actress Bingjie Jiang - moved in sync, driven in-engine at 60 frames per second.
To ensure the highest possible fidelity, Vicon solved directly onto the Siren custom skeleton, removing the complex targeting step. It also developed a new algorithm to help realistically animate the fingers.
“When we began working on Siren, we knew from the beginning that it was going to push several boundaries. To make this possible we needed the best motion-capture hardware and software,” said Kim Liberi, Epic Games’ CTO. “We use Vicon systems on our own stage, so we knew right away that they were the best choice for this project.”
The Siren project began as a collaboration between Epic and Tencent, with the goal of creating a proof-of-concept demonstration to show both the capabilities of Unreal Engine 4, and what the next generation of digital characters will look like. To handle the performance capture for the character, Epic and Tencent utilized Vicon’s Vantage optical motion-capture system, Shōgun and VUE video cameras to capture precise and authentic movement and to add the character animations over the reference footage in real time.
Along with Vicon, Epic and Tencent turned to Cubic Motion to provide the facial performance capture, tracking, solving and animation. Cubic Motion also provided a live solver, taking the video feeds of the actress’ performance to drive the CG in real time. The character’s facial rig and underlying controls were both provided by 3Lateral, which also handled all of the 3D and 4D scans of the actress.
“To create a realistic digital character requires more than just tracking movements, you need to be able to isolate and mimic everything down to the most minute details,” said Tim Doubleday, Vicon VFX product manager. “There have been some incredible attempts to create realistic digital avatars recently, and so when Epic approached us, we were eager to get involved and help push the field even further.”