THE LATEST PLANET OF THE APES FILM HAS CAESAR AND HIS SPECIES AT WAR WITH HUMANS LED BY A NASTY COLONEL.
In 2011, Weta Digital started a digital revolution by creating the lifelike Caesar and other non-human primates for the rebooted Planet of the Apes film
Rise of the Planet of the Apes using state-of-the-art motion- and performance-capture methods, new interactive fur grooming tools, and more. The studio continued the trend in
Dawn of the Planet of the Apes (2014), which was filmed mostly on location, requiring a mobile performance-capture system for the extreme locales and weather, as well as a rebuilt hair and fur system.
Once again, Weta upped the ante in this summer’s War for the Planet of the Apes, the third in this rebooted series, which features more of everything. A more expressive cast of characters. More detailed fur grooms. More complex performance capture in even more extreme weather conditions, including snow, as it interacts with the digital fur. More complicated environments. More simulations and effects.
“Evolutionary” is a word that easily comes to mind when describing the film. And just as the apes’ level of sentience has developed throughout each film, so, too, have Weta’s techniques and tools to support the CG cast.
Directed by Matt Reeves, who also directed Dawn,
War finds Caesar and his band locked in a deadly conflict with an army of humans led by a ruthless colonel (Woody Harrelson). After the apes suffer staggering losses, Caesar wrestles with his darker instincts and begins a quest to avenge his kind. As the revolution comes full term,
Caesar and the colonel face off in an epic battle that will determine the fate of both their species and the future of the planet.
Also returning is Weta Digital’s Dan Barrett, animation supervisor, and Dan Lemmon, VFX supervisor, both of whom worked on all three rebooted Planet of the
Apes.
Likewise, a number of the digital cast (and their performance actors) returns as well, including Caesar (Andy Serkis), Rocket (Terry Notary), Maurice (Karin Konoval), Cornelia (Judy Greer), and others. Some new additions include Bad Ape (Steve Zahn), Rex (Ty Olsson), Luca (Michael Adamthwaite), and Winter (Aleks Paunovic).
“We always finish a movie having learned a great deal and become excited about taking what we learned and applying it to new shows. But it is especially exciting when you can come back and work with some of the characters you are familiar with, that you love, and are able to put more energy and attention into them,” says Lemmon. “Also, our core technology continues to advance. We are always thrilled to be able to make updates and tell a new story, to put the characters into a new space and get them to look even better by improving their fur and the way the light hits them. In this movie, in particular, we beat up Caesar pretty badly; he takes a number of cuts and scrapes and hits. The variation in textural detail and changes over the course of the movie far exceed what we did on previous films.”
(TOP) NEW TO THE SERIES IS BAD APE (STEVE ZAHN). (BOTTOM) CAESAR (ANDY SERKIS) RETURNS IN HIS LEAD ROLE.
There are approximately 1,450 visual effects shots in the film – many a minute or more in length – all which were handled by Weta, with the exception of some marker removals, paint-outs, the addition of blood (by Blur Studio), and previs/postvis (Halon).
“The movie runs just over two hours, and almost every shot has visual effects,” says Lemmon. “The apes are in almost every shot. There’s also so much in terms of worldbuilding.”
Facial Makeover
It goes without saying that the Weta facial animation team is well acquainted with these characters and has a deep understanding of them and how to get them to express their emotions. “We are very clear that the success of the film hinges on us matching the performance of the actors. It’s incumbent on us to make sure their performance comes through,” says Barrett.
The updates to the base models of the recurring characters were subtle and mainly extended to the aging of Caesar – deeper wrinkling and added creases in his face, more gray in his hair and beard – before the addition of scars of war. And just as they did on the previous Apes films, the artists continued to adjust Caesar’s model and facial rig based on what they had learned on the previous production and the ever-growing complexity of the performances and facial expressions.
“This movie contains some of the finest facial performance for Caesar that we have ever done on a digital character. He goes places he hasn’t gone before emotionally, and Andy Serkis put in a stellar performance; he played the role with such sophistication and subtlety,” says Lemmon. “There is a lot of conflict in this film, and Caesar goes through a lot of internal struggle. For us, Andy set a high bar as to what Caesar could do and emotionally where he would go. We had to get all that complicated overlapping of various emotions and layers of subtext to come across in Caesar’s face.”
And just as the requirements for believable emotional digital performances grew, so have Weta’s facial animation tools to support that work. Weta still used the same single-camera facial-capture setup for War as it had for the previous films. They also used a similar way of building the face puppets, as many were reused as a base starting point for this film.
In the past, though, the team built the faces and heads in Autodesk’s Maya, but the evaluation was slow. For War, Weta devised a new way to build the puppets outside of Maya in software called Koru. That, combined with Weta’s own custom viewport within Maya, called Gazebo (built upon a fast OpenGL platform), gave the group incredible speed for executing complex and precise facial animation decisions. Receiving instant feedback resulted in a more intuitive direct manipulation model, giving animators the power and control to re-create any expression they saw in an actor’s performance. And this extended beyond the hero character to the hundreds of characters in a scene.
“We got some good speed improvements and were carrying the full-resolution, full face puppets in the scenes, which is a great help when you can see your animation and be confident that what you see is what will be rendered ultimately,” says Barrett. “I think we were above real time even with two characters on screen, displayed at full resolution.”
THE STRUCTURE OF BAD APE’S FACE MADE ARTICULATION A CHALLENGE.
Despite the new main characters, both Lemmon and Barrett point to Caesar as still being the most challenging character in terms of the facial animation. “There is something about the openness and the simplicity of Caesar’s face that can make it more challenging to get readable, pleasing performances than with Winter or Bad Ape,” says Barrett. “The cragginess and detail in their faces make it almost easier. But with Caesar, there is almost no place to hide. Yeah, it was Caesar once again.”
Furthermore, Caesar speaks more in this movie, requiring the lip sync and articulation of the phonemes to be believable yet still resemble an animal rather than a human.
Articulation was a concern for the old, gentle, wise yet naïve Bad Ape, as well. He speaks a lot in the film, and does so quickly while using his eyes to express himself. In early tests, he appeared too articulate. Even though his mouth would hit the same shapes that Zahn’s mouth hit, the chimp-like character’s muzzle was quite large so it appeared as if he was talking too fast or the movements were too large. The animators had to figure out how far to push him and where to dial it back.
“Most of our other key characters we already knew. But we had to make sure the new characters were brought up to a level so they could perform toe-to-toe with someone like Caesar,” says Lemmon.
Animation Improvements
The story point that these apes have become more evolved, more bipedal, made animation somewhat easier. As Barrett notes, for new actors, it is easier to learn to mimic bipedal versus quadrupedal motion. Thus, the animators had less tweaking to do on the overall mocap data, even for the battle scenes, which required some of the more physically powerful movements from the apes.
“The animators would do more keyframing, but across the board, we were able to keep more of the mocap,” says Barrett. “Our on-set motion-capture team did an amazing job.” This despite the extreme “elemental” conditions on set: rain, wind, and snow.
Also, the team introduced NaturalMotion’s Euphoria physics animation engine, typically used in video games, into the pipeline. As Lemmon points out, it’s difficult to make a body physically dynamic while fighting against a fall. Weta does a lot of motion capture and stunt work to get believable movements and falls, but the data has to be edited to fit the terrain or context where it will be used.
“Euphoria does a great job of combining motivated character motion with physical dynamics,” says Lemmon. “Through skillful use of the tool, animators could create motion that was believable and hit story points but was directable and also felt totally dynamic and physically accurate.”
The software proved useful not just during a fall, but afterward as well, as an injured ape struggled to stand up, for instance.
Weta also looked to improve on the horses, adding far more detail to the digital cavalry, which included quarter horses, Friesians, and Andalusians. Then they were groomed with millions of very short hairs, making their pelts more realistic than they were in Dawn.
The animators further improved the dynamic muscle system. “The horses are so lean, you see their muscles jiggling and moving even as they are walking gently down a path,” says Lemmon.
The real horses used on set proved to be excellent reference.
Approximately half the horses in War are from a plate, while the rest are CG. “In some instances, we are handing off from real horses to CG horses, and vice versa, so ours needed to look and perform like the real ones,” says Lemmon. “They had to be perfect matches.”
To animate the CG horses, Weta used an extensive library of animations when possible, otherwise the movement was keyframed.
IN WAR, HALF THE HORSES ARE CG, THE OTHER HALF TAKEN FROM PLATES. FOR THIS FILM, WETA ALSO IMPROVED ON THE CG HORSE MODELS AND THE DYNAMICS BETWEEN THE HORSES AND RIDERS.
For this film, the team was determined to achieve more intuitive dynamics for the apes on horseback. In Dawn, crews captured the performance of actors atop horses with saddles and stirrups, which was translated onto apes riding bareback. “That motion tended to look a bit off because the actors were taking weight, even minimally, on their feet, and that was not translated on the screen,” says Barrett. Evolution, however, helped resolve this problem in
War: The apes have evolved in their relationship with horses and have developed their own tack. As a result, the captured motion translated more intuitively onto the CG apes.
Fur-Real
Perhaps one of the biggest improvements on this film pertains to the fur grooms. “We pushed our fur technology quite a bit between Dawn and
War,” says Lemmon, noting much of that resulted from work Weta did for
The Jungle Book on King Louie and the monkeys. “That gave us a chance to push our grooming tools to the next level.”
The artists did a re-grooming pass on all the recurring characters. In terms of volume, Lemmon estimates that most of the characters have two to three times as many strands of hair now as they did in Dawn. Although the count varies from character to character, Caesar has close to a million hairs now; some of the denser--covered cast has between three million and 10 million hairs.
“We are also clumping and styling the hair in a way that is more realistic, closer to what we see in nature” Lemmon says.
More complicated environmental conditions introduced new levels of interaction and visual complexity as to how the fur behaves. In addition to dirt and rain, harsh winter wind and snow (freshly fallen flakes, melting snow, and so forth) have been added to the mix. These elements needed to realistically alter the texture and color of the fur and integrate into the simulation of the individual hair movement.
“Snow on fur is something we hadn’t really done before,” says Lemmon. ”The apes walk through the snow. There’s falling snow that sticks to their fur, clumps, and falls off. They roll around in the snow in one sequence. Having that interaction between the snow and fur, and transferring the snow from the air, ground, and back onto the character’s fur was a big deal that forced a change in our fur grooming system and our dynamics system.”
As a result, Weta rewrote sections of its hair engine, Wig, to allow it to integrate more seamlessly with the studio’s simulation software, Synapse. As a result, data could be passed back and forth between the two systems to create more sophisticated styling tools for the fur. “It wasn’t just about getting dynamics in participating media like snow onto the fur, but was also about using some of the simulation tools to isolate, style, and attach parameters to the hair in different ways,” Lemmon explains. “It gave us more flexibility in our core styling tools and in our simulations.”
Lighting conditions ranging from sterile winter exteriors to warm flame-lit cave environments also demanded realistic--looking fur. As a result, Weta moved away from Pixar’s RenderMan, which was mostly used for Dawn, and rendered
War entirely through Manuka, the studio’s new ray-tracing engine. By doing so, the artists can model the transport of light through the hairs and the shadowing of the hair back onto the skin far more accurately and realistically.
TDs also rewrote the shader.
“We moved to a dual-core model. In addition to modeling the hair strand with all the scales and the way the light breaks up, we model the cuticle and the medulla, so there is an inner core to the hair,” explains Lemmon. “That gives us a lot more realistic transfer of light through the hair, and we get a lot better glint and breakup, particularly in backlighting situations. That is what we struggled with in Jungle Book.”
WETA REVAMPED ITS FUR GROOMING TECHNOLOGY. AS A RESULT, THE CHARACTERS HAVE ON AVERAGE THREE TIMES MORE HAIR THAN THEY HAD FOR DAWN. CAESAR (ABOVE) HAS CLOSE TO A MILLION HAIRS IN WAR.
Moreover, a system built on top of Manuka, called PhysLight, enables the artists to accurately model the way the cameras pick up and respond to light. As a result, they can place the apes in the real world and treat them like actors, lighting them as a DP would light a set. “We can accurately match the lighting information that we capture on set and reproduce the way it interacts with the cameras in the same way that the film camera does, so we can very quickly and very accurately get the same response from our CG characters as we saw on set,” says Lemmon.
CG Sets and Environmental FX
War features a number of all-digital shots, along with numerous CG set extensions. One of the largest fully CG locales is the interior of the Hidden Fortress, a huge, hollowed-out cave behind a computer--generated waterfall – a complex setting that also required a great deal of finesse in terms of simulations, lighting, and character interactions.
Another complex environment is the prison, an especially large model that was very nuanced and detailed in its construction. Crews started with a practical prison set just outside of Vancouver, Canada, surrounded by greenscreens, and continued to extend it digitally. Then, the industrial-like backdrop of Greater Vancouver was digitally replaced to situate it in the Sierra Nevada.
In the final face-off between Caesar and the colonel, the world is exploding outside, requiring a good deal of effects work, as squadrons of Apache helicopters crash to the ground. There’s gunfire and missiles, all of which culminates in a huge avalanche, as the snow pack in the hills above the prison is loosened and comes raining down. Trees are pulled up by their roots. Clouds of snow race down the mountain.
“There’s a lot of research that’s been put into the avalanche in terms of what causes it physically and how to best replicate it from an R&D standpoint,” says Lemmon. “We are harnessing a lot of state of the art in terms of fluid and rigid simulations to rain down what starts as a fast-moving slab or cornice falling away quickly, turning into a giant cloud of snow that just rockets down the mountain.”
This scene required a heavy simulation and a number of new tools. “Sometimes the simulation ran for about three days on 30 machines,” says Lemmon. In some instances, the emitters were driven by particle simulations, some of which were created with external tools, either Maya or Side Effects’ Houdini. Then, the result was fed back into Synapse.
Weta also wanted to improve upon the explosions. “The film is called War for the Planet of the Apes, so we knew there was going to be explosions and pyrotechnics,” says Lemmon. He decided to take a page from the special effects handbook and incorporate similar methods into the Synapse simulations in terms of how the charges were broken up and the various components needed to produce different looks.
Parting Shot
Barrett and Lemmon have gotten to know these characters well over the span of the three movies. “Bringing these characters to life never stops being fun,” says Lemmon. But, also a bit sad.
“There were some shocking moments and things that, as friends of these apes, we had to steel ourselves for,” says Barrett.
When asked whether he developed an emotional attachment to some of the characters, Barrett doesn’t hesitate. “Absolutely. You can’t not, really. I feel quite intimately involved with these characters and found some of those moments, especially at the end of the film, quite wrenching,” he says. “It’s an end of an era.”
Barrett continues: “I feel like this is my gang, or at least I am part of their gang, and that is probably the reason why I went a third time on the series. I couldn’t say no to going into battle with these apes again.”
Karen Moltenbrey is the chief editor of CGW.