A look at animation in video games: why motion capture is the start, not the end, of the work
When a member of the press talks about the making of a video game, we’re usually interviewing the director, or the designer, or sometimes the writer of the game. There are a few cases here and there when we talk to the composer who scores the game. Past that, things get murky, and we rarely sit down with the people who go to work every day to create the games we play. I spoke with animator Ryan Duffin over Twitter once or twice before I realized I had no idea what an animator actually does for a living. So what the hell? I thought I’d ask. “It’s strange how it gets confused sometimes,” he said. Duffin has worked as an animator on games like Alan Wake, Ultimate Spider-Man, Battlefield 3, and is now working on Medal of Honor: Warfighter. “Sometimes I tell people I’m an animator and they say oh, so you model? You make the models? I say no, I animate. So you do the programming? No, that’s a programmer, I make the animation.” Okay, but what does that mean?
The before and after
“I get characters or a character in a T-pose, and then I pose them over time to bring them to life,” he explained. To put it in perspective, he could show me the “before” section of his work as a screen shot, but the “after” would have to be a video of a character actually moving. The screenshot and video embedded in this story show some of his personal work that illustrates how models land on his desk, and what the scene looks like once he’s done with it. Before an animator can do his thing, a model must be rigged and skinned. “Rigging is the process of creating a skeleton and setting it for an animator to be able to control the character with it. A quality rig allows an animator to not think too much about the tools they're using and focus on creating a performance,” Duffin explained. “Skinning is typically a part of the rigging process and involves binding the character model's mesh to the rigged skeleton in a way that deforms realistically through a full range of motion when the animator starts moving it around. This is extra important for games; in film you can fudge a character's mesh and skinning shot-for-shot to make them look right for the camera, but in games you get one skin that has to work for everything. That’s the animator’s job: to make things move in a way that’s interesting to watch. If other people create how a character is going to look, the animator creates how the character is going to move, which helps to create a sort of virtual performance. Good animation is believable, great animation can bring a character to life.
Motion capture, and the hidden animators
Wait, didn't motion capture remove the need for animators? You just put the goofy suit of ping pong balls on an actor, they act out the scene, you put the character’s skin over the performance, and you’re done! Simple. “That is the misconception of misconceptions, although it’s not exclusive to games,” Duffin said. In fact, an animator from 5TH Cell wrote an open letter to Andy Serkis explaining all the work that animators put into making Serkis’ performances as different characters believable. If the actor inside the suit deserved an Academy Award, the animator argued, the person who made sure that performance worked from a mechanical standpoint also deserved an award. “There is an enormous amount of data to clean up. In some films, and films that Andy Serkis in particular has been involved in, there are animators on those movies, and they don’t just export and plug in. For one thing, when you have a motion capture actor, they are never the same skeleton as your character in the game. So that’s a big thing, and that’s just because human beings are all unique. Nathan Drake’s build and Nolan North’s build, I assume Nolan North did performance capture for [the game], are not exactly the same.” The animator’s job is to take the fluid motion from the motion capture session and interpret that performance and make it work for the character model of the human, animal, or monster that will appear on the screen. “The integrity of motion changes, and the intent of the performance can too. There’s compression, there are errors that can happen, you can have bones flipped out and things like that and someone has to clean that up.” Duffin explains that with a good camera and performance you have the “tracing” of motion, which is not that good of a drawing. The animator’s job is to use that information to create a believable, complete performance using the skeletal structure of the final character.In the case of a movie like Rise of the Planet of the Apes, Andy Serkis created a wonderful performance with his body, but then those movements had to be moved to the model of the character, and that model had the proportions and skeletal structure, not to mention the facial structure, of an ape. This isn't a challenge that's unique to Andy Serkis films, but he's the most recognized motion capture “star,” and the behind the scenes materials for his films tend to gloss over the work of the animators who bring these characters to life. “It takes an animator’s eye to take that performance and make certain parts of it snappier or read better from whatever distance the shot is at, and things like that,” Duffin said. “Motion capture is the start of the process, certainly not the end. But yeah, there are people who say it’s 100 percent them in that suit and it goes 1:1, and that’s certainly not true. Not for games, and not for film.” It’s easy to see why people are misinformed about how motion capture works when almost every behind the scenes explanation of motion capture shows the performance on one side of the screen, and the finished, beautiful character on the other. It looks easy, but between those two images are animators who make sure the movements of the actor look good when performed by the skeleton and muscles of the characters. The problem is that it’s not sexy to show an animator sitting in front of a computer screen with a cup of coffee, making sure every frame looks believable. “Behind the scenes stuff is typically promotional, they make everything look easy,” Duffin said. “There are plenty of monsters and things like that, and you don’t want a troll from Lord of the Rings to move like a guy in a suit, even if they started with a guy in a suit, which is valid and they probably did, but it’s a starting point. It doesn’t allow you to press the animate button and ‘bam, I’m done!’” Duffin was resistant to motion capture when he worked with it for the first time. “I said we should be key framing this, it would be better, and I struggled with that at first, because I didn’t want to be doing it, and I thought that’s what game animation would turn into, just a lot of tracing. It wasn’t until my next job when I started key framing some super-realistic motion and I thought wow, this is taking a long time, and we’re going for super-realism, why not just motion capture it? And then the light bulb went on.” As a foundation for a realistic performance, motion capture is a wonderful tool that changes how animation is done, but it’s far from replacing it. “If we’re working on like Skylanders, or Okamior something like that, we’re going to animate it, or monsters or impossible feats and stunts. Sometimes it’s better to do key frame animation than motion capture wirework. It just depends. The motion capture hump was a big challenge for me, but I got over it.”
How to get started
“At this point, there is so much free schooling out there. I don’t want to say go to school or don’t go to school… but I know a lot of people who have taught themselves, and if you’re crazy motivated, the information is out there.” Before you think about college, it's important to teach yourself the basics. “Don’t even worry about software, people get hung up on that. They say they know 3D Studio Max, so they know how to animate. Well I can move clay around, but that doesn’t make me a sculptor.” It’s important to watch both live action and animated movies and pay attention to what makes a performance good. Play video games and notice how the characters move. Take martial arts classes so you can learn how bodies move and react to each other and be aware of your own body and muscles. “A punch isn’t just sticking my fist forward, it’s sticking my whole body forward, and my fist is just at the end of it,” Duffin said. “Subtley is a big thing, the biggest mistake a lot of young animators make is it over-animate stuff, and you see it in films that are aimed towards kids. The characters are really frenetic and just bouncing all over the place and striking a pose on every sentence they say. It’s really over done. But subtlety and nuance, that’s what gives things life. Knowing when to overact and to tone it down a little. There are great acting pieces in live action film where the actor is just sitting there and speaking, and emoting and barely moving at all and it’s amazing performance. An animator needs to know when that’s there.” So how important is it to be able to draw with a pencil and piece of paper? “It never hurts. The better artist you are, as a foundation, then the better artist you’ll be when you get into the particulars of it. Being able to express an idea on paper is super-valuable. You’ll never be able to prototype something on a computer as fast as you can thumbnail it out. If you can’t draw at all, that’s pretty bad. That being said, I know a lot of 3D animators who aren’t that great at drawing, myself included, that can draw enough to [share an idea]. You have a leg up if you can draw. If you’re getting into this, yes, you want to learn to draw as good as you can.” Duffin doesn’t draw on a day-to-day basis, but on Ultimate Spider-Man there were fight scenes that he drew out quickly to make sure the ideas worked well visually before committing the work to designing it into the computer. He also pointed out an early demo from Bayonetta that showed an animator creating key poses to show how the game would look like in action. You can see the key frames of each animation, although the frames in between each action have yet to be added to the scene. This shows how important it is to get animators working on games in the early stages of production; even this rough idea of animation in the game shows how striking the final movement will look.You also have to be flexible. “There’s not a standard process. If you go into a film studio, you have a pretty good idea about how things are going to work and there are differences but at least most people have worked for all the studios anyway, where in games you might be working with people who worked at the same studio for five or ten or fifteen years and they’re used to doing things a different way,” Duffin explained. On some projects he had a high level of autonomy and worked directly with the designers, and on others he was working closely with the art director who had very specific ideas for animation. He listed some of his favorite work, including around half the reloading animations for Battlefield 3. That’s an animation that players are going to see thousands of times as they play the game, and he was also keenly aware that there is a large community of gun owners who would pick the work apart; there is a right and wrong way to animate that sequence. The comic panel work of the cinematics in Ultimate Spider-Man was also a favorite. “That was a lot of fun, the comic book style of it and everything. I was on the cinematics team and we did the comic book panel stuff. It was cool to be a part of that, we won an Annie award for that.” The animations in the Uncharted series and Max Payne 3 also drew praise from Duffin as examples of games that make the character feel like they're a part of the environment.
What can you do today to begin experimenting with animation? “Scirra's Construct 2, I really like this one, Stencyl and GameMaker are great for simpler 2D games where UDK and Unity are a great place to start with 3D games,” he said. Pay attention to how characters move from a standing position to running, or jumping. Experiment with ways to make it look better. Think about the information given to the player through a character's actions and movement. “It’s overwhelming how much information is out there now about how to make animation and how to make games,” he said. “I would say if you’re 14 and you want to get into this, just stay super-focused and realize you’re going to be competing with a lot of people. It’s not going to be easy.” You can follow Ryan Duffin on Twitter, and listen to the ReAnimators podcast