Now for the Animation
Contents

Decisions, Decisions

Welcome to Muscle Beach

Now for the Animation

I finally have my system set up and my models created. It is time to create some real-time animation. The time-tested animation production method is to take a track of audio dialog and go through it, matching the visemes in your model set to the dialog. Then, in second pass, go through it and add any emotional elements you want. This, as you can imagine, is pretty time consuming. Complicating the matter is that there are not many off-the-shelf solutions to help you out. The job requires handling data in a very special way and most commercial animation packages are not up to the task without help.

Detecting the individual phonemes within an audio track is part of the puzzle that you can get help with. There is an excellent animation utility called Magpie Pro from Third Wish Software that simplifies this task. It can take an audio track and analyze it for phoneme patterns you provide automatically. While not entirely accurate, it will at least get you started. From there you can manually match up the visemes to the waveform until it looks right. The software also allows you to create additional channels for things such as emotions and eye movements. All this information can be exported as a text file containing the transition information. This in turn can be converted directly to a game-ready stream of data. You can see Magpie Pro in action in Figure 6.

Figure 6. Magpie Pro simplifies the task of isolating
phoneme patterns in your audio track.

Wire Me Up, Baby

With all the high-tech toys available these days, it may seem like a waste to spend all this time hand-synching dialog. What about this performance capture everyone has been talking about? There are many facial capture devices on the market. Some determine facial movements by looking at dots placed on the subject’s face. Others use a video analysis method for determining facial position. For more detailed information on this aspect, have a look at Jake Rodgers’s article “Animating Facial Expressions” in the November 1998 issue of Game Developer . The end result is a series of vectors that describe how certain points on the face move during a capture session. The number of points that can be captured varies based on the system used. However, typically you get from about eight to hundreds of sensor positions in either 2D or 3D. The data is commonly brought into an animation system like Softimage or Maya and the data points drive the deformation of a model. Filmbox by Kaydara is designed specifically to aid in the process of capturing, cleaning up, and applying this form of data. Filmbox can also apply suppressive expressions, inverse kinematic constraints, and perform audio analysis similar to Magpie Pro.

This form of motion capture clearly can speed up the process of generating animation information. However, it’s geared much more toward traditional animation and high-end performance animation. In this respect it doesn’t really suit the real-time game developer’s needs. It’s possible to drive a real-time character by using the raw motion capture data to drive a facial deformation model. However, for a real-time game application, I do not believe this is currently feasible.

In order to convert this stream of positional data into my limited real-time animation system, I would need to analyze the data and determine what visemes and emotions the performer is trying to convey. You need a filtering method that will take the multiple sample points and select the viseme or muscle action that is occurring. This is really the key to making motion capture data usable for real-time character animation. This area of research, termed gesture recognition, is pretty active right now. There is a lot of information out there for study. However, Quantumworks’ Geppetto provides gesture recognition from motion capture data to drive “muscle macros” as both a standalone and a plug-in for Filmbox.

Where Do We Go from Here?

Between viseme-based and muscle-based facial animation, you can see that there are a lot of possible approaches and creative areas to explore. In fact, the whole field has really opened up to game development in terms of opportunities for game productions as well as tool developers. Games are going to need content to start filling up those new DVD drives and I think facial animation is a great way to take our productions to the next level.

For Further Information

• Ekman, P. and W. Friesen. Manual for the Facial Action Coding System. Palo Alto, Calif.: Consulting Psychologist Press, 1977.

• Faigin, Gary. The Artist’s Complete Guide to Facial Expression. New York: Watson-Guptill Publications, 1990.

• Goldfinger, Eliot. Human Anatomy for Artists. New York: Oxford University Press, 1991.

• Landreth, C. “Faces with Personality: Modeling Faces That Exude Personality When Animated.” Computer Graphics World (February 1996): p. 58(3).

• Waters, Keith. “A Muscle Model for Animating Three-Dimensional Facial Expression,” SIGGRAPH Vol. 21, N. 4 (July 1987): pp. 17-24.

Facial Animation
http://mambo.ucsc.edu/psl/fan.html

Gesture Recognition
http://www.cs.cmu.edu/~face

Performance Animation Society
http://www.pasociety.org/

Magpie Pro
http://thirdwish.simplenet.com/

Filmbox
http://www.kaydara.com/

Geppetto
http://www.quantumworks.com/

Acknowledgements

Thanks to Steve Tice of Quantumworks Corporation for the skull model and the use of Geppetto as well as insight into muscle-based animation systems. The W. C. Fields image is courtesy of Virtual Celebrity Productions LLC (http://www.virtualceleb.com/) created using Geppetto. The female kiss image is courtesy of Tom Knight of Imagination Works.

When not massaging the faces of digital beauties or doing stunt falls in a mo-cap rig, Jeff can be found flapping his own lips at Darwin 3D. Send him some snappier dialogue at jeffl@darwin3d.com.

________________________________________________________

[Back to] Decisions, Decisions