Now
for the Animation
I finally
have my system set up and my models created. It is time to create some
real-time animation. The time-tested animation production method is to
take a track of audio dialog and go through it, matching the visemes in
your model set to the dialog. Then, in second pass, go through it and add
any emotional elements you want. This, as you can imagine, is pretty time
consuming. Complicating the matter is that there are not many
off-the-shelf solutions to help you out. The job requires handling data in
a very special way and most commercial animation packages are not up to
the task without help.
Detecting the individual phonemes within an audio track
is part of the puzzle that you can get help with. There is an excellent
animation utility called Magpie Pro from Third Wish Software that
simplifies this task. It can take an audio track and analyze it for
phoneme patterns you provide automatically. While not entirely accurate,
it will at least get you started. From there you can manually match up the
visemes to the waveform until it looks right. The software also allows you
to create additional channels for things such as emotions and eye
movements. All this information can be exported as a text file containing
the transition information. This in turn can be converted directly to a
game-ready stream of data. You can see Magpie Pro in action in Figure
6.
|
Figure 6.
Magpie Pro simplifies the task of isolating phoneme patterns in
your audio track. |
Wire Me Up,
Baby
With all the high-tech toys available these days, it
may seem like a waste to spend all this time hand-synching dialog. What
about this performance capture everyone has been talking about? There are
many facial capture devices on the market. Some determine facial movements
by looking at dots placed on the subject’s face. Others use a video
analysis method for determining facial position. For more detailed
information on this aspect, have a look at Jake Rodgers’s article
“Animating Facial Expressions” in the November 1998 issue of Game
Developer . The end result is a series of vectors that describe how
certain points on the face move during a capture session. The number of
points that can be captured varies based on the system used. However,
typically you get from about eight to hundreds of sensor positions in
either 2D or 3D. The data is commonly brought into an animation system
like Softimage or Maya and the data points drive the deformation of a
model. Filmbox by Kaydara is designed specifically to aid in the process
of capturing, cleaning up, and applying this form of data. Filmbox can
also apply suppressive expressions, inverse kinematic constraints, and
perform audio analysis similar to Magpie Pro.
This form of motion capture clearly can speed up the process of
generating animation information. However, it’s geared much more toward
traditional animation and high-end performance animation. In this respect
it doesn’t really suit the real-time game developer’s needs. It’s possible
to drive a real-time character by using the raw motion capture data to
drive a facial deformation model. However, for a real-time game
application, I do not believe this is currently feasible.
In order to convert this stream of positional data into
my limited real-time animation system, I would need to analyze the data
and determine what visemes and emotions the performer is trying to convey.
You need a filtering method that will take the multiple sample points and
select the viseme or muscle action that is occurring. This is really the
key to making motion capture data usable for real-time character
animation. This area of research, termed gesture recognition, is pretty
active right now. There is a lot of information out there for study.
However, Quantumworks’ Geppetto provides gesture recognition from motion
capture data to drive “muscle macros” as both a standalone and a plug-in
for Filmbox.
Where Do We Go from
Here?
Between viseme-based and muscle-based facial animation,
you can see that there are a lot of possible approaches and creative areas
to explore. In fact, the whole field has really opened up to game
development in terms of opportunities for game productions as well as tool
developers. Games are going to need content to start filling up those new
DVD drives and I think facial animation is a great way to take our
productions to the next level.
For Further
Information
• Ekman, P. and W. Friesen. Manual for the Facial Action Coding
System. Palo Alto, Calif.: Consulting Psychologist Press,
1977.
• Faigin, Gary. The Artist’s Complete Guide to
Facial Expression. New York: Watson-Guptill Publications,
1990.
• Goldfinger, Eliot. Human Anatomy for Artists.
New York: Oxford University Press, 1991.
• Landreth, C. “Faces with Personality: Modeling Faces
That Exude Personality When Animated.” Computer Graphics World
(February 1996): p. 58(3).
• Waters, Keith. “A Muscle Model for Animating
Three-Dimensional Facial Expression,” SIGGRAPH Vol. 21, N. 4 (July
1987): pp. 17-24.
Facial Animation
http://mambo.ucsc.edu/psl/fan.html
Gesture Recognition
http://www.cs.cmu.edu/~face
Performance Animation Society
http://www.pasociety.org/
Magpie Pro
http://thirdwish.simplenet.com/
Filmbox
http://www.kaydara.com/
Geppetto
http://www.quantumworks.com/
Acknowledgements
Thanks to Steve Tice of Quantumworks Corporation for the skull
model and the use of Geppetto as well as insight into muscle-based
animation systems. The W. C. Fields image is courtesy of Virtual Celebrity
Productions LLC (http://www.virtualceleb.com/)
created using Geppetto. The female kiss image is courtesy of Tom Knight of
Imagination Works.
When not
massaging the faces of digital beauties or doing stunt falls in a mo-cap
rig, Jeff can be found flapping his own lips at Darwin 3D. Send him some
snappier dialogue at jeffl@darwin3d.com.
________________________________________________________
[Back
to] Decisions,
Decisions