Face Robot - What is That?
Autodesk(R) Face Robot(R) is a new computer graphics software for digital acting, intended for professional character animators in the film and game industries. It addresses the technical problems of creating life-like facial animation for realistic human characters with a novel set of algorithms.
Face Robot is a new technology, so its scope and usage patterns are still evolving as we collaborate with customers. As Face Robot is used for more project, more information is provided to us about best practices and working methods. With your help, we are doing our best to outline the most up to date information on Face Robot as part of a standardized workflow that encompasses all steps from conceptual artwork and data acqusition to facial design and animation.
Life-like facial animation is arguably the hardest problem in computer graphics today and the most important thing to get right in computer animation. We are attuned to reading and evaluating facial features and expressions that any deviation from a truly lifelike representation is immediately obvious.
The human face only has one movable joint (the jaw), while everything else is based on deformations of soft tissue that are nearly impossible to capture with existing computer graphics tools.
Soft tissue simulation
Face Robot is the only system that realistically and consistently simulates how the soft tissue of the human face slides and deforms. Based on this soft tissue model, Face Robot allows artists to work at a higher level of abstraction. Instead of building shapes for each expression, the face can be manipulated directly through the salient features of facial anatomy, such as the corners of the mouth, the eyebrows, the jaw, etc. For motion capture animation, Face Robot requires a smaller number of markers (only 25 to 30) than traditional high-end approaches, yet provides high quality results because the markers drive a landmark-driven soft tissue model that fills in the gaps.
|Table of contents|
Face Robot Features
Soft Tissue Solver
- Automatic generation
- High quality deformation
- Works on a wide range of human heads
- Sculpt to define facial expression
- Wrinkle painting
- Connect skin stress to displacement and shading
- Symmetry tools
- Direct manipulation of facial controllers
- Visual animation interface
- Re-author and alter performances
- Reuse animation among different faces
- Point caching system to 3D applications
- SDK and multiple scripting languages
- Can be layered with shapes and facial rigs
- Imports C3D motion files
- Exports face rigs for games or shapes
- HD footage of actors performing facial expressions and phonemes
- Male and female examples
Who Does What with Face Robot?
- The face modeler creates a 3D polygon mesh for a humanoid head.
- A facial motion capture expert may acquire motion data for various performances.
- The actor tuning artist makes the head animatable.
- A facial rigging TD may add higher-level, studio-specific control systems.
- The digital acting animator applies animation to the head.
- A scene assembly TD / runtime assembly TD may then export the facial animation data for further processing or rendering.
How Face Robot compares to other options
Why is the face important?
Lifelike facial animation is arguably both the hardest problem in computer graphics today and the most important thing to get right in computer animation. The human face only has one joint (the jaw); everything else is based on deformations of soft tissue that are nearly impossible to capture with existing computer graphics tools. At the same time, we are so attuned to reading and evaluating facial features and expressions that any deviation from a truly lifelike representation is immediately obvious.
The approaches that studios are currently forced to take â€“ because soft tissue cannot be represented accurately â€“ require years of expertise and are so resource-intensive and time-consuming that truly life-like human facial animation is practically inaccessible for all but the most expensive productions.
In this context, FACE ROBOT matters for three primary reasons:
- The results look better. Because all facial poses are based on a consistent model of the face's soft tissue, the resulting expressions and their animations are more life-like and convincing.
- Artists are vastly more productive. Face Robot cuts the time required to take a model of a head to a fully animatable setup from weeks to hours. It cuts the time to build a library of facial expressions by orders of magnitude. It reduces the number of motion capture markers required for accurate capture by around 80%. It automatically generates life-like facial expressions that animate believably, thus eliminating most of the tedious tweaking required with traditional methods.
- It makes world-class facial animation accessible. Digital acting with emotive facial expressions is now feasible for almost any project. Faces can be animated in a fraction of the time for a fraction of the cost. Artists that know how to animate well no longer need years of specialized technical know-how and hundreds of tediously modeled shapes to create good-looking facial animation.
What is special about the FACE ROBOT technology?
Face Robot is the only system that simulates how the soft tissue of the human face slides and deforms in a realistic and consistent way. Based on this soft tissue model, Face Robot allows artists to work on a higher level of abstraction. Instead of building shapes for each expression, the face can be manipulated directly through the salient features of facial anatomy such as the corners of the mouth, the eyebrows, the jaw, etc.
For motion capture animation, Face Robot requires a much smaller number of markers (25-30) than traditional high-end approaches, yet provides significantly better results because the markers drive a self-consistent soft tissue model that fills in the gaps.
Also, for the first time, computer animators are able to take a 3D model of a humanoid head to a fully animatable state within hours (and in some cases minutes) instead of weeks, without requiring support from an experienced technical expert.
Because Face Robot addresses all of the most difficult technical challenges of animating a face, any good animator is empowered to create content with a level of quality demanded by today's feature films.
Does FACE ROBOT support motion capture?
Yes. In fact, Face Robot is almost perfect for motion capture. Instead of capturing a large number of markers that both overconstrain the face and create room for error, it allows productions to capture only those markers that represent the salient points of the face. The deformation of the face around these markers is driven by a soft tissue model that delivers believable, lifelike animation and removes the need for extensive cleanup and tweaking. Studios will save time gluing on markers, have much less data to clean, and will be able to get animation into projects sooner.
How was FACE ROBOT created?
Face Robot is the culmination of many years of animation research and development at Softimage. Softimage, founded in 1986, has been at the forefront of computer graphics technology for almost 20 years. The company has always been a pioneer, defining the state of the art in computer graphics; many fundamental animation technologies, such as inverse kinematics, first found their application in Softimage's 3D systems and enabled ground-breaking new results, such as the digital dinosaurs in Jurassic Park.
The technology for Face Robot was developed in secrecy by the Softimage Special Projects team and condenses numerous algorithmic advances and years of production experience with facial animation into the first coherent solution for facial animation. Led by Michael Isner, Thomas Kang, and Javier von der Pahlen at Softimage Special Projects, Face Robot was developed in close collaboration with one of the key partners in this project, Jeff Wilson at Blur Studio in Los Angeles, who helped define the artistic needs for creating life-like facial animation.
While Face Robot is designed as a separate product, it builds on and extends the advanced animation core of Softimage|XSI. Face Robot contains numerous new computer graphics algorithms that were specifically developed to solve the hardest problems in facial animation, but the proven geometry and animation core of XSI provides the solid ground required for this advance.
Face Robot also employs new user interface and software architecture extensions and a range of dedicated optimizations that were engineered into the core of SOFTIMAGE|XSI version 5.0 specifically for this purpose. Without dedicated optimizations, performed by the same team of engineers that was also responsible for the fastest implementation of subdivision surfaces in the industry, it would not have been possible to make the complex new algorithms available for direct manipulation and interactive use for artists.
What kinds of faces are supported by FACE ROBOT?
The technology at the core of Face Robot can be applied to any kind of face, be it realistic or cartoon human, animal or fantasy creature and the Softimage Special Projects team is available to assist productions that have specific needs.
In its current form, the software is optimized only for realistic human heads.
How does FACE ROBOT work?
Face Robot takes a new approach to facial animation based on a simulation of soft tissue. It is not based on shape libraries or a muscle system.
What is the workflow for FACE ROBOT?
Face Robot works in multiple stages that are relatively straightforward to use. See Overview of Working in Face Robot for the big picture.
Does FACE ROBOT support Lip-Synching?
Yes, but Face Robot is not a lip-synching tool. Face Robot co-exists with such tools and supports the lip-synching process by allowing artists to either create the required visime / phoneme shapes much faster (if exporting for other animation tools) or to directly reference poses that are driven by the soft tissue model, allowing for more believable transitions between states.
What can be achieved with FACE ROBOT?
Face Robot addresses many of the difficult technical challenges of facial animation through an elegant abstraction of facial anatomy and a model of how the soft tissue of the face deforms. It allows animators to work with the following aspects of the face:
- Blinking. Humans blink in many different ways; Face Robot lets animators apply several kinds of realistic blinking (fast, slow, tired, nervous) with a wide range of parameters.
- Focused gaze. The eyes move in synchronization to fixate objects of interest. They may also defocus to stare into empty space.
- Eye lid motion. As the eyes look up or down, the upper and lower eyelids move to make way and to provide an unobstructed view through the pupil.
- Corneal bulging. The eye ball is not completely spherical. At the cornea, the portion of the eye where the iris is visible, there is a noticeable outward bulge. As the eyes move around to look at objects, the corneal bulge deforms the eyelid around the iris and pushes it outward. In REM sleep, where the eyes move around under closed eyelids, the traveling corneal bulge is also visible.
- Eye bags. The soft area under the eyes is deformed by the orbicularis oculi (eye ring muscle) and by other muscles that contract tissue in the lower half of the face. They contribute to facial expressions and can indicate aggression and separate a fake smile from a genuine smile.
- Crow's feet. These wrinkles, radiating out from the corners of the eyes, are generated when the surrounding skin area is compressed. This happens, for example, through contraction of the orbicularis oculi muscle or the zygomatic major (smiling muscle).
- Mouth tissue deformation. The mouth is one of the most complex regions of the face, made up of muscles, fatty tissue and skin. As muscles push and pull the mouth to create various facial expressions, the skin deforms in a consistent way, preserving the physical integrity of the tissue.
- Effects on other areas of the face. All areas of the face are covered by and connected through a single, contiguous sheet of skin. Muscles attached at the mouth travel through and hence affect large areas of the face and in some cases even reach down to the chest area. When they contract, not only the mouth deforms, but rather through skin and muscle movements almost every area of the face is deformed and moved in subtle ways. In a smile, for example, skin tension on the cheeks is reduced and the ears are pulled slightly towards the back of the head; the cheeks bulge outward as skin and fatty tissue are compressed and slide over the skull.
- Lip deformation. Consisting entirely of soft tissue and equipped with complex muscle groups, the lips may be the most complex region of the human body in terms of deformation. They can pull and curl inward and outward, may be compressed or pushed forward and can move in almost any direction under muscular control.
- Extreme mouth states: certain expressions deform the lips in unusual ways that are different from simple widening or parting of the lips, such as snarling and pouting.
- Lip / skull adhesion. As the mouth deforms in ways that push the lips forward / outward, the lip tissue adheres to the gum / skull tissue and lips deform differently depending on the level of lip / skull adhesion.
- Corners of the mouth. The corners of the mouth are geometrically very complex, being effectively a three-dimensional intersection of the vertically curving surface of the lips and the horizontally curving surface of the cheek. Minuscule turns of the corner of the mouth, effected through minimal muscle contractions, can completely alter the meaning of an expression: a very slight smile can turn into a fake smile or a removed Mona Lisa smile.
- Nasolabial fold and commissural fold. These folds frame the mouth area and are affected by most movements of the mouth. They are important visual cues for reading facial expressions because their high-contrast shadowing can be made out even from a distance.
- Bulging cheeks. In a smile or sneer, the zygomatic major or levator labii superioris muscles, respectively, pile up tissue and result in bulges around the cheek bones and wrinkling around the eyes.
- Eyebrows. The eyebrows confirm and intensify expressions communicated by the eyes. Driven by muscle groups on the forehead, they can deform in complex ways and through exaggeration allow animators to create intriguing and unusual facial expressions. As the eyebrows move, the skin around them slides over the skull and a significant region of the face reaching as far down as the cheeks is affected.
- Brow wrinkles. As the frontalis, a muscle sheet covering the entire forehead, contracts, horizontal folds emerge across the forehead. These are important in communicating expressions like concern, terror or surprise.
- Frowning folds. The corrugator muscle, located just above the nose between the eyes, creates vertical frowning folds as it contracts.
- Nostril flaring. The nostrils can flare outward and upward as the face pulls into a sneer, or through voluntary muscle contraction.
- Nose ridge folds. In a sneer, some people develop folds that run horizontally or diagonally across the upper portion of the nose ridge and/or along the sides of the nose.
The Jaw and Neck
- Jaw motion. The human jaw has a wide range of motion: it can rotate open and closed, slide in and out and move side to side. As the law opens, the lips change shape, the skin on the cheeks stretches, the ears move and the fatty tissue below the jaw deforms.
- Neck motion. The frontal portion of the neck changes shape in a variety of ways. As the head rotates or tilts, the tissue is compressed or stretched and creates bulging and other deformations in the neck portion below the jaw and on the sides of the neck.
- Cheek deformation. As the jaw opens and then possibly moves side-to-side or in and out, the cheeks stretch and deform in complex ways as their tension increases and decreases, as tissue slides over bone and as muscles in the skin contract and relax.
- Neck tendons & muscles. As the neck moves and changes shape, tendons that connect the head to the chest and shoulders become visible under the skin and emerge by pushing outward and forming ridges. During animated speech and in more extreme facial expressions, the platysma, a thin sheet of muscle that extends downward from the jaw, tenses the skin and raises thin, vertical wrinkles.
- Adam's apple. During swallowing and during speech, the Adam's apple will slide up and down along the trachea create a visible bulge. When the head turns or tilts or when the jaw moves, the Adam's apple is also visible (particularly in men) as the skin of the neck slides over it.
- Multi-vertebrae spine. Facial expression is supported and accompanied by a wide range of movement of the head relative to the upper body and shoulders. Representing this motion requires the upper vertebrae of a fully poseable spine.
- Cheek puffing. Chewing with a full mouth, inflating the mouth with air or pushing the tongue against the sides of the mouth lets the cheek tissue bulge outward.
- Skin sliding. Some aspects of facial deformation are visible only in moving animation, not in stills. As a real face deforms, for example, skin slides in a very specific way across the face, connecting the areas of the face into one consistent whole. An accurate representation of this behavior contributes to a truly lifelike look.
- Ears. Ears slide over skull, pulled by skin as facial muscles contract.