Blending the Lip-Sync Animation with Other Animation

Lip-sync animation is implemented as a special type of action clip in the mixer, but in every other respect it acts like a normal action clip and can be treated as one as far as mixing goes.

• You can blend the weight of lip-sync animation in the speech clip with motion capture and/or keyframe animation that is on the same animation controls.

• You can add action clips of head movements, eye blinking, etc. to the animation mixer on different tracks, then have that animation running at the same time as the lip-sync animation that’s in the Speech action clip.


Blending Lip-Sync Animation with Mocap

To blend lip-sync animation with motion capture data, you can start with either type of animation and then add the other. For example, if you already have lip-sync animation and now want to add motion capture data, simply click on the Motion tab in the Library panel and select a C3D file in the list to apply that mocap. You can then edit this data as usual by using the options in the Tools > Adjust panel (see Adjusting the Retargeted Mocap Data).


Mocap animation on head is blended with lip-sync animation on the mouth controls.

You don’t need to do anything special to combine lip sync and mocap animation: just add the animations as usual and the different types of data will each affect the animation controls as they should. You can then control the amount that each type of animation contributes to the result.

The mocap and the lip-sync animation each contribute equally to the result, but what you usually want is to use the mouth animation from one source and the jaw animation from the other. To do this, you can have either source controlling the animation as you like:

• To have the lip-sync animation drive the animation, you can open the Adjust panel and reduce the weight of the Upper Lip, Lower Lip, and Jaw channels in the Global Controls group. Setting these values to zero means that the lip-sync operator (SpeechBlend) is the only thing affecting those animation controls. Of course, you can set these values to any number and even animate them.

• To have the mocap drive the animation, you can reduce the weight of the Lip, Jaw, and Tongue channels in the Speech Blend property editor (see Weighting the Lips, Jaw, and Tongue Globally). As with the Adjust controls, you can set these values to any number you like and animate them.

You may also choose to only use parts of the lip sync with the mocap. For instance, you can disable the lips and jaw channels of the lip sync so that you are using only the tongue lip-sync animation with the mocap.

You can also decouple the corrective shapes from the lips channel as a way to automate this work (see Decoupling the Corrective Shapes from their Viseme Poses for Mocap Animation).

• If you plot the mocap using the Plot tool (see Plotting (Baking) the Mocap and Keyframe Data), you can blend the mocap with the lip-sync animation in the mixer. Simply adjust the weight of each of the clips to achieve the desired result.


See Mixing and Weighting Clips [Nonlinear Animation] for more general information.

Blending Lip-Sync Animation with Keyframes

Lip-sync animation works in conjunction with keyframe animation. You can set keys directly on the animation controls (see Overview of Setting Keys [Animation]), in animation layers (see Animation Layers [Animation]), or store them as clips in the mixer (see Storing Animation in Action Sources [Nonlinear Animation]).

To add keyframes on top of the lip-sync animation, simply add an animation layer and set keyframes in that layer. These are considered to be an offset from the base layer of animation (the lip-sync animation).

If you have keyframed the lip, jaw, or tongue animation controls directly, the speech operator (the Speech action clip) will override this animation: clips in the mixer always override other animation at the same frames.

However, you can blend between this animation and the lip sync by setting the mixer options to allow blending with the timeline (choose Mix > Animation Mixer Properties in the mixer, then select Mix Current Animation in the Mixer Properties editor). See Mixing Fcurves with Action Clips [Nonlinear Animation] for more information.

Autodesk Softimage 2011 Subscription Advantage Pack