Speech Graphics now animates breath and laughter!
We are excited to introduce the latest advancements in audio-driven facial animation technology with the release of SGX 4.3! This update comprises groundbreaking features, including breath animation and laughter detection, as well as improved lip sync, enhanced language modules, and much more.
Play the video below to see examples of the audio-driven breath cycle and laughter detection in action, and read on to find out more about all of the latest features in SGX 4.3.
Audio-driven breath cycle
More than a simple breathing loop, SGX detects the sounds of breathing in the input audio, and the breath animation is synchronized to the audible inhales and exhales. Breath is also woven naturally into the pauses in speech. The intensity, speed, and rate of breath are all determined by the vocal performance.
In order to make use of automated breath animation, the only requirement is to have the new Chest Breath muscle defined (pictured).
Pictured: Adding a Chest Breath muscle enables breath animation to be driven by audio input.
Below are two examples of automatic breath animation: panting breath, and breath mixed with effort and speech.
Controlling the breath
If desired, breath behavior can be adjusted by two new Behavior Modifiers: Breath Magnitude and Breath Speed.
In addition to animating breath, SGX also outputs breath metadata, in the form of inhale and exhale segments. This metadata can be edited to change breath timing and is included in metadata export.
Laughter detection and animation
As part of our mission to automate animation for all of the vocal sounds a human can make, SGX 4.3 adds a new auto mode: Laugh. This is triggered whenever laughter is detected in the audio. The behavior of the new auto mode is defined by mapping it to a behavior mode with various laugh-like expressions. Below are a couple of examples of automatically generated laughter animation using the Laugh auto mode.
Improved lip sync and expressions
As of SGX 4.3, an algorithmic breakthrough in phonetic lip sync provides improvements in the alignment of speech motion with the voice. The timing improvements are especially noticeable for softer-edged sounds like glides and liquids. These improvements apply to all languages.
SGX 4.3 also includes the release of Mandarin 2.1 featuring improved Mandarin pronunciation.
Users should also notice generally improved expression intensities. That is, the intensity of the facial expressions should better match the intensity of the vocal performance.
All of these enhancements contribute to making SGX 4.3 animation more believable and engaging.
Improved language modules
Language modules in SGX 4.3 have been optimized for better performance and ease of use. Improvements include:
Single file modules: each language module now consists of just a single file, simplifying download and installation.
Faster loading: language modules now load significantly faster, reducing processing time.
Direct downloads: users can now download language modules directly from the Downloads site, rather than via delivery.
Dual animation levels
Character rigs can have two animation levels - high-level UIs such as control rigs, and low-level parameters such as bones, blendshapes or anim curves. We call these the control level and the baked level, respectively. Typically:
The control level is better suited for manually setting up the muscle poses.
The baked level is used for animation in the game engine.
A Character Control File that animates the control level can be retargeted to the baked level. Previously, this retargeting process produced a second ‘baked’ Character Control File. This resulted in two Character Control Files for a single character, and any changes made to the original file required re-generating the second file. This situation often led to confusion between the two files and situations where they became out of sync.
In SGX 4.3, a single Character Control File can now contain both animation levels. All that is required to make use of this feature is to select a second set of baked targets, and call a function to bake the muscles.
Muscle activation display
During playback of an event in SGX Director, you can now view the state of the muscles in an instance of SGX Studio bound to the event. The muscle sliders are updated whenever playback pauses or the play position is scrubbed.
Get in touch
With SGX 4.3, we continue to lead the way in the generation of high-quality animation at scale and with creative control. These new features empower creators to produce realistic, expressive, and engaging characters more quickly and efficiently than ever before. We can't wait to see how you leverage these tools to bring your characters to life. Stay tuned for more innovations and get in touch with our team to request a demo of SGX.
