This article is also available in:
Français
At Unreal Fest 2024, NVIDIA unveiled new NVIDIA ACE (Avatar Cloud Engine) plugins for Unreal Engine 5. NVIDIA ACE is a suite of technologies dedicated to digital humans, bringing “speech, intelligence, and animation” through generative AI.
In simple terms, ACE allows a digital human to respond to a real human.
Additionally, a new Audio2Face-3D plugin for Maya enables the animation of speaking characters in Maya based on audio dialogue. The source code is provided.
NVIDIA explains that a sample project for Unreal Engine will help developers who want to integrate NVIDIA ACE into their games and applications.
This includes lip-sync and facial animation based on audio (Audio2Face-3D), response generation (Nemotron Mini 4B Instruct model), and contextual information management (Retrieval-augmented generation).
NVIDIA explains that these new plugins for Unreal will be available soon. As for Maya, head over to GitHub. An article provides more details on how these tools work and can be used, along with another announcement about Pixel Streaming. A playlist of videos has also been set up on the NVIDIA Developer channel.
