Robotics Enthusiast. Aircraft Fleet Manager and Lease Manager. JobToRob - JOBs TO ROBOTs (J2R) Business Model. JobToRob: Jobs Hub for Robots.
WANDR: a model to generate realistic human motions for avatars In a groundbreaking development, researchers at the Max Planck Institute for Intelligent Systems and ETH Zurich have unveiled WANDR, a cutting-edge model capable of generating natural human motions for virtual avatars and 3D animated human-like characters. Introduced in a paper to be presented at the upcoming Conference on Computer Vision and Pattern Recognition (CVPR 2024) in June, WANDR unifies different data sources under a single model, paving the way for more realistic motions in the digital realm. As Markos Diomataris, the first author of the paper, explains, "At a high-level, our research aims at figuring out what it takes to create virtual humans able to behave like us. This essentially means learning to reason about the world, how to move in it, setting goals and trying to achieve them." The primary objective of this study was to create a model that could generate realistic motions for 3D avatars, allowing them to interact with their virtual environment seamlessly, such as reaching to grab objects. Diomataris emphasizes the complexity of human movements, stating, "Consider reaching for a coffee cup—it can be as straightforward as an arm extension or can involve the coordinated action of our entire body. Actions like bending down, extending our arm, and walking must come together to achieve the goal. At a granular level, we continuously make subtle adjustments to maintain balance and stay on course towards our objective." Traditionally, there have been two main approaches to teaching virtual agents new skills: reinforcement learning (RL) and compiling datasets of human demonstrations to train machine learning models. While RL allows agents to learn through trial and error, it can be computationally intensive and require careful reward design to prevent unnatural behaviors. On the other hand, using datasets provides richer information about a skill, but few datasets currently include reaching motions, which the researchers aimed to replicate. Prioritizing motion realism, the team chose to learn this skill from data. "WANDR is the first human motion generation model that is driven by an active feedback loop learned purely from data, without any extra steps of reinforcement learning (RL)," Diomataris explains. WANDR generates motion autoregressively, frame-by-frame, predicting actions that will progress the avatar to its next state. These predictions are conditioned by time and goal-dependent features, termed "intention," which are recomputed at every frame, acting as a feedback loop to guide the avatar in reaching a given goal using its wrist. This allows WANDR to approach and reach moving or sequential goals, even though it has never been explicitly trained for such tasks.
