人形机器人可完成多语言逼真唇形动作 在教育和老年护理领域具应用潜力

Core Insights - A new framework designed by scientists at Columbia University enables humanoid robots to generate realistic lip movements synchronized with audio, enhancing human-like communication capabilities [1][2] - The technology demonstrates strong generalization abilities, applicable to multiple languages including French, Chinese, and Arabic, even those not present in the training data [1] - The research represents a significant step towards creating robots that can perform functions while also engaging in humanized interactions [1] Group 1 - The existing robots lack the flexibility to perform intricate lip movements, and few technologies can convert speech into natural lip movement commands in real-time [1] - The research team previously published a study in 2024 describing a humanoid robot's ability to predict and replicate human smiles, laying the groundwork for this new lip synchronization technology [1] - The team developed a learning process that involves collecting visual data of the robot's lip movements to train a model and generate movement reference points [1] Group 2 - A module called the "facial action converter" was created to produce movement commands, allowing the robot's lips to smoothly match different words [1] - The humanoid robot's facial structure features soft silicone skin and magnetic connectors, providing 10 degrees of freedom for complex lip movements, capable of forming various shapes for 24 consonants and 16 vowels [1] - During validation, the team used ChatGPT to generate test sentences and synthesized videos with ideal lip movements for comparison, showing the method's superiority in minimizing differences [2] Group 3 - The framework can generate natural lip synchronization effects for 11 different non-English language phonetic structures [2] - The research team suggests potential applications for humanoid robots in education and elderly care, while also emphasizing the need for careful design to prevent misuse of the technology [2]