TY - GEN
T1 - Speech-Driven Robot Face Action Generation with Deep Generative Model for Social Robots
AU - Yu, Chuang
AU - Zhang, Heng
AU - Shangguan, Zhegong
AU - Hei, Xiaoxuan
AU - Cangelosi, Angelo
AU - Tapus, Adriana
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022.
PY - 2022/1/1
Y1 - 2022/1/1
N2 - The natural co-speech facial action as a kind of non-verbal behavior plays an essential role in human communication, which also leads to a natural and friendly human-robot interaction. However, a lot of previous works for robot speech-based behaviour generation are rule-based or handcrafted methods, which are time-consuming and with limited synchronization levels between the speech and the facial action. Based on the Generative Adversarial Networks (GAN) model, this paper developed an effective speech-driven facial action synthesizer, i.e., given an acoustic speech, a synchronous and realistic 3D facial action sequence is generated. In addition, a mapping between the 3D human facial action to the real robot facial action that regulates Zeno robot facial expressions is also completed. The evaluation results show the model has potential for natural human-robot interaction.
AB - The natural co-speech facial action as a kind of non-verbal behavior plays an essential role in human communication, which also leads to a natural and friendly human-robot interaction. However, a lot of previous works for robot speech-based behaviour generation are rule-based or handcrafted methods, which are time-consuming and with limited synchronization levels between the speech and the facial action. Based on the Generative Adversarial Networks (GAN) model, this paper developed an effective speech-driven facial action synthesizer, i.e., given an acoustic speech, a synchronous and realistic 3D facial action sequence is generated. In addition, a mapping between the 3D human facial action to the real robot facial action that regulates Zeno robot facial expressions is also completed. The evaluation results show the model has potential for natural human-robot interaction.
KW - Face action
KW - Human-robot interaction
KW - Social robot
U2 - 10.1007/978-3-031-24667-8_6
DO - 10.1007/978-3-031-24667-8_6
M3 - Conference contribution
AN - SCOPUS:85149814766
SN - 9783031246661
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 61
EP - 74
BT - Social Robotics - 14th International Conference, ICSR 2022, Proceedings
A2 - Cavallo, Filippo
A2 - Fiorini, Laura
A2 - Sorrentino, Alessandra
A2 - Cabibihan, John-John
A2 - He, Hongsheng
A2 - Liu, Xiaorui
A2 - Matsumoto, Yoshio
A2 - Ge, Shuzhi Sam
PB - Springer Science and Business Media Deutschland GmbH
T2 - 14th International Conference on Social Robotics, ICSR 2022
Y2 - 13 December 2022 through 16 December 2022
ER -