TY - GEN
T1 - Activity Recognition Based on RGB-D and Thermal Sensors for Socially Assistive Robots
AU - Sorostinean, Mihaela
AU - Tapus, Adriana
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/12/18
Y1 - 2018/12/18
N2 - For socially assistive robots, being able to recognize basic human actions is an important capability. The sensors, which are frequently mounted on most recent robots, such as RGB-D and thermal cameras, as well as the advances in deep learning have enabled the research on activity recognition to grow. In this paper, we collected our own dataset of actions in a home-like scenario, which contains thermal imagery in addition to RGB-D data and we proposed a method based on Long-term Recurrent Convolutional Networks (LRCN). We showed that our method has an accuracy comparable with the state-of-the-art. We also proved that thermal information can improve the recognition accuracy. Furthermore, we tested the real-time capability of our system and conducted a real-time experiment with a robot (Pepper robot from Softbank Robotics) so as to investigate the effect of a robot enabled with action recognition capability in a human-robot interaction.
AB - For socially assistive robots, being able to recognize basic human actions is an important capability. The sensors, which are frequently mounted on most recent robots, such as RGB-D and thermal cameras, as well as the advances in deep learning have enabled the research on activity recognition to grow. In this paper, we collected our own dataset of actions in a home-like scenario, which contains thermal imagery in addition to RGB-D data and we proposed a method based on Long-term Recurrent Convolutional Networks (LRCN). We showed that our method has an accuracy comparable with the state-of-the-art. We also proved that thermal information can improve the recognition accuracy. Furthermore, we tested the real-time capability of our system and conducted a real-time experiment with a robot (Pepper robot from Softbank Robotics) so as to investigate the effect of a robot enabled with action recognition capability in a human-robot interaction.
U2 - 10.1109/ICARCV.2018.8581349
DO - 10.1109/ICARCV.2018.8581349
M3 - Conference contribution
AN - SCOPUS:85060817685
T3 - 2018 15th International Conference on Control, Automation, Robotics and Vision, ICARCV 2018
SP - 1298
EP - 1304
BT - 2018 15th International Conference on Control, Automation, Robotics and Vision, ICARCV 2018
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 15th International Conference on Control, Automation, Robotics and Vision, ICARCV 2018
Y2 - 18 November 2018 through 21 November 2018
ER -