Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning

Bogdan Mocanu; Ruxandra Tapu

doi:10.1109/ICCE53296.2022.9730534

Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning

Bogdan Mocanu
, Ruxandra Tapu

Telecom SudParis

University 'Politehnica' of Bucharest

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper we have introduced a novel emotion recognition framework from raw speech signals. The system is based on ResNet architecture fed with spectrogram inputs. The CNN is further extended with a GhostVLAD feature aggregation layer that extracts a single, fixed size descriptor constructed at the level of the utterance. The system adopts a sentiment metric loss that integrates the relations between various classes of emotions. The experimental evaluation conducted on two publicly available databases: RAVDESS and CREMA-D validates the proposed methodology with average accuracy scores of 82% and 63%, respectively.

Original language	English
Title of host publication	2022 IEEE International Conference on Consumer Electronics, ICCE 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665441544
DOIs	https://doi.org/10.1109/ICCE53296.2022.9730534
Publication status	Published - 1 Jan 2022
Event	2022 IEEE International Conference on Consumer Electronics, ICCE 2022 - Virtual, Online, United States Duration: 7 Jan 2022 → 9 Jan 2022

Publication series

Name	Digest of Technical Papers - IEEE International Conference on Consumer Electronics
Volume	2022-January
ISSN (Print)	0747-668X

Conference

Conference	2022 IEEE International Conference on Consumer Electronics, ICCE 2022
Country/Territory	United States
City	Virtual, Online
Period	7/01/22 → 9/01/22

Keywords

GhostVLAD aggregation layer
multi-stage training
sentiment metric learning
speech emotion recognition

Access to Document

10.1109/ICCE53296.2022.9730534

Cite this

Mocanu, B., & Tapu, R. (2022). Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning. In 2022 IEEE International Conference on Consumer Electronics, ICCE 2022 (Digest of Technical Papers - IEEE International Conference on Consumer Electronics; Vol. 2022-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCE53296.2022.9730534

@inproceedings{f0a3da751e834cbf97902e633ef97535,

title = "Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning",

abstract = "In this paper we have introduced a novel emotion recognition framework from raw speech signals. The system is based on ResNet architecture fed with spectrogram inputs. The CNN is further extended with a GhostVLAD feature aggregation layer that extracts a single, fixed size descriptor constructed at the level of the utterance. The system adopts a sentiment metric loss that integrates the relations between various classes of emotions. The experimental evaluation conducted on two publicly available databases: RAVDESS and CREMA-D validates the proposed methodology with average accuracy scores of 82\% and 63\%, respectively.",

keywords = "GhostVLAD aggregation layer, multi-stage training, sentiment metric learning, speech emotion recognition",

author = "Bogdan Mocanu and Ruxandra Tapu",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE International Conference on Consumer Electronics, ICCE 2022 ; Conference date: 07-01-2022 Through 09-01-2022",

year = "2022",

month = jan,

day = "1",

doi = "10.1109/ICCE53296.2022.9730534",

language = "English",

series = "Digest of Technical Papers - IEEE International Conference on Consumer Electronics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2022 IEEE International Conference on Consumer Electronics, ICCE 2022",

}

Mocanu, B & Tapu, R 2022, Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning. in 2022 IEEE International Conference on Consumer Electronics, ICCE 2022. Digest of Technical Papers - IEEE International Conference on Consumer Electronics, vol. 2022-January, Institute of Electrical and Electronics Engineers Inc., 2022 IEEE International Conference on Consumer Electronics, ICCE 2022, Virtual, Online, United States, 7/01/22. https://doi.org/10.1109/ICCE53296.2022.9730534

Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning. / Mocanu, Bogdan; Tapu, Ruxandra.
2022 IEEE International Conference on Consumer Electronics, ICCE 2022. Institute of Electrical and Electronics Engineers Inc., 2022. (Digest of Technical Papers - IEEE International Conference on Consumer Electronics; Vol. 2022-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning

AU - Mocanu, Bogdan

AU - Tapu, Ruxandra

PY - 2022/1/1

Y1 - 2022/1/1

N2 - In this paper we have introduced a novel emotion recognition framework from raw speech signals. The system is based on ResNet architecture fed with spectrogram inputs. The CNN is further extended with a GhostVLAD feature aggregation layer that extracts a single, fixed size descriptor constructed at the level of the utterance. The system adopts a sentiment metric loss that integrates the relations between various classes of emotions. The experimental evaluation conducted on two publicly available databases: RAVDESS and CREMA-D validates the proposed methodology with average accuracy scores of 82% and 63%, respectively.

AB - In this paper we have introduced a novel emotion recognition framework from raw speech signals. The system is based on ResNet architecture fed with spectrogram inputs. The CNN is further extended with a GhostVLAD feature aggregation layer that extracts a single, fixed size descriptor constructed at the level of the utterance. The system adopts a sentiment metric loss that integrates the relations between various classes of emotions. The experimental evaluation conducted on two publicly available databases: RAVDESS and CREMA-D validates the proposed methodology with average accuracy scores of 82% and 63%, respectively.

KW - GhostVLAD aggregation layer

KW - multi-stage training

KW - sentiment metric learning

KW - speech emotion recognition

U2 - 10.1109/ICCE53296.2022.9730534

DO - 10.1109/ICCE53296.2022.9730534

M3 - Conference contribution

AN - SCOPUS:85127059517

T3 - Digest of Technical Papers - IEEE International Conference on Consumer Electronics

BT - 2022 IEEE International Conference on Consumer Electronics, ICCE 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 IEEE International Conference on Consumer Electronics, ICCE 2022

Y2 - 7 January 2022 through 9 January 2022

ER -

Emotion Recognition from Raw Speech Signals Using 2D CNN with Deep Metric Learning

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this