Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM

Enrico Marchetto; Geoffroy Peeters

doi:10.1007/978-3-030-01692-0_1

Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM

Enrico Marchetto
, Geoffroy Peeters

Sorbonne Université

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper we study the automatic recognition of sound categories (such as fridge, mixers or sawing sounds) from their vocal imitations. Vocal imitations are made of a succession over time of sounds produced using vocal mechanisms that can largely differ from the ones used in speech. We develop here a recognition approach inspired by automatic-speech-recognition systems, with an acoustic model (that maps the audio signal to a set of probability over “phonemes”) and a language model (that represents the expected succession of “phonemes” for each sound category). Since we do not know what are the underlying “phonemes” of vocal imitations we propose to automatically estimate them using Shift-Invariant Probabilistic Latent Component Analysis (SI-PLCA) applied to a dataset of vocal imitations. The kernel distributions of the SI-PLCA are considered as the “phonemes” of vocal imitation and its impulse distributions are used to compute the emission probabilities of the states of a set of Hidden Markov Models (HMMs). To evaluate our proposal, we test it for a task of automatically recognizing 12 sound categories from their vocal imitations.

Original language	English
Title of host publication	Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers
Editors	Matthew E.P. Davies, Mitsuko Aramaki, Richard Kronland-Martinet, Sølvi Ystad
Publisher	Springer Verlag
Pages	3-22
Number of pages	20
ISBN (Print)	9783030016913
DOIs	https://doi.org/10.1007/978-3-030-01692-0_1
Publication status	Published - 1 Jan 2018
Event	13th international Symposium on Computer Music Multidisciplinary Research, CMMR 2017 - Matosinhos, Portugal Duration: 25 Sept 2017 → 28 Sept 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11265 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	13th international Symposium on Computer Music Multidisciplinary Research, CMMR 2017
Country/Territory	Portugal
City	Matosinhos
Period	25/09/17 → 28/09/17

Keywords

Hidden markov model
Shift-invariant probabilistic-latent-component-analysis
Sound design
Sound recognition
Vocal imitation

Access to Document

10.1007/978-3-030-01692-0_1

Cite this

Marchetto, E., & Peeters, G. (2018). Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM. In M. E. P. Davies, M. Aramaki, R. Kronland-Martinet, & S. Ystad (Eds.), Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers (pp. 3-22). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11265 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01692-0_1

Marchetto, Enrico ; Peeters, Geoffroy. / Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM. Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers. editor / Matthew E.P. Davies ; Mitsuko Aramaki ; Richard Kronland-Martinet ; Sølvi Ystad. Springer Verlag, 2018. pp. 3-22 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2e82d52a8e7a4a639e7ca7b7b3033547,

title = "Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM",

abstract = "In this paper we study the automatic recognition of sound categories (such as fridge, mixers or sawing sounds) from their vocal imitations. Vocal imitations are made of a succession over time of sounds produced using vocal mechanisms that can largely differ from the ones used in speech. We develop here a recognition approach inspired by automatic-speech-recognition systems, with an acoustic model (that maps the audio signal to a set of probability over “phonemes”) and a language model (that represents the expected succession of “phonemes” for each sound category). Since we do not know what are the underlying “phonemes” of vocal imitations we propose to automatically estimate them using Shift-Invariant Probabilistic Latent Component Analysis (SI-PLCA) applied to a dataset of vocal imitations. The kernel distributions of the SI-PLCA are considered as the “phonemes” of vocal imitation and its impulse distributions are used to compute the emission probabilities of the states of a set of Hidden Markov Models (HMMs). To evaluate our proposal, we test it for a task of automatically recognizing 12 sound categories from their vocal imitations.",

keywords = "Hidden markov model, Shift-invariant probabilistic-latent-component-analysis, Sound design, Sound recognition, Vocal imitation",

author = "Enrico Marchetto and Geoffroy Peeters",

note = "Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG.; 13th international Symposium on Computer Music Multidisciplinary Research, CMMR 2017 ; Conference date: 25-09-2017 Through 28-09-2017",

year = "2018",

month = jan,

day = "1",

doi = "10.1007/978-3-030-01692-0\_1",

language = "English",

isbn = "9783030016913",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "3--22",

editor = "Davies, \{Matthew E.P.\} and Mitsuko Aramaki and Richard Kronland-Martinet and S{\o}lvi Ystad",

booktitle = "Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers",

}

Marchetto, E & Peeters, G 2018, Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM. in MEP Davies, M Aramaki, R Kronland-Martinet & S Ystad (eds), Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11265 LNCS, Springer Verlag, pp. 3-22, 13th international Symposium on Computer Music Multidisciplinary Research, CMMR 2017, Matosinhos, Portugal, 25/09/17. https://doi.org/10.1007/978-3-030-01692-0_1

Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM. / Marchetto, Enrico; Peeters, Geoffroy.
Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers. ed. / Matthew E.P. Davies; Mitsuko Aramaki; Richard Kronland-Martinet; Sølvi Ystad. Springer Verlag, 2018. p. 3-22 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11265 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM

AU - Marchetto, Enrico

AU - Peeters, Geoffroy

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In this paper we study the automatic recognition of sound categories (such as fridge, mixers or sawing sounds) from their vocal imitations. Vocal imitations are made of a succession over time of sounds produced using vocal mechanisms that can largely differ from the ones used in speech. We develop here a recognition approach inspired by automatic-speech-recognition systems, with an acoustic model (that maps the audio signal to a set of probability over “phonemes”) and a language model (that represents the expected succession of “phonemes” for each sound category). Since we do not know what are the underlying “phonemes” of vocal imitations we propose to automatically estimate them using Shift-Invariant Probabilistic Latent Component Analysis (SI-PLCA) applied to a dataset of vocal imitations. The kernel distributions of the SI-PLCA are considered as the “phonemes” of vocal imitation and its impulse distributions are used to compute the emission probabilities of the states of a set of Hidden Markov Models (HMMs). To evaluate our proposal, we test it for a task of automatically recognizing 12 sound categories from their vocal imitations.

AB - In this paper we study the automatic recognition of sound categories (such as fridge, mixers or sawing sounds) from their vocal imitations. Vocal imitations are made of a succession over time of sounds produced using vocal mechanisms that can largely differ from the ones used in speech. We develop here a recognition approach inspired by automatic-speech-recognition systems, with an acoustic model (that maps the audio signal to a set of probability over “phonemes”) and a language model (that represents the expected succession of “phonemes” for each sound category). Since we do not know what are the underlying “phonemes” of vocal imitations we propose to automatically estimate them using Shift-Invariant Probabilistic Latent Component Analysis (SI-PLCA) applied to a dataset of vocal imitations. The kernel distributions of the SI-PLCA are considered as the “phonemes” of vocal imitation and its impulse distributions are used to compute the emission probabilities of the states of a set of Hidden Markov Models (HMMs). To evaluate our proposal, we test it for a task of automatically recognizing 12 sound categories from their vocal imitations.

KW - Hidden markov model

KW - Shift-invariant probabilistic-latent-component-analysis

KW - Sound design

KW - Sound recognition

KW - Vocal imitation

U2 - 10.1007/978-3-030-01692-0_1

DO - 10.1007/978-3-030-01692-0_1

M3 - Conference contribution

AN - SCOPUS:85057383791

SN - 9783030016913

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 3

EP - 22

BT - Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers

A2 - Davies, Matthew E.P.

A2 - Aramaki, Mitsuko

A2 - Kronland-Martinet, Richard

A2 - Ystad, Sølvi

PB - Springer Verlag

T2 - 13th international Symposium on Computer Music Multidisciplinary Research, CMMR 2017

Y2 - 25 September 2017 through 28 September 2017

ER -

Marchetto E, Peeters G. Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM. In Davies MEP, Aramaki M, Kronland-Martinet R, Ystad S, editors, Music Technology with Swing - 13th International Symposium, CMMR 2017, Revised Selected Papers. Springer Verlag. 2018. p. 3-22. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-01692-0_1

Automatic Recognition of Sound Categories from Their Vocal Imitation Using Audio Primitives Automatically Found by SI-PLCA and HMM

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this