A gaussian mixture representation of gesture kinematics for on-line sign language video annotation

Fabio Martínez; Antoine Manzanera; Michèle Gouiffès; Annelies Braffort

doi:10.1007/978-3-319-27863-6_27

A gaussian mixture representation of gesture kinematics for on-line sign language video annotation

Fabio Martínez, Antoine Manzanera, Michèle Gouiffès, Annelies Braffort

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed approach starts by computing local kinematic cues, represented as mixtures of Gaussians which together correspond to gestures with a semantic equivalence in the sign language corpora. At each frame, a spatial pyramid partition allows a fine-to-coarse sub-regional description of motion-cues distribution. Then for each sub-region, a histogram of motion-cues occurrence is built, forming a frame-gesture descriptor which can be used for on-line annotation. The proposed approach is evaluated using a bag-of-features framework, in which every frame-level histogram is mapped to an SVM. Experimental results show competitive results in terms of accuracy and time computation for a signing dataset.

Original language	English
Title of host publication	Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings
Editors	Bahram Parvin, Darko Koracin, Rogerio Feris, Gunther Weber, Ioannis Pavlidis, Tim McGraw, Regis Kopper, Zhao Ye, Eric Ragan, George Bebis, Mark Elendt, Richard Boyle
Publisher	Springer Verlag
Pages	293-303
Number of pages	11
ISBN (Print)	9783319278629
DOIs	https://doi.org/10.1007/978-3-319-27863-6_27
Publication status	Published - 1 Jan 2015
Externally published	Yes
Event	11th International Symposium on Advances in Visual Computing , ISVC 2015 - Las Vegas, United States Duration: 14 Dec 2015 → 16 Dec 2015

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9475
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	11th International Symposium on Advances in Visual Computing , ISVC 2015
Country/Territory	United States
City	Las Vegas
Period	14/12/15 → 16/12/15

Access to Document

10.1007/978-3-319-27863-6_27

Cite this

Martínez, F., Manzanera, A., Gouiffès, M., & Braffort, A. (2015). A gaussian mixture representation of gesture kinematics for on-line sign language video annotation. In B. Parvin, D. Koracin, R. Feris, G. Weber, I. Pavlidis, T. McGraw, R. Kopper, Z. Ye, E. Ragan, G. Bebis, M. Elendt, & R. Boyle (Eds.), Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings (pp. 293-303). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9475). Springer Verlag. https://doi.org/10.1007/978-3-319-27863-6_27

Martínez, Fabio ; Manzanera, Antoine ; Gouiffès, Michèle et al. / A gaussian mixture representation of gesture kinematics for on-line sign language video annotation. Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings. editor / Bahram Parvin ; Darko Koracin ; Rogerio Feris ; Gunther Weber ; Ioannis Pavlidis ; Tim McGraw ; Regis Kopper ; Zhao Ye ; Eric Ragan ; George Bebis ; Mark Elendt ; Richard Boyle. Springer Verlag, 2015. pp. 293-303 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{e4861fc4c54c4bc49f511eaa996326a5,

title = "A gaussian mixture representation of gesture kinematics for on-line sign language video annotation",

abstract = "Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed approach starts by computing local kinematic cues, represented as mixtures of Gaussians which together correspond to gestures with a semantic equivalence in the sign language corpora. At each frame, a spatial pyramid partition allows a fine-to-coarse sub-regional description of motion-cues distribution. Then for each sub-region, a histogram of motion-cues occurrence is built, forming a frame-gesture descriptor which can be used for on-line annotation. The proposed approach is evaluated using a bag-of-features framework, in which every frame-level histogram is mapped to an SVM. Experimental results show competitive results in terms of accuracy and time computation for a signing dataset.",

author = "Fabio Mart{\'i}nez and Antoine Manzanera and Mich{\`e}le Gouiff{\`e}s and Annelies Braffort",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 11th International Symposium on Advances in Visual Computing , ISVC 2015 ; Conference date: 14-12-2015 Through 16-12-2015",

year = "2015",

month = jan,

day = "1",

doi = "10.1007/978-3-319-27863-6\_27",

language = "English",

isbn = "9783319278629",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "293--303",

editor = "Bahram Parvin and Darko Koracin and Rogerio Feris and Gunther Weber and Ioannis Pavlidis and Tim McGraw and Regis Kopper and Zhao Ye and Eric Ragan and George Bebis and Mark Elendt and Richard Boyle",

booktitle = "Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings",

}

Martínez, F, Manzanera, A, Gouiffès, M & Braffort, A 2015, A gaussian mixture representation of gesture kinematics for on-line sign language video annotation. in B Parvin, D Koracin, R Feris, G Weber, I Pavlidis, T McGraw, R Kopper, Z Ye, E Ragan, G Bebis, M Elendt & R Boyle (eds), Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9475, Springer Verlag, pp. 293-303, 11th International Symposium on Advances in Visual Computing , ISVC 2015, Las Vegas, United States, 14/12/15. https://doi.org/10.1007/978-3-319-27863-6_27

A gaussian mixture representation of gesture kinematics for on-line sign language video annotation. / Martínez, Fabio; Manzanera, Antoine; Gouiffès, Michèle et al.
Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings. ed. / Bahram Parvin; Darko Koracin; Rogerio Feris; Gunther Weber; Ioannis Pavlidis; Tim McGraw; Regis Kopper; Zhao Ye; Eric Ragan; George Bebis; Mark Elendt; Richard Boyle. Springer Verlag, 2015. p. 293-303 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9475).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A gaussian mixture representation of gesture kinematics for on-line sign language video annotation

AU - Martínez, Fabio

AU - Manzanera, Antoine

AU - Gouiffès, Michèle

AU - Braffort, Annelies

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2015.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed approach starts by computing local kinematic cues, represented as mixtures of Gaussians which together correspond to gestures with a semantic equivalence in the sign language corpora. At each frame, a spatial pyramid partition allows a fine-to-coarse sub-regional description of motion-cues distribution. Then for each sub-region, a histogram of motion-cues occurrence is built, forming a frame-gesture descriptor which can be used for on-line annotation. The proposed approach is evaluated using a bag-of-features framework, in which every frame-level histogram is mapped to an SVM. Experimental results show competitive results in terms of accuracy and time computation for a signing dataset.

AB - Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed approach starts by computing local kinematic cues, represented as mixtures of Gaussians which together correspond to gestures with a semantic equivalence in the sign language corpora. At each frame, a spatial pyramid partition allows a fine-to-coarse sub-regional description of motion-cues distribution. Then for each sub-region, a histogram of motion-cues occurrence is built, forming a frame-gesture descriptor which can be used for on-line annotation. The proposed approach is evaluated using a bag-of-features framework, in which every frame-level histogram is mapped to an SVM. Experimental results show competitive results in terms of accuracy and time computation for a signing dataset.

U2 - 10.1007/978-3-319-27863-6_27

DO - 10.1007/978-3-319-27863-6_27

M3 - Conference contribution

AN - SCOPUS:84952786663

SN - 9783319278629

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 293

EP - 303

BT - Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings

A2 - Parvin, Bahram

A2 - Koracin, Darko

A2 - Feris, Rogerio

A2 - Weber, Gunther

A2 - Pavlidis, Ioannis

A2 - McGraw, Tim

A2 - Kopper, Regis

A2 - Ye, Zhao

A2 - Ragan, Eric

A2 - Bebis, George

A2 - Elendt, Mark

A2 - Boyle, Richard

PB - Springer Verlag

T2 - 11th International Symposium on Advances in Visual Computing , ISVC 2015

Y2 - 14 December 2015 through 16 December 2015

ER -

Martínez F, Manzanera A, Gouiffès M, Braffort A. A gaussian mixture representation of gesture kinematics for on-line sign language video annotation. In Parvin B, Koracin D, Feris R, Weber G, Pavlidis I, McGraw T, Kopper R, Ye Z, Ragan E, Bebis G, Elendt M, Boyle R, editors, Advances in Visual Computing - 11th International Symposium, ISVC 2015, Proceedings. Springer Verlag. 2015. p. 293-303. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-27863-6_27

A gaussian mixture representation of gesture kinematics for on-line sign language video annotation

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this