Multimodal human machine interactions in virtual and augmented reality

Gérard Chollet; Anna Esposito; Annie Gentes; Patrick Horain; Walid Karam; Zhenbo Li; Catherine Pelachaud; Patrick Perrot; Dijana Petrovska-Delacrétaz; Dianle Zhou; Leila Zouari

doi:10.1007/978-3-642-00525-1_1

Multimodal human machine interactions in virtual and augmented reality

Gérard Chollet, Anna Esposito, Annie Gentes, Patrick Horain, Walid Karam, Zhenbo Li, Catherine Pelachaud, Patrick Perrot, Dijana Petrovska-Delacrétaz, Dianle Zhou, Leila Zouari

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.

Original language	English
Title of host publication	Multimodal Signals
Subtitle of host publication	Cognitive and Algorithmic Issues - COST Action 2102 and euCognition International School, Revised Selected and Invited Papers
Pages	1-23
Number of pages	23
DOIs	https://doi.org/10.1007/978-3-642-00525-1_1
Publication status	Published - 21 Jul 2009
Externally published	Yes
Event	COST Action 2102 and euCognition International School on Multimodal Signals: Cognitive and Algorithmic Issues - Vietri sul Mare, Italy Duration: 21 Apr 2008 → 26 Apr 2008

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5398 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	COST Action 2102 and euCognition International School on Multimodal Signals: Cognitive and Algorithmic Issues
Country/Territory	Italy
City	Vietri sul Mare
Period	21/04/08 → 26/04/08

Keywords

Face
Gesture
Human machine interactions (HMI)
Multimodality
Speech
Virtual words

Access to Document

10.1007/978-3-642-00525-1_1

Cite this

Chollet, G., Esposito, A., Gentes, A., Horain, P., Karam, W., Li, Z., Pelachaud, C., Perrot, P., Petrovska-Delacrétaz, D., Zhou, D., & Zouari, L. (2009). Multimodal human machine interactions in virtual and augmented reality. In Multimodal Signals: Cognitive and Algorithmic Issues - COST Action 2102 and euCognition International School, Revised Selected and Invited Papers (pp. 1-23). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5398 LNAI). https://doi.org/10.1007/978-3-642-00525-1_1

Chollet, Gérard ; Esposito, Anna ; Gentes, Annie et al. / Multimodal human machine interactions in virtual and augmented reality. Multimodal Signals: Cognitive and Algorithmic Issues - COST Action 2102 and euCognition International School, Revised Selected and Invited Papers. 2009. pp. 1-23 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{efd22e103e53498f9bd7905ef9d47911,

title = "Multimodal human machine interactions in virtual and augmented reality",

abstract = "Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.",

keywords = "Face, Gesture, Human machine interactions (HMI), Multimodality, Speech, Virtual words",

author = "G{\'e}rard Chollet and Anna Esposito and Annie Gentes and Patrick Horain and Walid Karam and Zhenbo Li and Catherine Pelachaud and Patrick Perrot and Dijana Petrovska-Delacr{\'e}taz and Dianle Zhou and Leila Zouari",

year = "2009",

month = jul,

day = "21",

doi = "10.1007/978-3-642-00525-1\_1",

language = "English",

isbn = "3642005241",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "1--23",

booktitle = "Multimodal Signals",

note = "COST Action 2102 and euCognition International School on Multimodal Signals: Cognitive and Algorithmic Issues ; Conference date: 21-04-2008 Through 26-04-2008",

}

Chollet, G, Esposito, A, Gentes, A, Horain, P, Karam, W, Li, Z, Pelachaud, C, Perrot, P, Petrovska-Delacrétaz, D, Zhou, D & Zouari, L 2009, Multimodal human machine interactions in virtual and augmented reality. in Multimodal Signals: Cognitive and Algorithmic Issues - COST Action 2102 and euCognition International School, Revised Selected and Invited Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5398 LNAI, pp. 1-23, COST Action 2102 and euCognition International School on Multimodal Signals: Cognitive and Algorithmic Issues, Vietri sul Mare, Italy, 21/04/08. https://doi.org/10.1007/978-3-642-00525-1_1

Multimodal human machine interactions in virtual and augmented reality. / Chollet, Gérard; Esposito, Anna; Gentes, Annie et al.
Multimodal Signals: Cognitive and Algorithmic Issues - COST Action 2102 and euCognition International School, Revised Selected and Invited Papers. 2009. p. 1-23 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5398 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Multimodal human machine interactions in virtual and augmented reality

AU - Chollet, Gérard

AU - Esposito, Anna

AU - Gentes, Annie

AU - Horain, Patrick

AU - Karam, Walid

AU - Li, Zhenbo

AU - Pelachaud, Catherine

AU - Perrot, Patrick

AU - Petrovska-Delacrétaz, Dijana

AU - Zhou, Dianle

AU - Zouari, Leila

PY - 2009/7/21

Y1 - 2009/7/21

N2 - Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.

AB - Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.

KW - Face

KW - Gesture

KW - Human machine interactions (HMI)

KW - Multimodality

KW - Speech

KW - Virtual words

U2 - 10.1007/978-3-642-00525-1_1

DO - 10.1007/978-3-642-00525-1_1

M3 - Conference contribution

AN - SCOPUS:67650551307

SN - 3642005241

SN - 9783642005244

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 1

EP - 23

BT - Multimodal Signals

T2 - COST Action 2102 and euCognition International School on Multimodal Signals: Cognitive and Algorithmic Issues

Y2 - 21 April 2008 through 26 April 2008

ER -

Chollet G, Esposito A, Gentes A, Horain P, Karam W, Li Z et al. Multimodal human machine interactions in virtual and augmented reality. In Multimodal Signals: Cognitive and Algorithmic Issues - COST Action 2102 and euCognition International School, Revised Selected and Invited Papers. 2009. p. 1-23. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-00525-1_1

Multimodal human machine interactions in virtual and augmented reality

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this