Using temporal association rules for the synthesis of embodied conversational agents with a specific stance

Thomas Janssoone; Chloé Clavel; Kévin Bailly; Gaël Richard

doi:10.1007/978-3-319-47665-0_16

Using temporal association rules for the synthesis of embodied conversational agents with a specific stance

Thomas Janssoone
, Chloé Clavel
, Kévin Bailly
, Gaël Richard

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In the field of Embodied Conversational Agent (ECA) one of the main challenges is to generate socially believable agents. The long run objective of the present study is to infer rules for the multimodal generation of agents’ socio-emotional behaviour. In this paper, we introduce the Social Multimodal Association Rules with Timing (SMART) algorithm. It proposes to learn the rules from the analysis of a multimodal corpus composed by audio-video recordings of human-human interactions. The proposed methodology consists in applying a Sequence Mining algorithm using automatically extracted Social Signals such as prosody, head movements and facial muscles activation as an input. This allows us to infer Temporal Association Rules for the behaviour generation. We show that this method can automatically compute Temporal Association Rules coherent with prior results found in the literature especially in the psychology and sociology fields. The results of a perceptive evaluation confirms the ability of a Temporal Association Rules based agent to express a specific stance.

Original language	English
Title of host publication	Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings
Editors	Peter Khooshabeh, David Traum, William Swartout, Stefan Scherer, Anton Leuski, Stefan Kopp
Publisher	Springer Verlag
Pages	175-189
Number of pages	15
ISBN (Print)	9783319476643
DOIs	https://doi.org/10.1007/978-3-319-47665-0_16
Publication status	Published - 1 Jan 2016
Externally published	Yes
Event	16th International Conference on Intelligent Virtual Agents, IVA 2016 - Los Angeles, United States Duration: 20 Sept 2016 → 23 Sept 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10011 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th International Conference on Intelligent Virtual Agents, IVA 2016
Country/Territory	United States
City	Los Angeles
Period	20/09/16 → 23/09/16

Keywords

Embodied conversational agent
Multi-modal social signal
Sequence mining
Signal processing

Access to Document

10.1007/978-3-319-47665-0_16

Cite this

Janssoone, T., Clavel, C., Bailly, K., & Richard, G. (2016). Using temporal association rules for the synthesis of embodied conversational agents with a specific stance. In P. Khooshabeh, D. Traum, W. Swartout, S. Scherer, A. Leuski, & S. Kopp (Eds.), Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings (pp. 175-189). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10011 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-47665-0_16

Janssoone, Thomas ; Clavel, Chloé ; Bailly, Kévin et al. / Using temporal association rules for the synthesis of embodied conversational agents with a specific stance. Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings. editor / Peter Khooshabeh ; David Traum ; William Swartout ; Stefan Scherer ; Anton Leuski ; Stefan Kopp. Springer Verlag, 2016. pp. 175-189 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{12318f785db44111a7fb675415a8e9f5,

title = "Using temporal association rules for the synthesis of embodied conversational agents with a specific stance",

abstract = "In the field of Embodied Conversational Agent (ECA) one of the main challenges is to generate socially believable agents. The long run objective of the present study is to infer rules for the multimodal generation of agents{\textquoteright} socio-emotional behaviour. In this paper, we introduce the Social Multimodal Association Rules with Timing (SMART) algorithm. It proposes to learn the rules from the analysis of a multimodal corpus composed by audio-video recordings of human-human interactions. The proposed methodology consists in applying a Sequence Mining algorithm using automatically extracted Social Signals such as prosody, head movements and facial muscles activation as an input. This allows us to infer Temporal Association Rules for the behaviour generation. We show that this method can automatically compute Temporal Association Rules coherent with prior results found in the literature especially in the psychology and sociology fields. The results of a perceptive evaluation confirms the ability of a Temporal Association Rules based agent to express a specific stance.",

keywords = "Embodied conversational agent, Multi-modal social signal, Sequence mining, Signal processing",

author = "Thomas Janssoone and Chlo{\'e} Clavel and K{\'e}vin Bailly and Ga{\"e}l Richard",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; 16th International Conference on Intelligent Virtual Agents, IVA 2016 ; Conference date: 20-09-2016 Through 23-09-2016",

year = "2016",

month = jan,

day = "1",

doi = "10.1007/978-3-319-47665-0\_16",

language = "English",

isbn = "9783319476643",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "175--189",

editor = "Peter Khooshabeh and David Traum and William Swartout and Stefan Scherer and Anton Leuski and Stefan Kopp",

booktitle = "Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings",

}

Janssoone, T, Clavel, C, Bailly, K & Richard, G 2016, Using temporal association rules for the synthesis of embodied conversational agents with a specific stance. in P Khooshabeh, D Traum, W Swartout, S Scherer, A Leuski & S Kopp (eds), Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10011 LNAI, Springer Verlag, pp. 175-189, 16th International Conference on Intelligent Virtual Agents, IVA 2016, Los Angeles, United States, 20/09/16. https://doi.org/10.1007/978-3-319-47665-0_16

Using temporal association rules for the synthesis of embodied conversational agents with a specific stance. / Janssoone, Thomas; Clavel, Chloé; Bailly, Kévin et al.
Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings. ed. / Peter Khooshabeh; David Traum; William Swartout; Stefan Scherer; Anton Leuski; Stefan Kopp. Springer Verlag, 2016. p. 175-189 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10011 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Using temporal association rules for the synthesis of embodied conversational agents with a specific stance

AU - Janssoone, Thomas

AU - Clavel, Chloé

AU - Bailly, Kévin

AU - Richard, Gaël

N1 - Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - In the field of Embodied Conversational Agent (ECA) one of the main challenges is to generate socially believable agents. The long run objective of the present study is to infer rules for the multimodal generation of agents’ socio-emotional behaviour. In this paper, we introduce the Social Multimodal Association Rules with Timing (SMART) algorithm. It proposes to learn the rules from the analysis of a multimodal corpus composed by audio-video recordings of human-human interactions. The proposed methodology consists in applying a Sequence Mining algorithm using automatically extracted Social Signals such as prosody, head movements and facial muscles activation as an input. This allows us to infer Temporal Association Rules for the behaviour generation. We show that this method can automatically compute Temporal Association Rules coherent with prior results found in the literature especially in the psychology and sociology fields. The results of a perceptive evaluation confirms the ability of a Temporal Association Rules based agent to express a specific stance.

AB - In the field of Embodied Conversational Agent (ECA) one of the main challenges is to generate socially believable agents. The long run objective of the present study is to infer rules for the multimodal generation of agents’ socio-emotional behaviour. In this paper, we introduce the Social Multimodal Association Rules with Timing (SMART) algorithm. It proposes to learn the rules from the analysis of a multimodal corpus composed by audio-video recordings of human-human interactions. The proposed methodology consists in applying a Sequence Mining algorithm using automatically extracted Social Signals such as prosody, head movements and facial muscles activation as an input. This allows us to infer Temporal Association Rules for the behaviour generation. We show that this method can automatically compute Temporal Association Rules coherent with prior results found in the literature especially in the psychology and sociology fields. The results of a perceptive evaluation confirms the ability of a Temporal Association Rules based agent to express a specific stance.

KW - Embodied conversational agent

KW - Multi-modal social signal

KW - Sequence mining

KW - Signal processing

UR - https://www.scopus.com/pages/publications/84994493007

U2 - 10.1007/978-3-319-47665-0_16

DO - 10.1007/978-3-319-47665-0_16

M3 - Conference contribution

AN - SCOPUS:84994493007

SN - 9783319476643

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 175

EP - 189

BT - Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings

A2 - Khooshabeh, Peter

A2 - Traum, David

A2 - Swartout, William

A2 - Scherer, Stefan

A2 - Leuski, Anton

A2 - Kopp, Stefan

PB - Springer Verlag

T2 - 16th International Conference on Intelligent Virtual Agents, IVA 2016

Y2 - 20 September 2016 through 23 September 2016

ER -

Janssoone T, Clavel C, Bailly K, Richard G. Using temporal association rules for the synthesis of embodied conversational agents with a specific stance. In Khooshabeh P, Traum D, Swartout W, Scherer S, Leuski A, Kopp S, editors, Intelligent Virtual Agents - 16th International Conference, IVA 2016, Proceedings. Springer Verlag. 2016. p. 175-189. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-47665-0_16

Using temporal association rules for the synthesis of embodied conversational agents with a specific stance

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this