Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder

Ryota Yahagi; Yuya Chiba; Takashi Nose; Akinori Ito

doi:10.1007/978-981-19-5538-9_9

Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder

Ryota Yahagi, Yuya Chiba, Takashi Nose, Akinori Ito

ENG - Communications Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Spoken dialogue systems need to determine when to respond to a user in addition to the response. Various cues, such as prosody, gaze, and facial expression are known to affect response timing. Recent studies have revealed that using the representation of a system response improves the performance of response timing prediction. However, it is difficult to directly use a future response with dialogue systems that require an entire user utterance to generate a response. This study proposes a neural-based response timing estimation model using past utterances to alleviate this problem. The proposed model is expected to consider the intention of the system response implicitly.

Original language	English
Title of host publication	Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021
Editors	Svetlana Stoyanchev, Stefan Ultes, Haizhou Li
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	133-141
Number of pages	9
ISBN (Print)	9789811955372
DOIs	https://doi.org/10.1007/978-981-19-5538-9_9
Publication status	Published - 2022
Event	12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021 - Virtual, Online Duration: 2021 Nov 15 → 2021 Nov 17

Publication series

Name	Lecture Notes in Electrical Engineering
Volume	943
ISSN (Print)	1876-1100
ISSN (Electronic)	1876-1119

Conference

Conference	12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021
City	Virtual, Online
Period	21/11/15 → 21/11/17

ASJC Scopus subject areas

Industrial and Manufacturing Engineering

Access to Document

10.1007/978-981-19-5538-9_9

Cite this

Yahagi, R., Chiba, Y., Nose, T., & Ito, A. (2022). Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder. In S. Stoyanchev, S. Ultes, & H. Li (Eds.), Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021 (pp. 133-141). (Lecture Notes in Electrical Engineering; Vol. 943). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-19-5538-9_9

Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder. / Yahagi, Ryota; Chiba, Yuya; Nose, Takashi et al.

Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021. ed. / Svetlana Stoyanchev; Stefan Ultes; Haizhou Li. Springer Science and Business Media Deutschland GmbH, 2022. p. 133-141 (Lecture Notes in Electrical Engineering; Vol. 943).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Yahagi, R, Chiba, Y, Nose, T & Ito, A 2022, Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder. in S Stoyanchev, S Ultes & H Li (eds), Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021. Lecture Notes in Electrical Engineering, vol. 943, Springer Science and Business Media Deutschland GmbH, pp. 133-141, 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021, Virtual, Online, 21/11/15. https://doi.org/10.1007/978-981-19-5538-9_9

Yahagi R, Chiba Y, Nose T , Ito A. Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder. In Stoyanchev S, Ultes S, Li H, editors, Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021. Springer Science and Business Media Deutschland GmbH. 2022. p. 133-141. (Lecture Notes in Electrical Engineering). doi: 10.1007/978-981-19-5538-9_9

Yahagi, Ryota ; Chiba, Yuya ; Nose, Takashi et al. / Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder. Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021. editor / Svetlana Stoyanchev ; Stefan Ultes ; Haizhou Li. Springer Science and Business Media Deutschland GmbH, 2022. pp. 133-141 (Lecture Notes in Electrical Engineering).

@inproceedings{7423f37f0d944e4992ccda32a96411f2,

title = "Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder",

abstract = "Spoken dialogue systems need to determine when to respond to a user in addition to the response. Various cues, such as prosody, gaze, and facial expression are known to affect response timing. Recent studies have revealed that using the representation of a system response improves the performance of response timing prediction. However, it is difficult to directly use a future response with dialogue systems that require an entire user utterance to generate a response. This study proposes a neural-based response timing estimation model using past utterances to alleviate this problem. The proposed model is expected to consider the intention of the system response implicitly.",

author = "Ryota Yahagi and Yuya Chiba and Takashi Nose and Akinori Ito",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021 ; Conference date: 15-11-2021 Through 17-11-2021",

year = "2022",

doi = "10.1007/978-981-19-5538-9_9",

language = "English",

isbn = "9789811955372",

series = "Lecture Notes in Electrical Engineering",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "133--141",

editor = "Svetlana Stoyanchev and Stefan Ultes and Haizhou Li",

booktitle = "Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021",

address = "Germany",

}

TY - GEN

T1 - Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder

AU - Yahagi, Ryota

AU - Chiba, Yuya

AU - Nose, Takashi

AU - Ito, Akinori

PY - 2022

Y1 - 2022

N2 - Spoken dialogue systems need to determine when to respond to a user in addition to the response. Various cues, such as prosody, gaze, and facial expression are known to affect response timing. Recent studies have revealed that using the representation of a system response improves the performance of response timing prediction. However, it is difficult to directly use a future response with dialogue systems that require an entire user utterance to generate a response. This study proposes a neural-based response timing estimation model using past utterances to alleviate this problem. The proposed model is expected to consider the intention of the system response implicitly.

AB - Spoken dialogue systems need to determine when to respond to a user in addition to the response. Various cues, such as prosody, gaze, and facial expression are known to affect response timing. Recent studies have revealed that using the representation of a system response improves the performance of response timing prediction. However, it is difficult to directly use a future response with dialogue systems that require an entire user utterance to generate a response. This study proposes a neural-based response timing estimation model using past utterances to alleviate this problem. The proposed model is expected to consider the intention of the system response implicitly.

UR - http://www.scopus.com/inward/record.url?scp=85142755278&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85142755278&partnerID=8YFLogxK

U2 - 10.1007/978-981-19-5538-9_9

DO - 10.1007/978-981-19-5538-9_9

M3 - Conference contribution

AN - SCOPUS:85142755278

SN - 9789811955372

T3 - Lecture Notes in Electrical Engineering

SP - 133

EP - 141

BT - Conversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021

A2 - Stoyanchev, Svetlana

A2 - Ultes, Stefan

A2 - Li, Haizhou

PB - Springer Science and Business Media Deutschland GmbH

T2 - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021

Y2 - 15 November 2021 through 17 November 2021

ER -

Multimodal Dialogue Response Timing Estimation Using Dialogue Context Encoder

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this