TY - CHAP
T1 - Dialogue System Live Competition
T2 - Identifying Problems with Dialogue Systems Through Live Event
AU - Higashinaka, Ryuichiro
AU - Funakoshi, Kotaro
AU - Inaba, Michimasa
AU - Tsunomori, Yuiko
AU - Takahashi, Tetsuro
AU - Akama, Reina
N1 - Publisher Copyright:
© 2020, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
PY - 2021
Y1 - 2021
N2 - We organized a competition entitled “the dialogue system live competition” in which the audience, consisting mainly of researchers in the dialogue community, watched and evaluated a live dialogue conducted between users and dialogue systems. The motivation behind the event was to cultivate state-of-the-art techniques in dialogue systems and enable the dialogue community to share the problems with current dialogue systems. There are two parts to the competition: preliminary selection and live event. In the preliminary selection, eleven systems were evaluated by crowd-sourcing. Three systems proceeded to the live event to perform dialogues with designated speakers and to be evaluated by the audience. This paper describes the design and procedure of the competition, the results of the preliminary selection and live event of the competition, and the problems we identified from the event.
AB - We organized a competition entitled “the dialogue system live competition” in which the audience, consisting mainly of researchers in the dialogue community, watched and evaluated a live dialogue conducted between users and dialogue systems. The motivation behind the event was to cultivate state-of-the-art techniques in dialogue systems and enable the dialogue community to share the problems with current dialogue systems. There are two parts to the competition: preliminary selection and live event. In the preliminary selection, eleven systems were evaluated by crowd-sourcing. Three systems proceeded to the live event to perform dialogues with designated speakers and to be evaluated by the audience. This paper describes the design and procedure of the competition, the results of the preliminary selection and live event of the competition, and the problems we identified from the event.
UR - http://www.scopus.com/inward/record.url?scp=85102632470&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85102632470&partnerID=8YFLogxK
U2 - 10.1007/978-981-15-9323-9_16
DO - 10.1007/978-981-15-9323-9_16
M3 - Chapter
AN - SCOPUS:85102632470
T3 - Lecture Notes in Electrical Engineering
SP - 185
EP - 199
BT - Lecture Notes in Electrical Engineering
PB - Springer Science and Business Media Deutschland GmbH
ER -