Dialogue System Live Competition: Identifying Problems with Dialogue Systems Through Live Event

Ryuichiro Higashinaka, Kotaro Funakoshi, Michimasa Inaba, Yuiko Tsunomori, Tetsuro Takahashi, Reina Akama

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

7 Citations (Scopus)


We organized a competition entitled “the dialogue system live competition” in which the audience, consisting mainly of researchers in the dialogue community, watched and evaluated a live dialogue conducted between users and dialogue systems. The motivation behind the event was to cultivate state-of-the-art techniques in dialogue systems and enable the dialogue community to share the problems with current dialogue systems. There are two parts to the competition: preliminary selection and live event. In the preliminary selection, eleven systems were evaluated by crowd-sourcing. Three systems proceeded to the live event to perform dialogues with designated speakers and to be evaluated by the audience. This paper describes the design and procedure of the competition, the results of the preliminary selection and live event of the competition, and the problems we identified from the event.

Original languageEnglish
Title of host publicationLecture Notes in Electrical Engineering
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages15
Publication statusPublished - 2021

Publication series

NameLecture Notes in Electrical Engineering
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119


Dive into the research topics of 'Dialogue System Live Competition: Identifying Problems with Dialogue Systems Through Live Event'. Together they form a unique fingerprint.

Cite this