TY - GEN
T1 - TYPIC
T2 - 13th International Conference on Language Resources and Evaluation Conference, LREC 2022
AU - Naito, Shoichi
AU - Sawada, Shintaro
AU - Nakagawa, Chihiro
AU - Inoue, Naoya
AU - Yamaguchi, Kenshi
AU - Shimizu, Iori
AU - Mim, Farjana Sultana
AU - Singh, Keshav
AU - Inui, Kentaro
N1 - Funding Information:
This work was partly supported by JSPS KAKENHI Grant Number 22H00524 and NEDO JP1234567. The authors would like to thank Paul Reisert, members of the Tohoku NLP Lab, and the anonymous reviewers for their helpful feedback. We also would like to thank the assessors and the annotators for their time and effort.
Publisher Copyright:
© European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.
PY - 2022
Y1 - 2022
N2 - Providing feedback on the argumentation of the learner is essential for developing critical thinking skills, however, it requires a lot of time and effort. To mitigate the overload on teachers, we aim to automate a process of providing feedback, especially giving diagnostic comments which point out the weaknesses inherent in the argumentation. It is recommended to give specific diagnostic comments so that learners can recognize the diagnosis without misinterpretation. However, it is not obvious how the task of providing specific diagnostic comments should be formulated. We present a formulation of the task as template selection and slot filling to make an automatic evaluation easier and the behavior of the model more tractable. The key to the formulation is the possibility of creating a template set that is sufficient for practical use. In this paper, we define three criteria that a template set should satisfy: expressiveness, informativeness, and uniqueness, and verify the feasibility of creating a template set that satisfies these criteria as a first trial. We will show that it is feasible through an annotation study that converts diagnostic comments given in a text to a template format. The corpus used in the annotation study is publicly available.
AB - Providing feedback on the argumentation of the learner is essential for developing critical thinking skills, however, it requires a lot of time and effort. To mitigate the overload on teachers, we aim to automate a process of providing feedback, especially giving diagnostic comments which point out the weaknesses inherent in the argumentation. It is recommended to give specific diagnostic comments so that learners can recognize the diagnosis without misinterpretation. However, it is not obvious how the task of providing specific diagnostic comments should be formulated. We present a formulation of the task as template selection and slot filling to make an automatic evaluation easier and the behavior of the model more tractable. The key to the formulation is the possibility of creating a template set that is sufficient for practical use. In this paper, we define three criteria that a template set should satisfy: expressiveness, informativeness, and uniqueness, and verify the feasibility of creating a template set that satisfies these criteria as a first trial. We will show that it is feasible through an annotation study that converts diagnostic comments given in a text to a template format. The corpus used in the annotation study is publicly available.
KW - argument
KW - argumentation
KW - debate
KW - diagnostic comment
KW - formative feedback
UR - http://www.scopus.com/inward/record.url?scp=85144340866&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85144340866&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85144340866
T3 - 2022 Language Resources and Evaluation Conference, LREC 2022
SP - 5916
EP - 5928
BT - 2022 Language Resources and Evaluation Conference, LREC 2022
A2 - Calzolari, Nicoletta
A2 - Bechet, Frederic
A2 - Blache, Philippe
A2 - Choukri, Khalid
A2 - Cieri, Christopher
A2 - Declerck, Thierry
A2 - Goggi, Sara
A2 - Isahara, Hitoshi
A2 - Maegaard, Bente
A2 - Mariani, Joseph
A2 - Mazo, Helene
A2 - Odijk, Jan
A2 - Piperidis, Stelios
PB - European Language Resources Association (ELRA)
Y2 - 20 June 2022 through 25 June 2022
ER -