講演抄録/キーワード |
講演名 |
2021-12-16 14:55
Fully automatic scoring of handwritten descriptive answers in Japanese language tests ○Hung Tuan Nguyen・Cuong Tuan Nguyen(TUAT)・Haruki Oka(UTokyo)・Tsunenori Ishioka(The National Center for University Entrance Examinations)・Masaki Nakagawa(TUAT) PRMU2021-32 |
抄録 |
(和) |
(まだ登録されていません) |
(英) |
This paper presents an experiment of automatically scoring handwritten descriptive answers in the trial tests for the new Japanese university entrance examination, which were made for about 120,000 examinees in 2017 and 2018. There are about 400,000 answers with more than 20 million characters. Although all answers have been scored by human examiners, handwritten characters are not labelled. We present our attempt to adapt deep neural network-based handwriting recognizers trained on a labelled handwriting dataset into this unlabeled answer set. Our proposed method combines different training strategies, ensembles multiple recognizers, and uses a language model built from a large general corpus to avoid overfitting into specific data. In our experiment, the proposed method records character accuracy of over 97% using about 2,000 verified labelled answers that account for less than 0.5% of the dataset. Then, the recognized answers are fed into a pre-trained automatic scoring system based on the BERT model without correcting misrecognized characters and providing rubric annotations. The automatic scoring system achieves from 0.84 to 0.98 of Quadratic Weighted Kappa (QWK). As QWK is over 0.8, it represents acceptable similarity of scoring between the automatic scoring system and the human examiners. These results are promising for further research on end-to-end automatic scoring of descriptive answers. |
キーワード |
(和) |
/ / / / / / / |
(英) |
handwritten language answers / handwriting recognition / automatic scoring / ensemble recognition / deep neural networks / / / |
文献情報 |
信学技報, vol. 121, no. 304, PRMU2021-32, pp. 45-50, 2021年12月. |
資料番号 |
PRMU2021-32 |
発行日 |
2021-12-09 (PRMU) |
ISSN |
Online edition: ISSN 2432-6380 |
著作権に ついて |
技術研究報告に掲載された論文の著作権は電子情報通信学会に帰属します.(許諾番号:10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
PDFダウンロード |
PRMU2021-32 |
|