| Presentation | 2021-12-16 Fully automatic scoring of handwritten descriptive answers in Japanese language tests Hung Tuan Nguyen, Cuong Tuan Nguyen, Haruki Oka, Tsunenori Ishioka, Masaki Nakagawa, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | This paper presents an experiment of automatically scoring handwritten descriptive answers in the trial tests for the new Japanese university entrance examination, which were made for about 120,000 examinees in 2017 and 2018. There are about 400,000 answers with more than 20 million characters. Although all answers have been scored by human examiners, handwritten characters are not labelled. We present our attempt to adapt deep neural network-based handwriting recognizers trained on a labelled handwriting dataset into this unlabeled answer set. Our proposed method combines different training strategies, ensembles multiple recognizers, and uses a language model built from a large general corpus to avoid overfitting into specific data. In our experiment, the proposed method records character accuracy of over 97% using about 2,000 verified labelled answers that account for less than 0.5% of the dataset. Then, the recognized answers are fed into a pre-trained automatic scoring system based on the BERT model without correcting misrecognized characters and providing rubric annotations. The automatic scoring system achieves from 0.84 to 0.98 of Quadratic Weighted Kappa (QWK). As QWK is over 0.8, it represents acceptable similarity of scoring between the automatic scoring system and the human examiners. These results are promising for further research on end-to-end automatic scoring of descriptive answers. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | handwritten language answershandwriting recognitionautomatic scoringensemble recognitiondeep neural networks |
| Paper # | PRMU2021-32 |
| Date of Issue | 2021-12-09 (PRMU) |
| Conference Information | |
| Committee | PRMU |
|---|---|
| Conference Date | 2021/12/16(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Seiichi Uchida(Kyushu Univ.) |
| Vice Chair | Masakazu Iwamura(Osaka Pref. Univ.) / Mitsuru Anpai(Denso IT Lab.) |
| Secretary | Masakazu Iwamura(NTT) / Mitsuru Anpai(Tottori Univ.) |
| Assistant | Kouta Yamaguchi(CyberAgent) / Yusuke Matsui(Univ. of Tokyo) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Fully automatic scoring of handwritten descriptive answers in Japanese language tests |
| Sub Title (in English) | |
| Keyword(1) | handwritten language answershandwriting recognitionautomatic scoringensemble recognitiondeep neural networks |
| 1st Author's Name | Hung Tuan Nguyen |
| 1st Author's Affiliation | Tokyo University of Agriculture and Technology(TUAT) |
| 2nd Author's Name | Cuong Tuan Nguyen |
| 2nd Author's Affiliation | Tokyo University of Agriculture and Technology(TUAT) |
| 3rd Author's Name | Haruki Oka |
| 3rd Author's Affiliation | The University of Tokyo(UTokyo) |
| 4th Author's Name | Tsunenori Ishioka |
| 4th Author's Affiliation | The National Center for University Entrance Examinations(The National Center for University Entrance Examinations) |
| 5th Author's Name | Masaki Nakagawa |
| 5th Author's Affiliation | Tokyo University of Agriculture and Technology(TUAT) |
| Date | 2021-12-16 |
| Paper # | PRMU2021-32 |
| Volume (vol) | vol.121 |
| Number (no) | PRMU-304 |
| Page | pp.pp.45-50(PRMU), |
| #Pages | 6 |
| Date of Issue | 2021-12-09 (PRMU) |