| Presentation | 2023-10-27 Online Structured Job Interview Assessment Using Multimodal Transformer and Prompt Learning Shengzhou Yi, Toshiaki Yamasaki, Toshihiko Yamasaki, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Online interviews have gained significant traction in recruitment processes due to their enhanced efficiency. Among various interview formats, structured interviews stand out for their ability to mitigate interviewer subjectivity by asking predetermined questions. In this research, we propose a multimodal transformer to assess the online structured job interviews using text and audio features. Because some instances lack audio modality, we use prompt learning to discern between samples with complete modalities and those without audio modality. To address the imbalanced distribution of the dataset, we introduce the re-sampling and margin ranking loss to improve the model robustness. We examine our approach in predicting seven assessment criteria. For the experimental results, the proposed approach achieves an average accuracy of 75.09% in the binary classification of high and low scores. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Multimodal RecognitionJob InterviewPrompt LearningMissing Modality |
| Paper # | MVE2023-28 |
| Date of Issue | 2023-10-19 (MVE) |
| Conference Information | |
| Committee | MVE / VRSJ-SIG-MR / IPSJ-EC / HI-SIG-DeMO / VRSJ-SIG-CS |
|---|---|
| Conference Date | 2023/10/26(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Kiyoshi Kiyokawa(NAIST) |
| Vice Chair | Sumaru Niida(KDDI Research) |
| Secretary | Sumaru Niida(NAIST) / (DNP) / (Kyushu Univ.) / (NTT) |
| Assistant | Hidehiko Shishido(Univ. of Tsukuba) / Atsushi Nakazawa(Kyoto Univ.) / Naoya Tojo(KDDI Research) / Naoki Hagiyama(NTT) / Yuji Tatada(Univ. of Tokyo) |
| Paper Information | |
| Registration To | Technical Committee on Media Experience and Virtual Environment / SIG-MR / Special Interest Group on Entertainment Computing / Special Interest Group on De-vice Media Oriented UI / SIG-CS |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Online Structured Job Interview Assessment Using Multimodal Transformer and Prompt Learning |
| Sub Title (in English) | |
| Keyword(1) | Multimodal RecognitionJob InterviewPrompt LearningMissing Modality |
| 1st Author's Name | Shengzhou Yi |
| 1st Author's Affiliation | The University of Tokyo(UTokyo) |
| 2nd Author's Name | Toshiaki Yamasaki |
| 2nd Author's Affiliation | talent and assessment Inc.(taleasse) |
| 3rd Author's Name | Toshihiko Yamasaki |
| 3rd Author's Affiliation | The University of Tokyo(UTokyo) |
| Date | 2023-10-27 |
| Paper # | MVE2023-28 |
| Volume (vol) | vol.123 |
| Number (no) | MVE-228 |
| Page | pp.pp.34-39(MVE), |
| #Pages | 6 |
| Date of Issue | 2023-10-19 (MVE) |