| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2022-07-04 16:00 Unsupervised Learning of a Dynamic Task Ordering Model for Crowdsourcing Ryo Yanagisawa (Waseda Univ.), Susumu Saito, Teppei Nakano (ifLab Inc.), Tetsunori Kobayashi, Tetsuji Ogawa (Waseda Univ.)  AI2022-14 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | An unsupervised learning method for a dynamic task ordering model that optimizes the number of orders according to the difficulty of the data was proposed as a framework for efficiently ensuring annotation quality through crowdsourcing. Since responses collected by crowdsourcing contain errors, the responses were collected from multiple workers for each sample and then aggregated by majority voting to ensure reliability. However, since the monetary cost increases as the number of orders increases, it is desirable to reduce the number of workers who perform majority voting while maintaining the high accuracy of the final label. Therefore, we focus on a dynamic task ordering model that continues to place orders to workers until the variation in responses becomes sufficiently small, based on the assumption that the smaller the variation in responses by multiple workers, the more reliable the majority decision is. The present study proposed a method for unsupervised learning of model parameters such that label errors and ordering costs are minimized. Experimental comparisons on an annotation task for livestock surveillance images demonstrated the effectiveness of the proposed method: it achieved performance comparable to supervised learning and significantly reduced the number of orders without significantly degrading accuracy compared to simple majority voting, which emphasizes accuracy. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | Crowdsourcing / Quality control / Unsupervised learning / Dynamic task ordering / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 122, no. 94, AI2022-14, pp. 72-76, July 2022. | |
| Paper # | AI2022-14 | |
| Date of Issue | 2022-06-27 (AI) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | AI2022-14 |
|
| Conference Information | |
| Committee | AI |
| Conference Date | 2022-07-04 - 2022-07-04 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Paper Information | |
| Registration To | AI |
| Conference Code | 2022-07-AI |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Unsupervised Learning of a Dynamic Task Ordering Model for Crowdsourcing |
| Sub Title (in English) | |
| Keyword(1) | Crowdsourcing |
| Keyword(2) | Quality control |
| Keyword(3) | Unsupervised learning |
| Keyword(4) | Dynamic task ordering |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Ryo Yanagisawa |
| 1st Author's Affiliation | Waseda University (Waseda Univ.) |
| 2nd Author's Name | Susumu Saito |
| 2nd Author's Affiliation | Intelligent Framework Lab Inc. (ifLab Inc.) |
| 3rd Author's Name | Teppei Nakano |
| 3rd Author's Affiliation | Intelligent Framework Lab Inc. (ifLab Inc.) |
| 4th Author's Name | Tetsunori Kobayashi |
| 4th Author's Affiliation | Waseda University (Waseda Univ.) |
| 5th Author's Name | Tetsuji Ogawa |
| 5th Author's Affiliation | Waseda University (Waseda Univ.) |
| 6th Author's Name | |
| 6th Author's Affiliation | () |
| 7th Author's Name | |
| 7th Author's Affiliation | () |
| 8th Author's Name | |
| 8th Author's Affiliation | () |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2022-07-04 16:00:00 |
| Presentation Time | 20 minutes |
| Registration for | AI |
| Paper # | AI2022-14 |
| Volume (vol) | vol.122 |
| Number (no) | no.94 |
| Page | pp.72-76 |
| #Pages | 5 |
| Date of Issue | 2022-06-27 (AI) |
[Return to Top Page]