| Presentation | 2023-06-16 Study on Sign Language Recognition Using Deep Learning Kouki Ikeda, Tsutomu Kimura, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | This study aims to recognize words used in sentences in sign language recognition using machine learning. In order to train a sign language recognition model that takes into account the transitions that exist in a sign sentence, more sign sentence data is required. Therefore, we increase the amount of data by using two-word sentences instead of complete sign sentences, which are difficult to collect. The recognition model is based on Conformer, which has shown high performance in speech recognition, and it extracts global and local features of the signed sentences. Additionally, certain parameters from the Conformer, trained on signed words, are incorporated into the recognition model as a sign language dictionary to extract features specific to signed words. Furthermore, we utilize Connectionist Temporal Classification as the loss function for the recognition model. With these structures in place, we achieved a maximum recognition rate of approximately 79% on the Kimura Lab's sign language dataset. However, the recognition rate for complete sentences decreased as the number of learned words increased, indicating that the sign language dictionary structure was not fully utilized. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Sign Language Recognition / Deep Learning / Connectionist Temporal Classification / Conformer |
| Paper # | WIT2023-2 |
| Date of Issue | 2023-06-09 (WIT) |
| Conference Information | |
| Committee | WIT |
|---|---|
| Conference Date | 2023/6/16(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Okinawa Industry Support Center |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Shinji Sakou(Nagoya Inst. of Tech.) |
| Vice Chair | Tomohiro Amemiya(Univ. of Tokyo) |
| Secretary | Tomohiro Amemiya(Saitama Industrial Tech. Center) |
| Assistant | Minako Hosono(AIST) / Aki Sugano(Univ. of Toyama) / Tomoyasu Komori(NHK) |
| Paper Information | |
| Registration To | Technical Committee on Well-being Information Technology |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Study on Sign Language Recognition Using Deep Learning |
| Sub Title (in English) | Recognition by conformer with the introduction of two-word sentences and sign language dictionary structure |
| Keyword(1) | Sign Language Recognition |
| Keyword(2) | Deep Learning |
| Keyword(3) | Connectionist Temporal Classification |
| Keyword(4) | Conformer |
| 1st Author's Name | Kouki Ikeda |
| 1st Author's Affiliation | National Institute of Technology,Toyota College(National Institute of Technology,Toyota College) |
| 2nd Author's Name | Tsutomu Kimura |
| 2nd Author's Affiliation | National Institute of Technology,Toyota College(National Institute of Technology,Toyota College) |
| Date | 2023-06-16 |
| Paper # | WIT2023-2 |
| Volume (vol) | vol.123 |
| Number (no) | WIT-81 |
| Page | pp.pp.6-11(WIT), |
| #Pages | 6 |
| Date of Issue | 2023-06-09 (WIT) |