| 講演名 | 2021-09-16 Ensemble BERT-BiLSTM-CNN Model for Sequence Classification Vuong Thi Hong(NII/総研大), Takasu Atsuhiro(NII), |
|---|---|
| PDFダウンロードページ |  PDFダウンロードページへ |
| 抄録(和) | Ensemble methods use multiple learning algorithms to obtain better predictive performance. Currently, deep learning models with multilayer processing architecture are showed that the performance is better than the traditional classification models. Ensemble deep learning models combine the advantages of both ensemble learning and deep learning such that the final model has better performance. This paper presents a novel ensemble deep learning method, achieving robust and effective sequence classification facing sparse data. We use the BERT (Bidirectional Encoder Representation from Transformers) as the word embedding method. Then, we integrate the BiLSTM (Bidirectional Long Short-Term Memory) and CNN (Convolutional Neural Network) with an attention mechanism for sequence classification. We evaluate our ensemble models with two datasets with the different baseline methods. The first dataset is from IMDB and contains 50,000 movie reviews, labeled with two sentiment classes. The second dataset is the Toxic Comment Dataset with more than 150,000 comments for six classes. The experimental results show that our proposed method provides an accurate, reliable, and effective solution for sequence data classification. |
| 抄録(英) | Ensemble methods use multiple learning algorithms to obtain better predictive performance. Currently, deep learning models with multilayer processing architecture are showed that the performance is better than the traditional classification models. Ensemble deep learning models combine the advantages of both ensemble learning and deep learning such that the final model has better performance. This paper presents a novel ensemble deep learning method, achieving robust and effective sequence classification facing sparse data. We use the BERT (Bidirectional Encoder Representation from Transformers) as the word embedding method. Then, we integrate the BiLSTM (Bidirectional Long Short-Term Memory) and CNN (Convolutional Neural Network) with an attention mechanism for sequence classification. We evaluate our ensemble models with two datasets with the different baseline methods. The first dataset is from IMDB and contains 50,000 movie reviews, labeled with two sentiment classes. The second dataset is the Toxic Comment Dataset with more than 150,000 comments for six classes. The experimental results show that our proposed method provides an accurate, reliable, and effective solution for sequence data classification. |
| キーワード(和) | Ensemble deep learning / Sequence classification / BERT / BiLSTM / CNN |
| キーワード(英) | Ensemble deep learning / Sequence classification / BERT / BiLSTM / CNN |
| 資料番号 | DE2021-12 |
| 発行日 | 2021-09-09 (DE) |
| 研究会情報 | |
| 研究会 | DE / IPSJ-DBS / IPSJ-IFAT |
|---|---|
| 開催期間 | 2021/9/16(から2日開催) |
| 開催地（和） | オンライン開催 |
| 開催地（英） | Online |
| テーマ（和） | ビッグデータを対象とした管理・情報検索・知識獲得および一般 |
| テーマ（英） | Management, information retrieval, knowledge acquisition and general for big data |
| 委員長氏名（和） | 吉田 尚史(駒澤大) |
| 委員長氏名（英） | Naofumi Yoshida(Komazawa Univ.) |
| 副委員長氏名（和） | 的野 晃整(産総研) / 鈴木 優(岐阜大) |
| 副委員長氏名（英） | Akiyoshi Matono(AIST) / Yu Suzuki(Gifu Univ.) |
| 幹事氏名（和） | 鷹野 孝典(神奈川工科大) / 新妻 弘崇(阪大) |
| 幹事氏名（英） | Kosuke Takano(Kanagawa Inst. of Tech.) / Hirotaka Niitsuma(Osaka Univ.) |
| 幹事補佐氏名（和） | 本多 賢(駒澤大) / 野宮 浩揮(京都工繊大) |
| 幹事補佐氏名（英） | Ken Honda(Komazawa Univ.) / Hiroki Nomiya(Kyoto Inst. of Tech) |
| 講演論文情報詳細 | |
| 申込み研究会 | Technical Committee on Data Engineering / Special Interest Group on Database System / Special Interest Group on Information Fundamentals and Access Technologies |
|---|---|
| 本文の言語 | ENG |
| タイトル（和） | |
| サブタイトル（和） | |
| タイトル（英） | Ensemble BERT-BiLSTM-CNN Model for Sequence Classification |
| サブタイトル（和） | |
| キーワード(1)（和/英） | Ensemble deep learning / Ensemble deep learning |
| キーワード(2)（和/英） | Sequence classification / Sequence classification |
| キーワード(3)（和/英） | BERT / BERT |
| キーワード(4)（和/英） | BiLSTM / BiLSTM |
| キーワード(5)（和/英） | CNN / CNN |
| 第 1 著者 氏名（和/英） | Vuong Thi Hong / Vuong Thi Hong |
| 第 1 著者 所属（和/英） | National Institute of Informatics/SOKENDAI(略称：NII/総研大) National Institute of Informatics/SOKENDAI(略称：NII/SOKENDAI) |
| 第 2 著者 氏名（和/英） | Takasu Atsuhiro / Takasu Atsuhiro |
| 第 2 著者 所属（和/英） | National Institute of Informatics(略称：NII) National Institute of Informatics(略称：NII) |
| 発表年月日 | 2021-09-16 |
| 資料番号 | DE2021-12 |
| 巻番号（vol） | vol.121 |
| 号番号（no） | DE-176 |
| ページ範囲 | pp.1-6(DE), |
| ページ数 | 6 |
| 発行日 | 2021-09-09 (DE) |