| Presentation | 2024-02-29 Communication cost and performance evaluation of each learning method in Federated learning with LLM Takumi Fukami, Yusuke Yamasaki, Iifan Tyou, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In recent years, a large amount of diverse data have been generated by various devices and organisations, and there has been a growing movement to utilise these data. In general, data collection is indispensable for data utilisation such as learning and prediction by AI. However, if the data is confidential information, the data should not be collected. One solution to this problem is federated learning, which can learn high-performance AI by sharing only the AI models learned by each data holder without collecting their data. It is well known that the larger the models to be learned, the higher the communication cost, but it is not clear how much communication is actually required in operation. In this paper, we report the communication cost and performance of learning the large language model, which requires a large amount of communication cost, using each learning method of federated learning. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Federated Learning / Natural language processing / LLM / Communication cost |
| Paper # | IN2023-66 |
| Date of Issue | 2024-02-22 (IN) |
| Conference Information | |
| Committee | NS / IN |
|---|---|
| Conference Date | 2024/2/29(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Okinawa Convention Center |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | General |
| Chair | Tetsuya Oishi(NTT) / Kunio Hato(NTT) |
| Vice Chair | Takumi Miyoshi(Shibaura Inst. of Tech.) / Tsutomu Murase(Nagoya Univ.) |
| Secretary | Takumi Miyoshi(NTT) / Tsutomu Murase(Kogakuin Univ.) |
| Assistant | Hiroshi Yamamoto(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Network Systems / Technical Committee on Information Networks |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Communication cost and performance evaluation of each learning method in Federated learning with LLM |
| Sub Title (in English) | |
| Keyword(1) | Federated Learning |
| Keyword(2) | Natural language processing |
| Keyword(3) | LLM |
| Keyword(4) | Communication cost |
| 1st Author's Name | Takumi Fukami |
| 1st Author's Affiliation | Nippon Telegraph and Telephone Corporation(NTT) |
| 2nd Author's Name | Yusuke Yamasaki |
| 2nd Author's Affiliation | Nippon Telegraph and Telephone Corporation(NTT) |
| 3rd Author's Name | Iifan Tyou |
| 3rd Author's Affiliation | Nippon Telegraph and Telephone Corporation(NTT) |
| Date | 2024-02-29 |
| Paper # | IN2023-66 |
| Volume (vol) | vol.123 |
| Number (no) | IN-398 |
| Page | pp.pp.7-12(IN), |
| #Pages | 6 |
| Date of Issue | 2024-02-22 (IN) |