| Presentation | 2023-08-31 Enhancing Communication Efficiency for UAV Networks through Knowledge Distillation and Transfer Learning in Federated Learning Yalong Li, Zhaoyang Du, Celimuge Wu, Tsutomu Yoshinaga, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Federated learning (FL) in unmanned aerial vehicles (UAVs) networks demands considerable communication resources to transfer model data between the central server and UAVs (FL clients). However, different UAVs may have different communication capabilities due to the UAV's maneuverability and heterogeneity, where limited communication resource could be bottle neck for FL performance. In this paper, we first introduce a knowledge distillation based approach that places two different models with different sizes, namely the teacher model and student model, for FL client to make a trade-off between the FL performance and communication cost. Then, we propose a novel model switching method to switch between the teacher model and student model to adapt to the dynamic feature of UAV networks. Specifically, considering available communication bandwidth and learning accuracy, we design a threshold-based model switching algorithm (TBMSA) and determine the threshold based on the k-means method (DTBKM) to accurately and quickly determine the switching point. In addition, for the knowledge transfer between models, we design a knowledge inheritance based on a transfer learning (KIBTL) algorithm, which transfers knowledge from one model to another. Experiments show that the proposed model switching algorithm achieves significant performance improvements as compared to existing baselines. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | UAV Networks / Federated Learning / Model Switching / Knowledge Distillation / Transfer Learning |
| Paper # | CQ2023-30 |
| Date of Issue | 2023-08-24 (CQ) |
| Conference Information | |
| Committee | CQ |
|---|---|
| Conference Date | 2023/8/31(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Tenjin-Misaki Sports Park |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Wireless Communications Quality, 6G, IoT, Resource Management, Wireless Transmission, Cross layer Technologies, etc. |
| Chair | Takefumi Hiraguri(Nippon Inst. of Tech.) |
| Vice Chair | Takahiro Matsuda(Tokyo Metropolitan Univ.) / Gou Hasegawa(Tohoku Univ.) / Sumaru Niida(KDDI Research) |
| Secretary | Takahiro Matsuda(NTT) / Gou Hasegawa(Tama Univ.) / Sumaru Niida(Tsukuba Univ.) |
| Assistant | Ryo Nakamura(Fukuoka Univ.) / Toshiro Nakahira(NTT) / Kenta Tsukatsune(Okayama Univ. of Science) |
| Paper Information | |
| Registration To | Technical Committee on Communication Quality |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Enhancing Communication Efficiency for UAV Networks through Knowledge Distillation and Transfer Learning in Federated Learning |
| Sub Title (in English) | |
| Keyword(1) | UAV Networks |
| Keyword(2) | Federated Learning |
| Keyword(3) | Model Switching |
| Keyword(4) | Knowledge Distillation |
| Keyword(5) | Transfer Learning |
| 1st Author's Name | Yalong Li |
| 1st Author's Affiliation | the university of electro-communications(UEC) |
| 2nd Author's Name | Zhaoyang Du |
| 2nd Author's Affiliation | the university of electro-communications(UEC) |
| 3rd Author's Name | Celimuge Wu |
| 3rd Author's Affiliation | the university of electro-communications(UEC) |
| 4th Author's Name | Tsutomu Yoshinaga |
| 4th Author's Affiliation | the university of electro-communications(UEC) |
| Date | 2023-08-31 |
| Paper # | CQ2023-30 |
| Volume (vol) | vol.123 |
| Number (no) | CQ-174 |
| Page | pp.pp.26-31(CQ), |
| #Pages | 6 |
| Date of Issue | 2023-08-24 (CQ) |