Summary
International Conference on Machine Vision Applications
2023
Session Number:P1
Session:
Number:P1-10
Combining Knowledge Distillation and Transfer Learning for Sensor Fusion in Visible and Thermal Camera-based Person Classification
John Vijay, Kawanishi Yasutomo,
pp.-
Publication Date:2023/07/23
Online ISSN:2188-5079
PDF download
Summary:
Visible and thermal camera-based sensor fusion has been shown to address the limitations and enhance the robustness of visible camera-based person classification. In this paper, we propose to further enhance the classification accuracy of visible-thermal person classification using transfer learning, knowledge distillation, and the vision transformer. In our work, the visible-thermal person classifier is implemented using the vision transformer. The proposed classifier is trained using transfer learning and knowledge distillation techniques. To train the proposed classifier, visible and thermal teacher models are implemented using the vision transformers. The multimodal classifier learns from the two teachers using a novel loss function which incorporates knowledge distillation. The proposed method is validated on the public Speaking Faces dataset containing 141 people. A comparative analysis with baseline algorithms and an ablation study is performed. The results show that the proposed framework reports an enhanced classification accuracy.