MTAKD: multi-teacher agreement knowledge distillation for edge AI skin disease diagnosis

Abstract

Skin disease diagnosis remains challenging in remote areas due to limited access to dermatology specialists and unreliable internet connectivity. Edge AI offers a potential solution by offloading the inference process from cloud servers to mobile devices. This research proposes a novel Multi-Teacher Knowledge Distillation (MTAKD) framework to optimize small model performance for mobile edge deployment. MTAKD uses dynamic teacher agreement as an indicator of knowledge reliability and weights multiple knowledge sources for each input. MTAKD integrates three novel algorithms, such as Agreement Weighted Knowledge Distillation for prediction knowledge, Attention Agreement Knowledge Distillation for spatial attention guidance, and Relational Agreement Knowledge Distillation for embedding relations. MTAKD achieves mean accuracies of 87.53% on the ISIC 2019 dataset and 44.75% on the Fitzpatrick17k-C dataset, outperforming the highest accuracy on benchmark frameworks by 0.75 and 1.1%. In addition, the student model demonstrates improved explainability, with insertion metric scores of 0.6796 AUC on ISIC 2019 and 0.1724 AUC on Fitzpatrick17k-C. Deployment on a mobile prototype demonstrates significant efficiency gains with 49.8 times smaller size and 352 times faster inference. These results support the proposed MTAKD as an effective and practical solution for edge AI skin disease diagnosis.

Ähnliche Arbeiten