Lightweight Edge AI Framework for Real-Time Disease Prediction on Resource-Constrained Healthcare Devices

Abstract

The conflicting requirements of high diagnostic accuracy and limited computational resources pose a big challenge in real-time detection of disease, especially for resource constrained healthcare devices. We suggest a thin-edge AI solution that allows an efficient prediction of the disease in a certain device located at the edge of the network without depending on cloud connectivity. The framework starts with a medical images and numerical clinical data tailored pre-processing pipeline which consist of normalization, augmentation and noise reduction. In essence, we use a small deep learning model like MobileNetV3 or EfficientNet-Lite that use depthwise separable convolutions that reduce a model's floating point operations and parameters by many orders of magnitude. We then implement a multi-stage compression strategy to further take into account the constraints of the edge devices. Post-training quantization converts weights and activations from 32-bit to 8-bit integer representation, while structured pruning removes redundant connections via a binary mask. Knowledge distillation then transfers representational knowledge from a larger teacher model to the compact student model, optimizing a combined loss of cross-entropy and Kullback-Leibler divergence. The final compressed model is deployed on platforms like the NVIDIA Jetson Nano or ARM-based microcontrollers, where inference executes locally using a sigmoid or softmax activation for binary or multi-class classification, respectively. The primary contribution of this work is a holistic integration of architectural efficiency, compression techniques, and edge deployment that maintains diagnostic accuracy while achieving minimal latency and zero network dependency. Experimental results demonstrate that the proposed framework significantly reduces model size and inference time compared to standard approaches, making it suitable for real-time applications in remote or low-resource clinical settings. This work therefore provides a practical pathway toward democratizing AI-driven healthcare diagnostics