A Hybrid CNN-Vision Transformer Framework for Multi-Class Plant Disease Identification from Leaf Images

Abstract

Accurate and timely identification of plant diseases from leaf images remains a critical challenge in precision agriculture, particularly when diseases manifest with spatially disjoint symptoms and subtle textural variations. We propose a hybrid deep learning framework that synergistically combines convolutional neural networks (CNNs) with Vision Transformers (ViTs) for robust multi-class plant disease diagnosis. The proposed architecture processes an input leaf image through two parallel pathways: a CNN backbone extracts hierarchical local features such as necrotic regions and vein patterns, while a Vision Transformer encoder models global contextual relationships across image patches via multi-head self-attention. These complementary representations are then fused through vector concatenation and passed to a fully connected classification head with softmax activation. The entire network is trained end-to-end using categorical cross-entropy loss optimized with Adam, and we apply dropout, weight decay, and early stopping to mitigate overfitting during transfer learning. The primary novelty lies in the explicit integration of local texture details and global disease distribution patterns within a single unified framework, which addresses the limitations of pure CNNs in capturing long-range dependencies and of pure ViTs in preserving fine-grained spatial information. Our method achieves superior classification performance across multiple plant species and disease categories, demonstrating significant improvements in accuracy and robustness compared to standalone CNN or ViT baselines. This work provides a practical and scalable solution for automated plant disease surveillance, with potential to reduce crop losses and support sustainable agricultural practices.