Food Image Recognition: From CNNs to Transformers and Multimodal Learning

Food recognition is a challenging fine-grained classification task with practical applications in health monitoring, dietary assessment, and intelligent food services. Although convolutional neural networks, transformer-based vision models, and multimodal approaches have advanced rapidly, their comparative strengths for food recognition remain insufficiently explored. To bridge this gap, in this paper we present a unified experimental framework that enables robust evaluation and yields new insights into the design and deployment of effective food recognition systems. Results reveal that transformer architectures consistently outperform convolutional baselines, achieving over 94% accuracy, while multimodal frameworks demonstrate competitive zero-shot performance without task-specific fine-tuning. Beyond raw performance, our analysis uncovers systematic error patterns linked to high intra-class variability and visual similarity across food categories. We further introduce a lightweight web application for real time inference and explainable predictions, highlighting the practical implications of our findings.