Evaluating the Generalization of Machine Learning and Deep Learning Models for Intrusion Detection Systems

Intrusion Detection Systems (IDS) play a crucial role in safeguarding digital networks against evolving cyber threats. Traditional IDS approaches, such as signature-based and anomaly-based detection, face challenges in adapting to novel attack patterns. This study evaluates the generalization capability of Machine Learning (ML) and Deep Learning (DL) models in detecting cyber-attacks across different datasets. Using CIC-IDS2017 and CIC-IDS2018 datasets, we implement and assess multiple classification models, including Decision Trees, Random Forest, Multilayer Perceptron, and Convolutional Neural Networks. Experimental results demonstrate that while models perform well when trained and tested on the same dataset, their effectiveness significantly declines when applied to unseen datasets. The MLP model outper-forms traditional ML classifiers in cross-dataset generalization, highlighting the need for robust model adaptation strategies. These findings emphasize the importance of enhancing IDS models for real-world deployment by improving their ability to generalize across diverse cyber-attack patterns.