Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes

Maxime Lanvin; Pierre François Gimenez; Yufei Han; Frédéric Majorczyk; Ludovic Mé; Éric Totel

doi:10.1007/978-3-031-31108-6_2

Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes

Maxime Lanvin
, Pierre François Gimenez
, Yufei Han
, Frédéric Majorczyk
, Ludovic Mé
, Éric Totel

IRISA

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

Résumé

Among the difficulties encountered in building datasets to evaluate intrusion detection tools, a tricky part is the process of labelling the events into malicious and benign classes. The labelling correctness is paramount for the quality of the evaluation of intrusion detection systems but is often considered as the ground truth by practitioners and is rarely verified. Another difficulty lies in the correct capture of the network packets. If it is not the case, the characteristics of the network flows generated from the capture could be modified and lead to false results. In this paper, we present several flaws we identified in the labelling of the CICIDS2017 dataset and in the traffic capture, such as packet misorder, packet duplication and attack that were performed but not correctly labelled. Finally, we assess the impact of these different corrections on the evaluation of supervised intrusion detection approaches.

langue originale	Anglais
titre	Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers
rédacteurs en chef	Slim Kallel, Mohamed Jmaiel, Ahmed Hadj Kacem, Mohammad Zulkernine, Frédéric Cuppens, Nora Cuppens
Editeur	Springer Science and Business Media Deutschland GmbH
Pages	18-33
Nombre de pages	16
ISBN (imprimé)	9783031311079
Les DOIs	https://doi.org/10.1007/978-3-031-31108-6_2
état	Publié - 1 janv. 2023
Evénement	17th International Conference on Risks and Security of Internet and Systems, CRiSIS 2022 - Sousse, Tunisie Durée: 7 déc. 2022 → 9 déc. 2022

Série de publications

Nom	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13857 LNCS
ISSN (imprimé)	0302-9743
ISSN (Electronique)	1611-3349

Une conférence

Une conférence	17th International Conference on Risks and Security of Internet and Systems, CRiSIS 2022
Pays/Territoire	Tunisie
La ville	Sousse
période	7/12/22 → 9/12/22

Accès au document

10.1007/978-3-031-31108-6_2

Contient cette citation

Lanvin, M., Gimenez, P. F., Han, Y., Majorczyk, F., Mé, L., & Totel, É. (2023). Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes. Dans S. Kallel, M. Jmaiel, A. Hadj Kacem, M. Zulkernine, F. Cuppens, & N. Cuppens (eds.), Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers (p. 18-33). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol 13857 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-31108-6_2

Lanvin, Maxime ; Gimenez, Pierre François ; Han, Yufei et al. / Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes. Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers. Editeur / Slim Kallel ; Mohamed Jmaiel ; Ahmed Hadj Kacem ; Mohammad Zulkernine ; Frédéric Cuppens ; Nora Cuppens. Springer Science and Business Media Deutschland GmbH, 2023. p. 18-33 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0ebd3134b6984768bd1f0df8a9e0cb23,

title = "Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes",

abstract = "Among the difficulties encountered in building datasets to evaluate intrusion detection tools, a tricky part is the process of labelling the events into malicious and benign classes. The labelling correctness is paramount for the quality of the evaluation of intrusion detection systems but is often considered as the ground truth by practitioners and is rarely verified. Another difficulty lies in the correct capture of the network packets. If it is not the case, the characteristics of the network flows generated from the capture could be modified and lead to false results. In this paper, we present several flaws we identified in the labelling of the CICIDS2017 dataset and in the traffic capture, such as packet misorder, packet duplication and attack that were performed but not correctly labelled. Finally, we assess the impact of these different corrections on the evaluation of supervised intrusion detection approaches.",

keywords = "dataset labelling, intrusion detection, machine learning",

author = "Maxime Lanvin and Gimenez, \{Pierre Fran{\c c}ois\} and Yufei Han and Fr{\'e}d{\'e}ric Majorczyk and Ludovic M{\'e} and {\'E}ric Totel",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 17th International Conference on Risks and Security of Internet and Systems, CRiSIS 2022 ; Conference date: 07-12-2022 Through 09-12-2022",

year = "2023",

month = jan,

day = "1",

doi = "10.1007/978-3-031-31108-6\_2",

language = "English",

isbn = "9783031311079",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "18--33",

editor = "Slim Kallel and Mohamed Jmaiel and \{Hadj Kacem\}, Ahmed and Mohammad Zulkernine and Fr{\'e}d{\'e}ric Cuppens and Nora Cuppens",

booktitle = "Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers",

}

Lanvin, M, Gimenez, PF, Han, Y, Majorczyk, F, Mé, L & Totel, É 2023, Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes. Dans S Kallel, M Jmaiel, A Hadj Kacem, M Zulkernine, F Cuppens & N Cuppens (eds), Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), VOL. 13857 LNCS, Springer Science and Business Media Deutschland GmbH, p. 18-33, 17th International Conference on Risks and Security of Internet and Systems, CRiSIS 2022, Sousse, Tunisie, 7/12/22. https://doi.org/10.1007/978-3-031-31108-6_2

Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes. / Lanvin, Maxime; Gimenez, Pierre François; Han, Yufei et al.
Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers. Ed. / Slim Kallel; Mohamed Jmaiel; Ahmed Hadj Kacem; Mohammad Zulkernine; Frédéric Cuppens; Nora Cuppens. Springer Science and Business Media Deutschland GmbH, 2023. p. 18-33 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol 13857 LNCS).

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

TY - GEN

T1 - Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes

AU - Lanvin, Maxime

AU - Gimenez, Pierre François

AU - Han, Yufei

AU - Majorczyk, Frédéric

AU - Mé, Ludovic

AU - Totel, Éric

PY - 2023/1/1

Y1 - 2023/1/1

N2 - Among the difficulties encountered in building datasets to evaluate intrusion detection tools, a tricky part is the process of labelling the events into malicious and benign classes. The labelling correctness is paramount for the quality of the evaluation of intrusion detection systems but is often considered as the ground truth by practitioners and is rarely verified. Another difficulty lies in the correct capture of the network packets. If it is not the case, the characteristics of the network flows generated from the capture could be modified and lead to false results. In this paper, we present several flaws we identified in the labelling of the CICIDS2017 dataset and in the traffic capture, such as packet misorder, packet duplication and attack that were performed but not correctly labelled. Finally, we assess the impact of these different corrections on the evaluation of supervised intrusion detection approaches.

AB - Among the difficulties encountered in building datasets to evaluate intrusion detection tools, a tricky part is the process of labelling the events into malicious and benign classes. The labelling correctness is paramount for the quality of the evaluation of intrusion detection systems but is often considered as the ground truth by practitioners and is rarely verified. Another difficulty lies in the correct capture of the network packets. If it is not the case, the characteristics of the network flows generated from the capture could be modified and lead to false results. In this paper, we present several flaws we identified in the labelling of the CICIDS2017 dataset and in the traffic capture, such as packet misorder, packet duplication and attack that were performed but not correctly labelled. Finally, we assess the impact of these different corrections on the evaluation of supervised intrusion detection approaches.

KW - dataset labelling

KW - intrusion detection

KW - machine learning

U2 - 10.1007/978-3-031-31108-6_2

DO - 10.1007/978-3-031-31108-6_2

M3 - Conference contribution

AN - SCOPUS:85161178971

SN - 9783031311079

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 18

EP - 33

BT - Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers

A2 - Kallel, Slim

A2 - Jmaiel, Mohamed

A2 - Hadj Kacem, Ahmed

A2 - Zulkernine, Mohammad

A2 - Cuppens, Frédéric

A2 - Cuppens, Nora

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th International Conference on Risks and Security of Internet and Systems, CRiSIS 2022

Y2 - 7 December 2022 through 9 December 2022

ER -

Lanvin M, Gimenez PF, Han Y, Majorczyk F, Mé L, Totel É. Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes. Dans Kallel S, Jmaiel M, Hadj Kacem A, Zulkernine M, Cuppens F, Cuppens N, rédacteurs en chef, Risks and Security of Internet and Systems - 17th International Conference, CRiSIS 2022, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2023. p. 18-33. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-31108-6_2

Errors in the CICIDS2017 Dataset and the Significant Differences in Detection Performances It Makes

Résumé

Série de publications

Une conférence

Accès au document

Empreinte digitale

Contient cette citation