Using Locally Learnt Word Representations for better Textual Anomaly Detection

Alicia Breidenstein; Matthieu Labeau

Using Locally Learnt Word Representations for better Textual Anomaly Detection

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The literature on general purpose textual Anomaly Detection is quite sparse, as most textual anomaly detection methods are implemented as out of domain detection in the context of pre-established classification tasks. Notably, in a field where pre-trained representations and models are of common use, the impact of the pre-training data on a task that lacks supervision has not been studied. In this paper, we use the simple setting of k-classes out anomaly detection and search for the best pairing of representation and classifier. We show that well-chosen embeddings allow a simple anomaly detection baseline such as OC-SVM to achieve similar results and even outperform deep state-of-the-art models.

Original language	English
Title of host publication	Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop
Editors	Shabnam Tafreshi, Arjun Reddy Akula, Joao Sedoc, Aleksandr Drozd, Anna Rogers, Anna Rumshisky
Publisher	Association for Computational Linguistics (ACL)
Pages	82-91
Number of pages	10
ISBN (Electronic)	9798891761025
Publication status	Published - 1 Jan 2024
Event	5th Workshop on Insights from Negative Results in NLP, Insights 2024 - Mexico City, Mexico Duration: 20 Jun 2024 → …

Publication series

Name	Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop

Conference

Conference	5th Workshop on Insights from Negative Results in NLP, Insights 2024
Country/Territory	Mexico
City	Mexico City
Period	20/06/24 → …

Cite this

Breidenstein, A., & Labeau, M. (2024). Using Locally Learnt Word Representations for better Textual Anomaly Detection. In S. Tafreshi, A. R. Akula, J. Sedoc, A. Drozd, A. Rogers, & A. Rumshisky (Eds.), Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop (pp. 82-91). (Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop). Association for Computational Linguistics (ACL).

Breidenstein, Alicia ; Labeau, Matthieu. / Using Locally Learnt Word Representations for better Textual Anomaly Detection. Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop. editor / Shabnam Tafreshi ; Arjun Reddy Akula ; Joao Sedoc ; Aleksandr Drozd ; Anna Rogers ; Anna Rumshisky. Association for Computational Linguistics (ACL), 2024. pp. 82-91 (Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop).

@inproceedings{d6b7d052d5094547bbbefc3e9c3522a2,

title = "Using Locally Learnt Word Representations for better Textual Anomaly Detection",

abstract = "The literature on general purpose textual Anomaly Detection is quite sparse, as most textual anomaly detection methods are implemented as out of domain detection in the context of pre-established classification tasks. Notably, in a field where pre-trained representations and models are of common use, the impact of the pre-training data on a task that lacks supervision has not been studied. In this paper, we use the simple setting of k-classes out anomaly detection and search for the best pairing of representation and classifier. We show that well-chosen embeddings allow a simple anomaly detection baseline such as OC-SVM to achieve similar results and even outperform deep state-of-the-art models.",

author = "Alicia Breidenstein and Matthieu Labeau",

note = "Publisher Copyright: {\textcopyright} 2024 Association for Computational Linguistics.; 5th Workshop on Insights from Negative Results in NLP, Insights 2024 ; Conference date: 20-06-2024",

year = "2024",

month = jan,

day = "1",

language = "English",

series = "Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "82--91",

editor = "Shabnam Tafreshi and Akula, \{Arjun Reddy\} and Joao Sedoc and Aleksandr Drozd and Anna Rogers and Anna Rumshisky",

booktitle = "Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop",

}

Breidenstein, A & Labeau, M 2024, Using Locally Learnt Word Representations for better Textual Anomaly Detection. in S Tafreshi, AR Akula, J Sedoc, A Drozd, A Rogers & A Rumshisky (eds), Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop. Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 82-91, 5th Workshop on Insights from Negative Results in NLP, Insights 2024, Mexico City, Mexico, 20/06/24.

Using Locally Learnt Word Representations for better Textual Anomaly Detection. / Breidenstein, Alicia; Labeau, Matthieu.
Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop. ed. / Shabnam Tafreshi; Arjun Reddy Akula; Joao Sedoc; Aleksandr Drozd; Anna Rogers; Anna Rumshisky. Association for Computational Linguistics (ACL), 2024. p. 82-91 (Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Using Locally Learnt Word Representations for better Textual Anomaly Detection

AU - Breidenstein, Alicia

AU - Labeau, Matthieu

PY - 2024/1/1

Y1 - 2024/1/1

N2 - The literature on general purpose textual Anomaly Detection is quite sparse, as most textual anomaly detection methods are implemented as out of domain detection in the context of pre-established classification tasks. Notably, in a field where pre-trained representations and models are of common use, the impact of the pre-training data on a task that lacks supervision has not been studied. In this paper, we use the simple setting of k-classes out anomaly detection and search for the best pairing of representation and classifier. We show that well-chosen embeddings allow a simple anomaly detection baseline such as OC-SVM to achieve similar results and even outperform deep state-of-the-art models.

AB - The literature on general purpose textual Anomaly Detection is quite sparse, as most textual anomaly detection methods are implemented as out of domain detection in the context of pre-established classification tasks. Notably, in a field where pre-trained representations and models are of common use, the impact of the pre-training data on a task that lacks supervision has not been studied. In this paper, we use the simple setting of k-classes out anomaly detection and search for the best pairing of representation and classifier. We show that well-chosen embeddings allow a simple anomaly detection baseline such as OC-SVM to achieve similar results and even outperform deep state-of-the-art models.

UR - https://www.scopus.com/pages/publications/105000103649

M3 - Conference contribution

AN - SCOPUS:105000103649

T3 - Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop

SP - 82

EP - 91

BT - Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop

A2 - Tafreshi, Shabnam

A2 - Akula, Arjun Reddy

A2 - Sedoc, Joao

A2 - Drozd, Aleksandr

A2 - Rogers, Anna

A2 - Rumshisky, Anna

PB - Association for Computational Linguistics (ACL)

T2 - 5th Workshop on Insights from Negative Results in NLP, Insights 2024

Y2 - 20 June 2024

ER -

Breidenstein A, Labeau M. Using Locally Learnt Word Representations for better Textual Anomaly Detection. In Tafreshi S, Akula AR, Sedoc J, Drozd A, Rogers A, Rumshisky A, editors, Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2024. p. 82-91. (Insights 2024 - 5th Workshop on Insights from Negative Results in NLP, Proceedings of the Workshop).

Using Locally Learnt Word Representations for better Textual Anomaly Detection

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this