A comparative study of classification based personal E-mail filtering

Yanlei Diao; Hongjun Lu; Dekai Wu

doi:10.1007/3-540-45571-x_48

A comparative study of classification based personal E-mail filtering

Yanlei Diao
, Hongjun Lu
, Dekai Wu

The Hong Kong University of Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper addresses personal E-mail filtering by casting it in the framework of text classification. Modeled as semi-structured documents, Email messages consist of a set of fields with predefined semantics and a number of variable length free-text fields. While most work on classification either concentrates on structured data or free text, the work in this paper deals with both of them. To perform classification, a naive Bayesian classifier was designed and implemented, and a decision tree based classifier was implemented. The design considerations and implementation issues are discussed. Using a relatively large amount of real personal E-mail data, a comprehensive comparative study was conducted using the two classifiers. The importance of different features is reported. Results of other issues related to building an effective personal E-mail classifier are presented and discussed. It is shown that both classifiers can perform filtering with reasonable accuracy. While the decision tree based classifier outperforms the Bayesian classifier when features and training size are selected optimally for both, a carefully designed naive Bayesian classifier is more robust.

Original language	English
Title of host publication	Knowledge Discovery and Data Mining
Subtitle of host publication	Current Issues and New Applications - 4th Pacific-Asia Conference, PAKDD 2000, Proceedings
Editors	Takao Terano, Huan Liu, Arbee L.P. Chen
Publisher	Springer Verlag
Pages	408-419
Number of pages	12
ISBN (Print)	3540673822, 9783540673828
DOIs	https://doi.org/10.1007/3-540-45571-x_48
Publication status	Published - 1 Jan 2000
Externally published	Yes
Event	4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2000 - Kyoto, Japan Duration: 18 Apr 2000 → 20 Apr 2000

Publication series

Name	Lecture Notes in Computer Science
Volume	1805
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2000
Country/Territory	Japan
City	Kyoto
Period	18/04/00 → 20/04/00

Access to Document

10.1007/3-540-45571-x_48

Cite this

Diao, Y., Lu, H., & Wu, D. (2000). A comparative study of classification based personal E-mail filtering. In T. Terano, H. Liu, & A. L. P. Chen (Eds.), Knowledge Discovery and Data Mining: Current Issues and New Applications - 4th Pacific-Asia Conference, PAKDD 2000, Proceedings (pp. 408-419). (Lecture Notes in Computer Science; Vol. 1805). Springer Verlag. https://doi.org/10.1007/3-540-45571-x_48

@inproceedings{02b73535fc6f45f3a5f7041d08570e8c,

title = "A comparative study of classification based personal E-mail filtering",

abstract = "This paper addresses personal E-mail filtering by casting it in the framework of text classification. Modeled as semi-structured documents, Email messages consist of a set of fields with predefined semantics and a number of variable length free-text fields. While most work on classification either concentrates on structured data or free text, the work in this paper deals with both of them. To perform classification, a naive Bayesian classifier was designed and implemented, and a decision tree based classifier was implemented. The design considerations and implementation issues are discussed. Using a relatively large amount of real personal E-mail data, a comprehensive comparative study was conducted using the two classifiers. The importance of different features is reported. Results of other issues related to building an effective personal E-mail classifier are presented and discussed. It is shown that both classifiers can perform filtering with reasonable accuracy. While the decision tree based classifier outperforms the Bayesian classifier when features and training size are selected optimally for both, a carefully designed naive Bayesian classifier is more robust.",

author = "Yanlei Diao and Hongjun Lu and Dekai Wu",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2000.; 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2000 ; Conference date: 18-04-2000 Through 20-04-2000",

year = "2000",

month = jan,

day = "1",

doi = "10.1007/3-540-45571-x\_48",

language = "English",

isbn = "3540673822",

series = "Lecture Notes in Computer Science",

publisher = "Springer Verlag",

pages = "408--419",

editor = "Takao Terano and Huan Liu and Chen, \{Arbee L.P.\}",

booktitle = "Knowledge Discovery and Data Mining",

}

Diao, Y, Lu, H & Wu, D 2000, A comparative study of classification based personal E-mail filtering. in T Terano, H Liu & ALP Chen (eds), Knowledge Discovery and Data Mining: Current Issues and New Applications - 4th Pacific-Asia Conference, PAKDD 2000, Proceedings. Lecture Notes in Computer Science, vol. 1805, Springer Verlag, pp. 408-419, 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2000, Kyoto, Japan, 18/04/00. https://doi.org/10.1007/3-540-45571-x_48

A comparative study of classification based personal E-mail filtering. / Diao, Yanlei; Lu, Hongjun; Wu, Dekai.
Knowledge Discovery and Data Mining: Current Issues and New Applications - 4th Pacific-Asia Conference, PAKDD 2000, Proceedings. ed. / Takao Terano; Huan Liu; Arbee L.P. Chen. Springer Verlag, 2000. p. 408-419 (Lecture Notes in Computer Science; Vol. 1805).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A comparative study of classification based personal E-mail filtering

AU - Diao, Yanlei

AU - Lu, Hongjun

AU - Wu, Dekai

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2000.

PY - 2000/1/1

Y1 - 2000/1/1

N2 - This paper addresses personal E-mail filtering by casting it in the framework of text classification. Modeled as semi-structured documents, Email messages consist of a set of fields with predefined semantics and a number of variable length free-text fields. While most work on classification either concentrates on structured data or free text, the work in this paper deals with both of them. To perform classification, a naive Bayesian classifier was designed and implemented, and a decision tree based classifier was implemented. The design considerations and implementation issues are discussed. Using a relatively large amount of real personal E-mail data, a comprehensive comparative study was conducted using the two classifiers. The importance of different features is reported. Results of other issues related to building an effective personal E-mail classifier are presented and discussed. It is shown that both classifiers can perform filtering with reasonable accuracy. While the decision tree based classifier outperforms the Bayesian classifier when features and training size are selected optimally for both, a carefully designed naive Bayesian classifier is more robust.

AB - This paper addresses personal E-mail filtering by casting it in the framework of text classification. Modeled as semi-structured documents, Email messages consist of a set of fields with predefined semantics and a number of variable length free-text fields. While most work on classification either concentrates on structured data or free text, the work in this paper deals with both of them. To perform classification, a naive Bayesian classifier was designed and implemented, and a decision tree based classifier was implemented. The design considerations and implementation issues are discussed. Using a relatively large amount of real personal E-mail data, a comprehensive comparative study was conducted using the two classifiers. The importance of different features is reported. Results of other issues related to building an effective personal E-mail classifier are presented and discussed. It is shown that both classifiers can perform filtering with reasonable accuracy. While the decision tree based classifier outperforms the Bayesian classifier when features and training size are selected optimally for both, a carefully designed naive Bayesian classifier is more robust.

U2 - 10.1007/3-540-45571-x_48

DO - 10.1007/3-540-45571-x_48

M3 - Conference contribution

AN - SCOPUS:84942797676

SN - 3540673822

SN - 9783540673828

T3 - Lecture Notes in Computer Science

SP - 408

EP - 419

BT - Knowledge Discovery and Data Mining

A2 - Terano, Takao

A2 - Liu, Huan

A2 - Chen, Arbee L.P.

PB - Springer Verlag

T2 - 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2000

Y2 - 18 April 2000 through 20 April 2000

ER -

A comparative study of classification based personal E-mail filtering

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this