Distributed Adaptive Model Rules for mining big data streams

Anh Thu Vu; Gianmarco De Francisci Morales; Joao Gama; Albert Bifet

doi:10.1109/BigData.2014.7004251

Distributed Adaptive Model Rules for mining big data streams

Anh Thu Vu
, Gianmarco De Francisci Morales
, Joao Gama
, Albert Bifet

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Decision rules are among the most expressive data mining models. We propose the first distributed streaming algorithm to learn decision rules for regression tasks. The algorithm is available in samoa (Scalable Advanced Massive Online Analysis), an open-source platform for mining big data streams. It uses a hybrid of vertical and horizontal parallelism to distribute Adaptive Model Rules (AMRules) on a cluster. The decision rules built by AMRules are comprehensible models, where the antecedent of a rule is a conjunction of conditions on the attribute values, and the consequent is a linear combination of the attributes. Our evaluation shows that this implementation is scalable in relation to CPU and memory consumption. On a small commodity Samza cluster of 9 nodes, it can handle a rate of more than 30000 instances per second, and achieve a speedup of up to 4.7x over the sequential version.

Original language	English
Title of host publication	Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014
Editors	Jimmy Lin, Jian Pei, Xiaohua Tony Hu, Wo Chang, Raghunath Nambiar, Charu Aggarwal, Nick Cercone, Vasant Honavar, Jun Huan, Bamshad Mobasher, Saumyadipta Pyne
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	345-353
Number of pages	9
ISBN (Electronic)	9781479956654
DOIs	https://doi.org/10.1109/BigData.2014.7004251
Publication status	Published - 1 Jan 2014
Externally published	Yes
Event	2nd IEEE International Conference on Big Data, Big Data 2014 - Washington, United States Duration: 27 Oct 2014 → 30 Oct 2014

Publication series

Name	Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

Conference

Conference	2nd IEEE International Conference on Big Data, Big Data 2014
Country/Territory	United States
City	Washington
Period	27/10/14 → 30/10/14

Access to Document

10.1109/BigData.2014.7004251

Cite this

Vu, A. T., De Francisci Morales, G., Gama, J., & Bifet, A. (2014). Distributed Adaptive Model Rules for mining big data streams. In J. Lin, J. Pei, X. T. Hu, W. Chang, R. Nambiar, C. Aggarwal, N. Cercone, V. Honavar, J. Huan, B. Mobasher, & S. Pyne (Eds.), Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014 (pp. 345-353). Article 7004251 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2014.7004251

Vu, Anh Thu ; De Francisci Morales, Gianmarco ; Gama, Joao et al. / Distributed Adaptive Model Rules for mining big data streams. Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014. editor / Jimmy Lin ; Jian Pei ; Xiaohua Tony Hu ; Wo Chang ; Raghunath Nambiar ; Charu Aggarwal ; Nick Cercone ; Vasant Honavar ; Jun Huan ; Bamshad Mobasher ; Saumyadipta Pyne. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 345-353 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014).

@inproceedings{ccdf0698e277482f9d4bf246169f42d9,

title = "Distributed Adaptive Model Rules for mining big data streams",

abstract = "Decision rules are among the most expressive data mining models. We propose the first distributed streaming algorithm to learn decision rules for regression tasks. The algorithm is available in samoa (Scalable Advanced Massive Online Analysis), an open-source platform for mining big data streams. It uses a hybrid of vertical and horizontal parallelism to distribute Adaptive Model Rules (AMRules) on a cluster. The decision rules built by AMRules are comprehensible models, where the antecedent of a rule is a conjunction of conditions on the attribute values, and the consequent is a linear combination of the attributes. Our evaluation shows that this implementation is scalable in relation to CPU and memory consumption. On a small commodity Samza cluster of 9 nodes, it can handle a rate of more than 30000 instances per second, and achieve a speedup of up to 4.7x over the sequential version.",

author = "Vu, \{Anh Thu\} and \{De Francisci Morales\}, Gianmarco and Joao Gama and Albert Bifet",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 2nd IEEE International Conference on Big Data, Big Data 2014 ; Conference date: 27-10-2014 Through 30-10-2014",

year = "2014",

month = jan,

day = "1",

doi = "10.1109/BigData.2014.7004251",

language = "English",

series = "Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "345--353",

editor = "Jimmy Lin and Jian Pei and Hu, \{Xiaohua Tony\} and Wo Chang and Raghunath Nambiar and Charu Aggarwal and Nick Cercone and Vasant Honavar and Jun Huan and Bamshad Mobasher and Saumyadipta Pyne",

booktitle = "Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014",

}

Vu, AT, De Francisci Morales, G, Gama, J & Bifet, A 2014, Distributed Adaptive Model Rules for mining big data streams. in J Lin, J Pei, XT Hu, W Chang, R Nambiar, C Aggarwal, N Cercone, V Honavar, J Huan, B Mobasher & S Pyne (eds), Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014., 7004251, Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014, Institute of Electrical and Electronics Engineers Inc., pp. 345-353, 2nd IEEE International Conference on Big Data, Big Data 2014, Washington, United States, 27/10/14. https://doi.org/10.1109/BigData.2014.7004251

Distributed Adaptive Model Rules for mining big data streams. / Vu, Anh Thu; De Francisci Morales, Gianmarco; Gama, Joao et al.
Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014. ed. / Jimmy Lin; Jian Pei; Xiaohua Tony Hu; Wo Chang; Raghunath Nambiar; Charu Aggarwal; Nick Cercone; Vasant Honavar; Jun Huan; Bamshad Mobasher; Saumyadipta Pyne. Institute of Electrical and Electronics Engineers Inc., 2014. p. 345-353 7004251 (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Distributed Adaptive Model Rules for mining big data streams

AU - Vu, Anh Thu

AU - De Francisci Morales, Gianmarco

AU - Gama, Joao

AU - Bifet, Albert

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Decision rules are among the most expressive data mining models. We propose the first distributed streaming algorithm to learn decision rules for regression tasks. The algorithm is available in samoa (Scalable Advanced Massive Online Analysis), an open-source platform for mining big data streams. It uses a hybrid of vertical and horizontal parallelism to distribute Adaptive Model Rules (AMRules) on a cluster. The decision rules built by AMRules are comprehensible models, where the antecedent of a rule is a conjunction of conditions on the attribute values, and the consequent is a linear combination of the attributes. Our evaluation shows that this implementation is scalable in relation to CPU and memory consumption. On a small commodity Samza cluster of 9 nodes, it can handle a rate of more than 30000 instances per second, and achieve a speedup of up to 4.7x over the sequential version.

AB - Decision rules are among the most expressive data mining models. We propose the first distributed streaming algorithm to learn decision rules for regression tasks. The algorithm is available in samoa (Scalable Advanced Massive Online Analysis), an open-source platform for mining big data streams. It uses a hybrid of vertical and horizontal parallelism to distribute Adaptive Model Rules (AMRules) on a cluster. The decision rules built by AMRules are comprehensible models, where the antecedent of a rule is a conjunction of conditions on the attribute values, and the consequent is a linear combination of the attributes. Our evaluation shows that this implementation is scalable in relation to CPU and memory consumption. On a small commodity Samza cluster of 9 nodes, it can handle a rate of more than 30000 instances per second, and achieve a speedup of up to 4.7x over the sequential version.

U2 - 10.1109/BigData.2014.7004251

DO - 10.1109/BigData.2014.7004251

M3 - Conference contribution

AN - SCOPUS:84921726705

T3 - Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

SP - 345

EP - 353

BT - Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014

A2 - Lin, Jimmy

A2 - Pei, Jian

A2 - Hu, Xiaohua Tony

A2 - Chang, Wo

A2 - Nambiar, Raghunath

A2 - Aggarwal, Charu

A2 - Cercone, Nick

A2 - Honavar, Vasant

A2 - Huan, Jun

A2 - Mobasher, Bamshad

A2 - Pyne, Saumyadipta

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2nd IEEE International Conference on Big Data, Big Data 2014

Y2 - 27 October 2014 through 30 October 2014

ER -

Vu AT, De Francisci Morales G, Gama J, Bifet A. Distributed Adaptive Model Rules for mining big data streams. In Lin J, Pei J, Hu XT, Chang W, Nambiar R, Aggarwal C, Cercone N, Honavar V, Huan J, Mobasher B, Pyne S, editors, Proceedings - 2014 IEEE International Conference on Big Data, Big Data 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 345-353. 7004251. (Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014). doi: 10.1109/BigData.2014.7004251

Distributed Adaptive Model Rules for mining big data streams

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this