The role of regularization in classification of high-dimensional noisy gaussian mixture

Francesca Mignacco; Florent Krzakala; Yue M. Lu; Pierfrancesco Urbani; Lenka Zdeborova

The role of regularization in classification of high-dimensional noisy gaussian mixture

Francesca Mignacco
, Florent Krzakala
, Yue M. Lu
, Pierfrancesco Urbani
, Lenka Zdeborova

Université Paris-Saclay
PSL research University & IPSL
Harvard University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

36 Citations (Scopus)

Abstract

We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to _ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

Original language	English
Title of host publication	37th International Conference on Machine Learning, ICML 2020
Editors	Hal Daume, Aarti Singh
Publisher	International Machine Learning Society (IMLS)
Pages	6830-6839
Number of pages	10
ISBN (Electronic)	9781713821120
Publication status	Published - 1 Jan 2020
Externally published	Yes
Event	37th International Conference on Machine Learning, ICML 2020 - Virtual, Online Duration: 13 Jul 2020 → 18 Jul 2020

Publication series

Name	37th International Conference on Machine Learning, ICML 2020
Volume	PartF168147-9

Conference

Conference	37th International Conference on Machine Learning, ICML 2020
City	Virtual, Online
Period	13/07/20 → 18/07/20

Cite this

Mignacco, F., Krzakala, F., Lu, Y. M., Urbani, P., & Zdeborova, L. (2020). The role of regularization in classification of high-dimensional noisy gaussian mixture. In H. Daume, & A. Singh (Eds.), 37th International Conference on Machine Learning, ICML 2020 (pp. 6830-6839). (37th International Conference on Machine Learning, ICML 2020; Vol. PartF168147-9). International Machine Learning Society (IMLS).

Mignacco, Francesca ; Krzakala, Florent ; Lu, Yue M. et al. / The role of regularization in classification of high-dimensional noisy gaussian mixture. 37th International Conference on Machine Learning, ICML 2020. editor / Hal Daume ; Aarti Singh. International Machine Learning Society (IMLS), 2020. pp. 6830-6839 (37th International Conference on Machine Learning, ICML 2020).

@inproceedings{3e90902d200d460994893ea76573714f,

title = "The role of regularization in classification of high-dimensional noisy gaussian mixture",

abstract = "We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to \_ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.",

author = "Francesca Mignacco and Florent Krzakala and Lu, \{Yue M.\} and Pierfrancesco Urbani and Lenka Zdeborova",

note = "Publisher Copyright: {\textcopyright} 2020 37th International Conference on Machine Learning, ICML 2020. All rights reserved.; 37th International Conference on Machine Learning, ICML 2020 ; Conference date: 13-07-2020 Through 18-07-2020",

year = "2020",

month = jan,

day = "1",

language = "English",

series = "37th International Conference on Machine Learning, ICML 2020",

publisher = "International Machine Learning Society (IMLS)",

pages = "6830--6839",

editor = "Hal Daume and Aarti Singh",

booktitle = "37th International Conference on Machine Learning, ICML 2020",

}

Mignacco, F, Krzakala, F, Lu, YM, Urbani, P & Zdeborova, L 2020, The role of regularization in classification of high-dimensional noisy gaussian mixture. in H Daume & A Singh (eds), 37th International Conference on Machine Learning, ICML 2020. 37th International Conference on Machine Learning, ICML 2020, vol. PartF168147-9, International Machine Learning Society (IMLS), pp. 6830-6839, 37th International Conference on Machine Learning, ICML 2020, Virtual, Online, 13/07/20.

The role of regularization in classification of high-dimensional noisy gaussian mixture. / Mignacco, Francesca; Krzakala, Florent; Lu, Yue M. et al.
37th International Conference on Machine Learning, ICML 2020. ed. / Hal Daume; Aarti Singh. International Machine Learning Society (IMLS), 2020. p. 6830-6839 (37th International Conference on Machine Learning, ICML 2020; Vol. PartF168147-9).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The role of regularization in classification of high-dimensional noisy gaussian mixture

AU - Mignacco, Francesca

AU - Krzakala, Florent

AU - Lu, Yue M.

AU - Urbani, Pierfrancesco

AU - Zdeborova, Lenka

PY - 2020/1/1

Y1 - 2020/1/1

N2 - We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to _ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

AB - We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to _ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

UR - https://www.scopus.com/pages/publications/85105183212

M3 - Conference contribution

AN - SCOPUS:85105183212

T3 - 37th International Conference on Machine Learning, ICML 2020

SP - 6830

EP - 6839

BT - 37th International Conference on Machine Learning, ICML 2020

A2 - Daume, Hal

A2 - Singh, Aarti

PB - International Machine Learning Society (IMLS)

T2 - 37th International Conference on Machine Learning, ICML 2020

Y2 - 13 July 2020 through 18 July 2020

ER -

The role of regularization in classification of high-dimensional noisy gaussian mixture

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this