The role of regularization in classification of high-dimensional noisy gaussian mixture

Francesca Mignacco; Florent Krzakala; Yue M. Lu; Pierfrancesco Urbani; Lenka Zdeborova

The role of regularization in classification of high-dimensional noisy gaussian mixture

Francesca Mignacco
, Florent Krzakala
, Yue M. Lu
, Pierfrancesco Urbani
, Lenka Zdeborova

Université Paris-Saclay
PSL research University & IPSL
Harvard University

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

36 Citations (Scopus)

Résumé

We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to _ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

langue originale	Anglais
titre	37th International Conference on Machine Learning, ICML 2020
rédacteurs en chef	Hal Daume, Aarti Singh
Editeur	International Machine Learning Society (IMLS)
Pages	6830-6839
Nombre de pages	10
ISBN (Electronique)	9781713821120
état	Publié - 1 janv. 2020
Modification externe	Oui
Evénement	37th International Conference on Machine Learning, ICML 2020 - Virtual, Online Durée: 13 juil. 2020 → 18 juil. 2020

Série de publications

Nom	37th International Conference on Machine Learning, ICML 2020
Volume	PartF168147-9

Une conférence

Une conférence	37th International Conference on Machine Learning, ICML 2020
La ville	Virtual, Online
période	13/07/20 → 18/07/20

Autres fichiers et liens

Lien vers la publication dans Scopus

Contient cette citation

Mignacco, F., Krzakala, F., Lu, Y. M., Urbani, P., & Zdeborova, L. (2020). The role of regularization in classification of high-dimensional noisy gaussian mixture. Dans H. Daume, & A. Singh (eds.), 37th International Conference on Machine Learning, ICML 2020 (p. 6830-6839). (37th International Conference on Machine Learning, ICML 2020; Vol PartF168147-9). International Machine Learning Society (IMLS).

Mignacco, Francesca ; Krzakala, Florent ; Lu, Yue M. et al. / The role of regularization in classification of high-dimensional noisy gaussian mixture. 37th International Conference on Machine Learning, ICML 2020. Editeur / Hal Daume ; Aarti Singh. International Machine Learning Society (IMLS), 2020. p. 6830-6839 (37th International Conference on Machine Learning, ICML 2020).

@inproceedings{3e90902d200d460994893ea76573714f,

title = "The role of regularization in classification of high-dimensional noisy gaussian mixture",

abstract = "We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to \_ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.",

author = "Francesca Mignacco and Florent Krzakala and Lu, \{Yue M.\} and Pierfrancesco Urbani and Lenka Zdeborova",

note = "Publisher Copyright: {\textcopyright} 2020 37th International Conference on Machine Learning, ICML 2020. All rights reserved.; 37th International Conference on Machine Learning, ICML 2020 ; Conference date: 13-07-2020 Through 18-07-2020",

year = "2020",

month = jan,

day = "1",

language = "English",

series = "37th International Conference on Machine Learning, ICML 2020",

publisher = "International Machine Learning Society (IMLS)",

pages = "6830--6839",

editor = "Hal Daume and Aarti Singh",

booktitle = "37th International Conference on Machine Learning, ICML 2020",

}

Mignacco, F, Krzakala, F, Lu, YM, Urbani, P & Zdeborova, L 2020, The role of regularization in classification of high-dimensional noisy gaussian mixture. Dans H Daume & A Singh (eds), 37th International Conference on Machine Learning, ICML 2020. 37th International Conference on Machine Learning, ICML 2020, VOL. PartF168147-9, International Machine Learning Society (IMLS), p. 6830-6839, 37th International Conference on Machine Learning, ICML 2020, Virtual, Online, 13/07/20.

The role of regularization in classification of high-dimensional noisy gaussian mixture. / Mignacco, Francesca; Krzakala, Florent; Lu, Yue M. et al.
37th International Conference on Machine Learning, ICML 2020. Ed. / Hal Daume; Aarti Singh. International Machine Learning Society (IMLS), 2020. p. 6830-6839 (37th International Conference on Machine Learning, ICML 2020; Vol PartF168147-9).

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

TY - GEN

T1 - The role of regularization in classification of high-dimensional noisy gaussian mixture

AU - Mignacco, Francesca

AU - Krzakala, Florent

AU - Lu, Yue M.

AU - Urbani, Pierfrancesco

AU - Zdeborova, Lenka

PY - 2020/1/1

Y1 - 2020/1/1

N2 - We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to _ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

AB - We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the generalization error of regularized convex classifiers, including ridge, hinge and logistic regression, in the highdimensional limit where the number n of samples and their dimension d go to infinity while their ratio is fixed to _ = n=d. We discuss surprising effects of the regularization that in some cases allows to reach the Bayes-optimal performances. We also illustrate the interpolation peak at low regularization, and analyze the role of the respective sizes of the two clusters.

UR - https://www.scopus.com/pages/publications/85105183212

M3 - Conference contribution

AN - SCOPUS:85105183212

T3 - 37th International Conference on Machine Learning, ICML 2020

SP - 6830

EP - 6839

BT - 37th International Conference on Machine Learning, ICML 2020

A2 - Daume, Hal

A2 - Singh, Aarti

PB - International Machine Learning Society (IMLS)

T2 - 37th International Conference on Machine Learning, ICML 2020

Y2 - 13 July 2020 through 18 July 2020

ER -

Mignacco F, Krzakala F, Lu YM, Urbani P, Zdeborova L. The role of regularization in classification of high-dimensional noisy gaussian mixture. Dans Daume H, Singh A, rédacteurs en chef, 37th International Conference on Machine Learning, ICML 2020. International Machine Learning Society (IMLS). 2020. p. 6830-6839. (37th International Conference on Machine Learning, ICML 2020).

The role of regularization in classification of high-dimensional noisy gaussian mixture

Résumé

Série de publications

Une conférence

Autres fichiers et liens

Empreinte digitale

Contient cette citation