LS-CMA-ES: A second-order algorithm for covariance matrix adaptation

Anne Auger; Marc Schoenauer; Nicolas Vanhaecke

doi:10.1007/978-3-540-30217-9_19

LS-CMA-ES: A second-order algorithm for covariance matrix adaptation

Anne Auger, Marc Schoenauer, Nicolas Vanhaecke

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

Abstract

Evolution Strategies, a class of Evolutionary Algorithms based on Gaussian mutation and deterministic selection, are today considered the best choice as far as parameter optimization is concerned. However, there are multiple ways to tune the covariance matrix of the Gaussian mutation. After reviewing the state of the art in covariance matrix adaptation, a new approach is proposed, in which the update of the covariance matrix is based on a quadratic approximation of the target function, obtained by some Least-Square minimization. A dynamic criterion is designed to detect situations where the approximation is not accurate enough, and original Covariance Matrix Adaptation (CMA) should rather be directly used. The resulting algorithm is experimentally validated on benchmark functions, outperforming CMA-ES on a large class of problems.

Original language	English
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Editors	Xin Yao, John A. Bullinaria, Jonathan Rowe, Peter Tino, Ata Kaban, Edmund Burke, Jose A. Lozano, Jim Smith, Juan J. Merelo-Guervos, Hans-Paul Schwefel
Publisher	Springer Verlag
Pages	182-191
Number of pages	10
ISBN (Print)	3540230920, 9783540230922
DOIs	https://doi.org/10.1007/978-3-540-30217-9_19
Publication status	Published - 1 Jan 2004
Externally published	Yes

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	3242
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Access to Document

10.1007/978-3-540-30217-9_19

Cite this

Auger, A., Schoenauer, M., & Vanhaecke, N. (2004). LS-CMA-ES: A second-order algorithm for covariance matrix adaptation. In X. Yao, J. A. Bullinaria, J. Rowe, P. Tino, A. Kaban, E. Burke, J. A. Lozano, J. Smith, J. J. Merelo-Guervos, & H.-P. Schwefel (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 182-191). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3242). Springer Verlag. https://doi.org/10.1007/978-3-540-30217-9_19

Auger, Anne ; Schoenauer, Marc ; Vanhaecke, Nicolas. / LS-CMA-ES : A second-order algorithm for covariance matrix adaptation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). editor / Xin Yao ; John A. Bullinaria ; Jonathan Rowe ; Peter Tino ; Ata Kaban ; Edmund Burke ; Jose A. Lozano ; Jim Smith ; Juan J. Merelo-Guervos ; Hans-Paul Schwefel. Springer Verlag, 2004. pp. 182-191 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{dab0e30f2c334453b1376f2a97a948b3,

title = "LS-CMA-ES: A second-order algorithm for covariance matrix adaptation",

abstract = "Evolution Strategies, a class of Evolutionary Algorithms based on Gaussian mutation and deterministic selection, are today considered the best choice as far as parameter optimization is concerned. However, there are multiple ways to tune the covariance matrix of the Gaussian mutation. After reviewing the state of the art in covariance matrix adaptation, a new approach is proposed, in which the update of the covariance matrix is based on a quadratic approximation of the target function, obtained by some Least-Square minimization. A dynamic criterion is designed to detect situations where the approximation is not accurate enough, and original Covariance Matrix Adaptation (CMA) should rather be directly used. The resulting algorithm is experimentally validated on benchmark functions, outperforming CMA-ES on a large class of problems.",

author = "Anne Auger and Marc Schoenauer and Nicolas Vanhaecke",

year = "2004",

month = jan,

day = "1",

doi = "10.1007/978-3-540-30217-9\_19",

language = "English",

isbn = "3540230920",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "182--191",

editor = "Xin Yao and Bullinaria, \{John A.\} and Jonathan Rowe and Peter Tino and Ata Kaban and Edmund Burke and Lozano, \{Jose A.\} and Jim Smith and Merelo-Guervos, \{Juan J.\} and Hans-Paul Schwefel",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

Auger, A, Schoenauer, M & Vanhaecke, N 2004, LS-CMA-ES: A second-order algorithm for covariance matrix adaptation. in X Yao, JA Bullinaria, J Rowe, P Tino, A Kaban, E Burke, JA Lozano, J Smith, JJ Merelo-Guervos & H-P Schwefel (eds), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3242, Springer Verlag, pp. 182-191. https://doi.org/10.1007/978-3-540-30217-9_19

LS-CMA-ES: A second-order algorithm for covariance matrix adaptation. / Auger, Anne; Schoenauer, Marc; Vanhaecke, Nicolas.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). ed. / Xin Yao; John A. Bullinaria; Jonathan Rowe; Peter Tino; Ata Kaban; Edmund Burke; Jose A. Lozano; Jim Smith; Juan J. Merelo-Guervos; Hans-Paul Schwefel. Springer Verlag, 2004. p. 182-191 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3242).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

TY - CHAP

T1 - LS-CMA-ES

T2 - A second-order algorithm for covariance matrix adaptation

AU - Auger, Anne

AU - Schoenauer, Marc

AU - Vanhaecke, Nicolas

PY - 2004/1/1

Y1 - 2004/1/1

N2 - Evolution Strategies, a class of Evolutionary Algorithms based on Gaussian mutation and deterministic selection, are today considered the best choice as far as parameter optimization is concerned. However, there are multiple ways to tune the covariance matrix of the Gaussian mutation. After reviewing the state of the art in covariance matrix adaptation, a new approach is proposed, in which the update of the covariance matrix is based on a quadratic approximation of the target function, obtained by some Least-Square minimization. A dynamic criterion is designed to detect situations where the approximation is not accurate enough, and original Covariance Matrix Adaptation (CMA) should rather be directly used. The resulting algorithm is experimentally validated on benchmark functions, outperforming CMA-ES on a large class of problems.

AB - Evolution Strategies, a class of Evolutionary Algorithms based on Gaussian mutation and deterministic selection, are today considered the best choice as far as parameter optimization is concerned. However, there are multiple ways to tune the covariance matrix of the Gaussian mutation. After reviewing the state of the art in covariance matrix adaptation, a new approach is proposed, in which the update of the covariance matrix is based on a quadratic approximation of the target function, obtained by some Least-Square minimization. A dynamic criterion is designed to detect situations where the approximation is not accurate enough, and original Covariance Matrix Adaptation (CMA) should rather be directly used. The resulting algorithm is experimentally validated on benchmark functions, outperforming CMA-ES on a large class of problems.

U2 - 10.1007/978-3-540-30217-9_19

DO - 10.1007/978-3-540-30217-9_19

M3 - Chapter

AN - SCOPUS:27144432208

SN - 3540230920

SN - 9783540230922

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 182

EP - 191

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

A2 - Yao, Xin

A2 - Bullinaria, John A.

A2 - Rowe, Jonathan

A2 - Tino, Peter

A2 - Kaban, Ata

A2 - Burke, Edmund

A2 - Lozano, Jose A.

A2 - Smith, Jim

A2 - Merelo-Guervos, Juan J.

A2 - Schwefel, Hans-Paul

PB - Springer Verlag

ER -

Auger A, Schoenauer M, Vanhaecke N. LS-CMA-ES: A second-order algorithm for covariance matrix adaptation. In Yao X, Bullinaria JA, Rowe J, Tino P, Kaban A, Burke E, Lozano JA, Smith J, Merelo-Guervos JJ, Schwefel HP, editors, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag. 2004. p. 182-191. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-540-30217-9_19

LS-CMA-ES: A second-order algorithm for covariance matrix adaptation

Abstract

Publication series

Access to Document

Fingerprint

Cite this