1. State-of-the-art
Intensive surrogate model exploitation
Intensive Surrogate Model Exploitation in
Self-adaptive Surrogate-assisted CMA-ES
(saACM-ES)
Ilya Loshchilov1
, Marc Schoenauer2
and Michèle Sebag 2
1
LIS, École Polytechnique Fédérale de Lausanne
2
TAO, INRIA − CNRS − Université Paris-Sud
July 9th, 2013
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 1/ 19

2. State-of-the-art
Intensive surrogate model exploitation
Historical overview: PPSN’2010, GECCO’2012
PPSN’2010
CMA-ES assisted with comparison-based surrogate models
(ACM algorithm) 1
.
Good performance but exploitation is independent on surrogate
model quality, surrogate hyper-parameters are fixed.
GECCO’2012
Self-adaptive surrogate-assisted CMA-ES (IPOP-saACM-ES and
BIPOP-saACM-ES) on noiseless2
and noisy testbeds3
.
BIPOP-saACM-ES demonstrates good performance w.r.t. all
algorithms tested during the BBOB-2009, 2010 and 2012.
1[Loshchilov, Schoenauer and Sebag; PPSN 2010] "Comparison-based optimizers need
comparison-based surrogates"
2[Loshchilov, Schoenauer and Sebag; GECCO-BBOB 2012] "Black-box optimization
benchmarking of IPOP-saACM-ES and BIPOP-saACM-ES on the BBOB-2012 noiseless testbed"
3[Loshchilov, Schoenauer and Sebag; GECCO-BBOB 2012] "Black-box optimization
benchmarking of IPOP-saACM-ES on the BBOB-2012 noisy testbed"
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 2/ 19

3. State-of-the-art
Intensive surrogate model exploitation
Content
1 State-of-the-art
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗
ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
2 Intensive surrogate model exploitation
Intensive surrogate model exploitation
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 3/ 19

4. State-of-the-art
Intensive surrogate model exploitation
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
(µ, λ)-Covariance Matrix Adaptation Evolution Strategy
Rank-µ Update 4 5
yi ∼ N (0, C) , C = I
xi = m + σ yi, σ = 1
sampling of λ
solutions
Cµ = 1
µ yi:λyT
i:λ
C ← (1 − 1) × C + 1 × Cµ
calculating C from
best µ out of λ
mnew ← m + 1
µ yi:λ
new distribution
The adaptation increases the probability of successful steps to appear again.
Other components of CMA-ES: step-size adaptation, evolution path.
4[Hansen et al., ECJ 2003] "Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation
(CMA-ES)"
5 the slide adopted by courtesy of Nikolaus Hansen
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 4/ 19

5. State-of-the-art
Intensive surrogate model exploitation
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
Invariance: Guarantee for Generalization
Invariance properties of CMA-ES
Invariance to order-preserving
transformations in function space
true for all comparison-based algorithms
Translation and rotation invariance
thanks to C
−3 −2 −1 0 1 2 3
−3
−2
−1
0
1
2
3
−3 −2 −1 0 1 2 3
−3
−2
−1
0
1
2
3
CMA-ES is almost parameterless (as a consequence of invariances)
Tuning on a small set of functions Hansen & Ostermeier 2001
Default values generalize to whole classes
Exception: population size for multi-modal functions a b
a[Auger & Hansen, CEC 2005] "A restart CMA evolution strategy with increasing population size"
b[Loshchilov et al., PPSN 2012] "Alternative Restart Strategies for CMA-ES"
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 5/ 19

6. State-of-the-art
Intensive surrogate model exploitation
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
Contents
1 State-of-the-art
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗
ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
2 Intensive surrogate model exploitation
Intensive surrogate model exploitation
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 6/ 19

7. State-of-the-art
Intensive surrogate model exploitation
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
s∗
ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
Using Ranking SVM as the surrogate model
Build a global model using Ranking SVM 6
xi ≻ xj iff ˆF(xi) < ˆF(xj)
Comparison-based surrogate models → invariance to
rank-preserving transformations of F(x)
How to choose an appropriate Kernel?
Use covariance matrix C adapted by CMA-ES in Gaussian
kernel7
K(xi, xj) = e−
(xi−xj )T (xi−xj)
2σ2
; KC(xi, xj) = e−
(xi−xj )T C−1(xi−xj )
2σ2
Invariance to rotation of the search space thanks to C
6[Runarsson et al., PPSN 2006] "Ordinal Regression in Evolutionary Computation"
7[Loshchilov et al., PPSN 2010] "Comparison-based optimizers need comparison-based
surrogates"
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 7/ 19

8. State-of-the-art
Intensive surrogate model exploitation
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
s∗
ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
Surrogate-assisted
CMA-ES with online
adaptation of model
hyper-parameters a
a[Loshchilov et al., GECCO 2012]
"Self-Adaptive Surrogate-Assisted Covariance
Matrix Adaptation Evolution Strategy"
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 8/ 19

9. State-of-the-art
Intensive surrogate model exploitation
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
Results on Black-Box Optimization Competition
BIPOP-s∗aACM and IPOP-s∗aACM (with restarts) on 24 noiseless 20 dimensional functions
0 1 2 3 4 5 6 7 8 9
log10 of (ERT / dimension)
0.0
0.5
1.0
Proportionoffunctions
RANDOMSEARCH
SPSA
BAYEDA
DIRECT
DE-PSO
GA
LSfminbnd
LSstep
RCGA
Rosenbrock
MCS
ABC
PSO
POEMS
EDA-PSO
NELDERDOERR
NELDER
oPOEMS
FULLNEWUOA
ALPS
GLOBAL
PSO_Bounds
BFGS
ONEFIFTH
Cauchy-EDA
NBC-CMA
CMA-ESPLUSSEL
NEWUOA
AVGNEWUOA
G3PCX
1komma4mirser
1plus1
CMAEGS
DEuniform
DE-F-AUC
MA-LS-CHAIN
VNS
iAMALGAM
IPOP-CMA-ES
AMALGAM
IPOP-ACTCMA-ES
IPOP-saACM-ES
MOS
BIPOP-CMA-ES
BIPOP-saACM-ES
best 2009
f1-24
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 9/ 19

10. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Contents
1 State-of-the-art
Covariance Matrix Adaptation Evolution Strategy (CMA-ES)
s∗
ACM-ES: Self-Adaptive Surrogate-Assisted CMA-ES
2 Intensive surrogate model exploitation
Intensive surrogate model exploitation
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 10/ 19

11. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Intensive surrogate model exploitation
When optimizing ˆF, λ = kλλdef , µ = µdef
Parameter settings: kλ = 1 for D<10 and kλ = 10, 100, 1000 for
10, 20, 40-D.
An interpretation: trust-region search.
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 11/ 19

12. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Divergence prevention
Surrogate model error estimation can be imprecise
The proposed simple fix:
Exploit surrogate only if the model is "reasonably" precise:
kλ > 1 is used only of ˆn ≥ ˆnkλ
Parameter settings: ˆnkλ
= 4.
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 12/ 19

13. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Surrogate model hyper-parameters adaptation: the
original saACM
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 13/ 19

14. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Surrogate model hyper-parameters adaptation: the
proposed saACM-k
CPU time per function evaluation: 1.22 sec. (kλ = 1), 0.313 sec.
(kλ = 10), 0.311 sec. (kλ = 100), 0.367 sec. (kλ = 1000), 1.52
sec. (kλ = 10000).
Here, for kλ = 100 it is even computationally cheaper!
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 14/ 19

15. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Ill-conditioned problems
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 15/ 19

16. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
All BBOB problems
* smaller budget for surrogate-assisted search: 104
D for
BIPOP-saACM-k versus 106
D for BIPOP-saACM.
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 16/ 19

17. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Hybridization
HCMA = NEWUOA (first 10n evaluations) + STEP (to exploit
separability if any) + BIPOP-saACM-k.
0 1 2 3 4 5 6 7 8
log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Proportionoffunction+targetpairs
fmincon
lmm-CMA-ES
IPOP-texp
MOS
BIPOP-saACM-k
HCMA
best 2009f1-24,20-D
* smaller budget for surrogate-assisted search: 104
D forIlya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 17/ 19

18. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Conclusion
Pros:
Faster convergence, especially on ill-condition problems.
Potentially smaller CPU per function evaluation.
Cons:
Decrease of performance if surrogate error estimation is
imprecise.
Perspective
Kullback-Leibler divergence measure for surrogate model
control.8
8[Loshchilov, Schoenauer and Sebag; CAP 2013] "KL-based Control of the Learning Schedule
for Surrogate Black-Box Optimization"
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 18/ 19

19. State-of-the-art
Intensive surrogate model exploitation
Intensive surrogate model exploitation
Thank you for your attention!
Questions?
Ilya Loshchilov, Marc Schoenauer and Michèle Sebag Intensive Surrogate Model Exploitation in saACM-ES 19/ 19