This work provides an in-depth investigation of the effects of population initialization on Differential Evolution (DE) for dealing with large scale optimization problems. Firstly, we conduct a statistical parameter sensitive analysis to study the effects of DE’s control parameters on its performance of solving large scale problems. This study reveals the optimal parame- ter configurations which can lead to the statistically superior performance over the CEC-2013 large-scale test problems. Thus identified optimal parameter configurations interestingly favour much larger population sizes while agreeing with the other parameter settings compared to the most commonly employed parameter configuration. Based on one of the identified optimal configurations and the most commonly used configuration, which only differ in the population size, we investigate the influence of various population initialization techniques on DE’s performance. This study indicates that initialization plays a more crucial role in DE with a smaller population size. However, this observation might be the result of insufficient convergence due to the use of a large population size under the limited computational budget, which deserve more investigations.

1. Effects of
Population Initialization on
Differential Evolution for
Large-Scale Optimization
Borhan Kazimipour
Xiaodong Li
A.K. Qin

2. Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 2Population Initialization on DE for LSO

3. Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 3Population Initialization on DE for LSO

4. What is it all about?
CEC 2014, Beijing, China 4Population Initialization on DE for LSO
Investigating the effect of advanced population
initialization techniques on DE for large scale
Large scale
is not
discovered
Initialization
is important
DE is
powerful

5. What is it all about?
• What is missing in previous works?
– Lack: Large scale optimization has received little attention on this topic.
– Ambiguity: Advanced statistical tools have not been employed to validate the
significance of improvement.
– Contradiction: Some claims appose others
CEC 2014, Beijing, China 5Population Initialization on DE for LSO

6. What is it all about?
CEC 2014, Beijing, China 6Population Initialization on DE for LSO
Contribution
Dimensionality
Advanced
Statistical Tests
Parameter
Configuration

7. What is it all about?
• Basic Facts
– Differential Evolution (DE) is one the most promising optimizers and
winner of many optimization competitions.
– Many researches have claimed that adopting advanced population
initialization techniques improve Evolutionary Algorithms (EAs), including
DE.
– Those claims have not been deeply studied in Large Scale Optimization
(LSO) domain.
• The Goal
– This research aims to study the effect of advanced population
initialization techniques on a widely used DE variant when it comes to
deal with large scale problems considering the effects of DE parameter
setting.
CEC 2014, Beijing, China 7Population Initialization on DE for LSO

8. Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 8Population Initialization on DE for LSO

9. Definitions
•
CEC 2014, Beijing, China 9Population Initialization on DE for LSO

10. Differential Evolution (DE)
CEC 2014, Beijing, China 10Population Initialization on DE for LSO
+ Effective:
the winner of many competitions
+ Popular:
with many publications and
applications
- Population-based algorithm:
Sensitive to initial population
- Has parameters to control
exploration-exploitation balance:
Sensitive to parameter setting
- Suffers from Curse of
Dimensionality

11. Process and Operators
• DE Operators
0. Initialization
1. Mutation
2. Recombination
3. Selection
CEC 2014, Beijing, China 11Population Initialization on DE for LSO

12. 0- Population Initialization
• Common Parameters
– Number of decision variables or problem dimensionality (given)
– Variable ranges (given)
– Population size
• Technique-Specific Parameters
– Chaotic Number Generators: map type and number of iterations
– Uniform Design: number of levels
– Opposition-Based Learning: original population initializer
– …
CEC 2014, Beijing, China 12Population Initialization on DE for LSO

13. Categorize of Population Initialization
CEC 2014, Beijing, China 13Population Initialization on DE for LSO
Population Initialization
Randomness
Stochastic
Pseudo-Random
Number
Generators
Chaotic Number
Generator
Deterministic
Quasi-Random
Sequence
Uniform
Experimental
Design
Composition
ality
Non-Composite Composite
Hybrid
Multi-Step
Generality
Generic
Application
Specific

14. Categorize of Population Initialization
CEC 2014, Beijing, China 14Population Initialization on DE for LSO
Population Initialization
Randomness
Stochastic
Pseudo-Random
Number
Generators
Chaotic Number
Generator
Deterministic
Quasi-Random
Sequence
Uniform
Experimental
Design
Composition
ality
Non-Composite Composite
Hybrid
Multi-Step
Generality
Generic
Application
Specific

15. 1- Mutation
• A DE Mutation Strategy (rand/1)
– r1 and r2 are randomly chosen from population
– F is scaling factor
• Scaling Factor (F)
– Controls exploration-exploitation balance
– Too small F values increase the risk of premature convergence
– Too large F values decrease the convergence speed, degrades efficiency and may
result in early termination
CEC 2014, Beijing, China 15Population Initialization on DE for LSO

16. 2- Recombination
• Binomial Crossover
• Crossover Rate (CR)
– CR determines the number of variables of target vector which must be interchanged
with the corresponding variables of mutant vector
– Small CR values can boost convergence speed when a few decision variables are
interacting with each others (separable functions)
– Large CR values are more effective when lots of decision variables are interacting
(non-separable functions).
In dealing with black-box problems, we have no idea about the separability of the
objective function.
CEC 2014, Beijing, China 16Population Initialization on DE for LSO

17. 3- Selection
• Elite Selection
CEC 2014, Beijing, China 17Population Initialization on DE for LSO

18. 3- Selection
• Elite Selection
CEC 2014, Beijing, China 18Population Initialization on DE for LSO
Yes!
No more
parameters!

19. Differential Evolution (DE)
• Important Parameters
– NP: Population Size
– CR: Crossover Rate
– F: Scale Factor
CEC 2014, Beijing, China 19Population Initialization on DE for LSO

20. Differential Evolution (DE)
• Important Parameters
– NP: Population Size
– CR: Crossover Rate
– F: Scale Factor
CEC 2014, Beijing, China 20Population Initialization on DE for LSO
Population Initialization Technique

21. Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 21Population Initialization on DE for LSO

22. Experiments
• Two- parts Experiment:
A.Parameter Calibration
– Aim: to find the most effective parameter configuration for
DE/rand/1/bin to deal with large scale problems
B.Population Initialization
– Aim: to investigate whether advanced population initialization techniques
can improve common techniques using the most effective parameter
setting.
CEC 2014, Beijing, China 22Population Initialization on DE for LSO

23. Experiments Setup
Parts A & B
• Benchmark
– CEC 2013 LSGO Benchmarks
– 15 functions
– 1000 dimensions
– Categories
1. fully separable functions (f1 - f3),
2. partially separable functions with a separable subcomponent (f4 - f7),
3. partially separable functions with no separable subcomponents (f8 - f11),
4. overlapping functions (f12 - f14),
5. fully non-separable function (f15).
• Statistical Tests
– Iman and Davenport (a.k.a. Friedman rank) test is used for ranking
– Li post-hoc procedure is used as significance test
CEC 2014, Beijing, China 23Population Initialization on DE for LSO

24. Experiments
Part A
A. Parameter Calibration
– PRNG as population initializer
– 14 population sizes [10,20,30,40,50,60,70,80,90,100,150,200,250,300]
– 3 CR values [0.1, 0.5, 0.9]
– 2 F values [0.5, 0.8]
– 84 configurations X 15 functions X 51 runs
CEC 2014, Beijing, China 24Population Initialization on DE for LSO

25. Experiments Results
Part A
• Range of parameter values which perform significantly better than the others based on
Li post-hoc procedure
– NP = [80,90,100,150,200,250]
– CR = 0.9
– F = 0.5
CEC 2014, Beijing, China 25Population Initialization on DE for LSO

26. Experiments Results
Part A
• Among 84 configurations (on 15 functions in 51 runs), the best configuration based on
Iman and Davenport test is found to be:
– NP = 150
– CR = 0.9
– F = 0.5
CEC 2014, Beijing, China 26Population Initialization on DE for LSO

27. Experiments Results
Part A
CC: 3,000,000 FE = 50 NP X 60,000 iterations
CS: 3,000,000 FE = 150 NP X 20,000 iterations
CEC 2014, Beijing, China 27Population Initialization on DE for LSO

28. Experiments Results
Part A
• What we learn from Part A?
– NP, CR an F must be set carefully.
– NP is more relaxed than CR and F values.
– Most effective values for CR and F are the same in low and high dimensional
problems
– Higher dimension problems demand larger populations (even if computational
budget is fixed.)
– Note: The findings are based on the dedicated computational budget; Large increment or
decrement of this limit may affect the results.
CEC 2014, Beijing, China 28Population Initialization on DE for LSO

29. Experiments
Part B
A. Parameter Calibration
– PRNG as population initializer
– 14 population sizes [10,20,30,40,50,60,70,80,90,100,150,200,250,300]
– 3 CR values [0.1, 0.5, 0.9]
– 2 F values [0.5, 0.8]
– 84 configurations X 15 functions X 51 runs
B. Population Initialization Techniques
– Pseudo-Random Number Generators (PRNG)
– Chaotic Number Generator (CNG)
– Sobol Ste (SBL)
– Good Lattice Point (GLP)
– Opposition-Based Learning (OBL)
– Quasi-Opposition-Based Learning (QOBL)
CEC 2014, Beijing, China 29Population Initialization on DE for LSO
Using the best
configuration found
in Part A

30. Experiments Results
Part B
No significant improvement is visible
CEC 2014, Beijing, China 30Population Initialization on DE for LSO

31. Experiments Results
Part B
• What we learn from Part B?
– Advanced population initializers may improve DE/rand/1/bin, but not significantly.
– When proper values for the control parameters are used, population initialization
has only a minor effect.
– Size of population plays more important role than the way it is initialized.
– Note: The findings are based on the dedicated computational budget; Large increment or
decrement of this limit may affect the results.
CEC 2014, Beijing, China 31Population Initialization on DE for LSO

32. Discussions
• Important finding:
– Obtained results challenges the general belief of significant advantages of
advanced techniques in high dimensional spaces.
• How we discuss the contradiction?
1. Significant effect of parameters: None of the previous studies has tried to
compare population initialization techniques on the well-tuned optimizers.
2. Importance of advanced statistical tools: Some statistically minor
improvements of using advanced initializers may wrongly considered as
significant contributions.
Note: This study is well-conducted based on a systematic framework and the findings
are statistically validated. However, the we are well aware of the need of further
investigations to generalise the findings from DE/rand/1/bin to other EAs.
CEC 2014, Beijing, China 32Population Initialization on DE for LSO

33. Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 33Population Initialization on DE for LSO

34. Future Work
• Expansion to other EAs:
– Repeating this study on other popular EAs for further generalization of
the findings.
• Involving other metrics:
– Considering other performance metrics besides final objective values
can help us to investigate whether advanced initializers are able to
significantly improve EAs according to other aspects
• Effect of budget
– computational budget has significant effects on the performance of EAs
when armed with different population initialization techniques.
CEC 2014, Beijing, China 34Population Initialization on DE for LSO

35. Outlines
1. Introduction
2. Background
3. Experiments
4. Future Work
5. Questions
CEC 2014, Beijing, China 35Population Initialization on DE for LSO

36. Thank you
☺☺☺☺
Any question or comment?
36CEC 2014, Beijing, China Population Initialization on DE for LSO