Estimating the Evolution Direction of Populations to Improve Genetic Algorithms
•
0 j'aime•346 vues
Meta-heuristics have been successfully used to solve a wide variety of problems. However, one issue many techniques have is their risk of being trapped into local optima, or to create a limited variety of solutions (problem known as ``population drift''). During recent and past years, different kinds of techniques have been proposed to deal with population drift, for example hybridizing genetic algorithms with local search techniques or using niche techniques. This paper proposes a technique, based on Singular Value Decomposition (SVD), to enhance Genetic Algorithms (GAs) population diversity. SVD helps to estimate the evolution direction and drive next generations towards orthogonal dimensions. The proposed SVD-based GA has been evaluated on 11 benchmark problems and compared with a simple GA and a GA with a distance-crowding schema. Results indicate that SVD-based GA achieves significantly better solutions and exhibits a quicker convergence than the alternative techniques.
Recommandé
Contenu connexe
Similaire à Estimating the Evolution Direction of Populations to Improve Genetic Algorithms
Similaire à Estimating the Evolution Direction of Populations to Improve Genetic Algorithms (20)
Plus de Annibale Panichella
Plus de Annibale Panichella (20)
Dernier
Dernier (14)
Estimating the Evolution Direction of Populations to Improve Genetic Algorithms
- 1. Estimating the Evolution Direction of Populations To Improve GAs Andrea De Lucia*, Massimiliano Di Penta+ Rocco Oliveto×, Annibale Panichella* The autors are listed in Alphabetic Order * SoftwareEngineering Lab , University of Salerno, Italy + RCOST, University of Sannio, Italy × STAT Department, University of Molise, Italy
- 2. Motivations • Why estimate evolution direction? • Othogonal evolution directions. How? • Singular Value Decomposition (SVD) • SVD base Genetic Algorithm Empirical Evaluation • GA vs. SVD-GA
- 3. What is the evolution direction? P(t) = Popolation at generation t
- 4. What is the evolution direction? P(t+k) = Popolation after k generations P(t) = Popolation at generation t
- 5. What is the evolution direction? P(t+k) = Popolation after k generations P(t) = Popolation at generation t
- 6. Why estimate evolution direction? ( ) ]2;0[,1)1(sin)(min 8 ∈+−= xxxf Motivating Example 1
- 7. Why estimate evolution direction? ( ) ]2;0[,1)1(sin)(min 8 ∈+−= xxxf Motivating Example 1 0.5 1.5 0.5 1.5 1. Optimum is x = 1
- 8. Why estimate evolution direction? ( ) ]2;0[,1)1(sin)(min 8 ∈+−= xxxf Motivating Example 1 0.5 1.5 0.5 1.5 2. f(x) is convex 1. Optimum is x = 1
- 9. Why estimate evolution direction? Motivating Example 1 0.5 1.5 0.5 1.5 It causes poor or slow convergence of GA ( ) ]2;0[,1)1(sin)(min 8 ∈+−= xxxf 2. f(x) is convex 1. Optimum is x = 1
- 10. Why estimate evolution direction? Motivating Example 1 ( ) ]2;0[,1)1(sin)(min 8 ∈+−= xxxf 0.5 1.5 0.5 1.5 Current direction xΔ
- 11. Why estimate evolution direction? Motivating Example 1 ( ) ]2;0[,1)1(sin)(min 8 ∈+−= xxxf 0.5 1.5 0.5 1.5 The estimated current evolution direction can be used to improve the convergence speed of a GA xΔ Probable future direction Current direction
- 12. Why estimate evolution direction? Motivating Example 2 ( ) ]3.2;0[,1)1(sin)(min 8 ∈+−= xxxf xΔ 0.5 1.5 Current direction
- 13. Why estimate evolution direction? Motivating Example 2 ( ) ]3.2;0[,1)1(sin)(min 8 ∈+−= xxxf xΔ 0.5 1.5 We can use the evolution direction for driving the next generations along orthogonal directions Then, we improve the probability of converging to a global optimum Current direction
- 14. How? ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = nmmm n n aaa aaa aaa A ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! Obs. 1 Obs. 2 Obs. m
- 15. How? ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = nmmm n n aaa aaa aaa A ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! Column Vectors ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = nmmm n n aaa aaa aaa A ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! A column vector Aj represents the effect of the factor j for all observations
- 16. How? ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = nmmm n n aaa aaa aaa A ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = nmmm n n aaa aaa aaa A ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! Column Vectors Row Vectors ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ = nmmm n n aaa aaa aaa A ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! A row vector Aj represents the distribution of the observation j within the factors space
- 17. Linear Algebra It measures the relationship between observations within the factors space ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⋅ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =⋅ nmnn m m nmmm n n T aaa aaa aaa aaa aaa aaa AA ,,2,1 2,2,22,1 1,1,21,1 ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! ! "#"" ! ! ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⋅ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =⋅ nmmm n n nmnn m m T aaa aaa aaa aaa aaa aaa AA ,2,1, ,22,21,2 ,12,11,1 ,,2,1 2,2,22,1 1,1,21,1 ! "#"" ! ! ! "#"" ! ! It measures the relationship between factors within the observations space
- 18. Eigenvalues and Eigenvectors It measures the relationship between observations within the factors space ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⋅ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =⋅ nmnn m m nmmm n n T aaa aaa aaa aaa aaa aaa AA ,,2,1 2,2,22,1 1,1,21,1 ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! ! "#"" ! ! ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⋅ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =⋅ nmmm n n nmnn m m T aaa aaa aaa aaa aaa aaa AA ,2,1, ,22,21,2 ,12,11,1 ,,2,1 2,2,22,1 1,1,21,1 ! "#"" ! ! ! "#"" ! ! It measures the relationship between factors within the observations space The eigenvectors of (A AT) form an orthogonal basis of the space Rm Each column vector of A can be expressed as linear combination of U={u1, u2, …, um} { } 0and,,,)( 21 =×=⋅ T jim T uuuuuAArsEigenVecto !
- 19. Eigenvalues and Eigenvectors It measures the relationship between observations within the factors space ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⋅ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =⋅ nmnn m m nmmm n n T aaa aaa aaa aaa aaa aaa AA ,,2,1 2,2,22,1 1,1,21,1 ,2,1, ,22,21,2 ,12,11,1 ! "#"" ! ! ! "#"" ! ! ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ ⋅ ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ =⋅ nmmm n n nmnn m m T aaa aaa aaa aaa aaa aaa AA ,2,1, ,22,21,2 ,12,11,1 ,,2,1 2,2,22,1 1,1,21,1 ! "#"" ! ! ! "#"" ! ! It measures the relationship between factors within the observations space The eigenvectors of (AT A) form an orthogonal basis of the space Rn Each row vector of A can be expressed as linear combination of V={v1, v2, …, vm} { } 0and,,,)( 21 =×=⋅ T jin T vvvvvAArsEigenVecto !
- 20. How? Singual Value Decomposition THEOREM: Let A be a mxn matrix with rank k. There are three matrix U Σ V such that VUA nkkkkmnm ×××× ⋅⋅= Σ Where: • U contains the left singular vectors of A.They are eigenvectors of (AT A). • V contains the right singular vectors of A. They are eigenvectors of (AAT ) . . • Σ is a diagonal matrix containing the non-zero singular values of A (found on the diagonal entries of Σ) are the square roots of the non-zero eigenvalues of both (AT A) (AAT ) .
- 21. THEOREM: Let A be a mxn matrix with rank k. There are three matrix U Σ V such that How? Singual Value Decomposition VUA nkkkkmnm ×××× ⋅⋅= Σ u1 u2 u3 Column Vectors of A
- 22. THEOREM: Let A be a mxn matrix with rank k. There are three matrix U Σ V such that How? Singual Value Decomposition VUA nkkkkmnm ×××× ⋅⋅= Σ u1 u2 u3 Column Vectors of A Row Vectors of A v1 v2 v3
- 23. THEOREM: Let A be a mxn matrix with rank k. There are three matrix U Σ V such that How? Singual Value Decomposition VUA nkkkkmnm ×××× ⋅⋅= Σ u1 u2 u3 Column Vectors of A Row Vectors of A v1 v2 v3 ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ =Σ kσ σ ! "#" ! 0 01 They are in descendig order
- 24. The basic idea is that a population of solutions P provided by GA at generation t can be viewed as a m x n matrix Using SVD for Evolution Direction Individual 1 Individual 2 Individual m
- 25. Using SVD for Evolution Direction From algebraic theorem VUP ⋅⋅= Σ v1 v2
- 26. Suppose we have two different populations Pt and P t+k at two different generations t and t + k Using SVD for Evolution Direction We can compute two SVD decompositions
- 27. Suppose we have two different populations Pt and P t+k at two different generations t and t + k Using SVD for Evolution Direction We can compute two SVD decomposition The currect evolution direction is related to T VUd ⋅≈
- 28. By definition Σ is a scaling operator Then, we construct a new orthogonal population as follows Using SVD for Evolution Direction By definition V is a Rotating operator
- 32. Case Study f1- f6 are multimodal functions f7 – f11 are convex unimodal functions
- 33. Case Study We compared three GAs over 50 independent runs: - simple GA - simple GA with Distance Crowding - SVD-GA Population size = 100 Initial population = uniformly and randomly generated within the solution spaces Fitness scaling function = fitness scaling rank scheme Crossover function = arithmetic crossover with probability Pc = 0.60 Mutation function = uniform mutation function with probability Pm = 0.02 for f1 and f8, while Pm = 0.05 for all the other functions. Elitism = n equals to 2 SVD-Frequency: orthogonal sub-populations are generated by SVD every two generations
- 40. Conclusion
- 41. Conclusion
- 42. Conclusion
- 43. Conclusion