Priti Srinivas Sajja is an Associate Professor working with Post Graduate Department of Computer Science, Sardar Patel University, India since 1994. She specializes in Artificial Intelligence especially in knowledge-based systems, soft computing and multiagent systems. She is co-author of Knowledge-Based Systems (2009) and Intelligent Technologies for Web Applications (2012). She has 104 publications in books, book chapters, journals, and in the proceedings of national and international conferences. Three of her publications have won best research paper awards. Visit pritisajja.info for material.

1. Genetic Algorithms
Priti Srinivas Sajja
Associate Professor
Department of Computer Science
Sardar Patel University
Visit priti sajja.info for detail
Created By Priti Srinivas Sajja 1

2. Genetic Algorithms
Introduction
Introduction
Fundamentals
Algorithm
Function
Optimization - 1
Function
Optimization - 2
Ordering Problems
Edge
Recombination
Schema
Genetic
Programming
2
Created By Priti Srinivas Sajja

3. Genetic Algorithms
Introduction
Introduction • The basic purpose of a genetic algorithm (GA) is to mimic
Nature’s evolutionary approach
Fundamentals
• The algorithm is based on the process of natural
selection—Charles Darwin’s “survival of the fittest.”
Algorithm
Function • GAs can be used in problem solving, function optimizing,
Optimization - 1 machine learning, and in innovative systems.
Function
• Historically, GAs were first conceived
Optimization - 2
and used by John Holland (1975)
Ordering Problems at the University of Michigan.
Edge • Genetic algorithms are useful and efficient when:
Recombination • The search space is large, complex, or poorly understood
Schema • Domain knowledge is scarce or expert knowledge is difficult to
encode to narrow the search space
Genetic • No mathematical analysis is available
Programming
Image adapted from 3
Created By Priti Srinivas Sajja
http://today.mun.ca/news.php?news_id=2376

4. Genetic Algorithms
Introduction
Introduction
• According to Hales (2006), instead of just accepting a random
testing strategy to arrive at a solution, one may choose a
Fundamentals
strategy like:
Algorithm
• Generate a set of random solutions
Function • Repeat
Optimization - 1 • Test each solution in the set (rank them)
Function • Remove any bad solutions from the set (bad solutions
Optimization - 2 refer to solutions which are less likely to provide
effective answers according to fitness criteria given by
Ordering Problems experts)
Edge • Duplicate any good solutions or make small changes
Recombination to some of them
• Until an appropriate solution is achieved.
Schema
Genetic
Programming
4
Created By Priti Srinivas Sajja

5. Genetic Algorithms
Introduction • DNA is the building block of bio-cells and strings of DNA
is chromosomes.
Fundamentals
Fundamentals • Chromosomes can be bit strings, real numbers,
permutations of elements, lists of rules, and data
Algorithm structures.
Function • On each DNA string, there is a set of genes responsible
Optimization - 1 for some property of a human (or living thing) to which
Function it belongs.
Optimization - 2 • Such properties, to name a few, are height, skin color,
Ordering Problems
hair color, and eye color.
• A genotype is a collection of such genes representing
Edge possible solutions in a domain space.
Recombination
• This space, referred to as a search space, comprises all
Schema possible solutions to the problem at hand.
Genetic
Programming
5
Created By Priti Srinivas Sajja

6. Genetic Algorithms
Introduction • With every evolutionary step, known as a generation,
the individuals in the current population are decoded
Fundamentals
Fundamentals
and evaluated according to some predefined quality
criterion, referred to as the fitness or fitness function.
Algorithm
• The degree of fitness is related to the strength of the
Function genes in an individual.
Optimization - 1
Function • When new child is born, the parents’ genes are
Optimization - 2 inherited sometimes directly, without any
modification, if they are strong enough. This process is
Ordering Problems
known as duplication (exact resemblance of the child
Edge to its parent) in new generation.
Recombination
• Strong genes are normally represented in the new
Schema generation.
Genetic
Programming
6
Created By Priti Srinivas Sajja

7. Genetic Algorithms
Introduction • An initial population is considered to have a fixed number
of individuals (also known as offspring) containing building
Fundamentals blocks or chromosomes on which genes are set.
Fundamentals
• A fit individual with strong genes can reproduce itself.
Algorithm • Otherwise, in the next generation, the strong genes from
one or more individuals can be combined, resulting in a
Function stronger individual. Over time, the individuals in the
Optimization - 1 population become better adapted to their environment.
Function
Optimization - 2
Ordering Problems (a) (b) (c) (d)
A B Crossover
Individual A
Individual B
Edge
A Mutated
1 1 1 1
Recombination 0 0 1 0
0 1 0 1
1 1 1 1
Schema
Genetic Figure : Operations in the process of natural selection
Programming
7
Created By Priti Srinivas Sajja

8. Genetic Algorithms
Introduction
Fundamentals Start with initial population
by randomly selected Initial population
Individuals
G A Cycle
Algorithm
Modify
Function with Selection Crossover Mutation
operations
Optimization - 1
Function Evaluate
fitness of new Evaluating new individuals through fitness function
Optimization - 2 individuals
Ordering Problems Update population with
better individuals and Modify the population
repeat
Edge
Recombination Figure : Genetic cycle
Schema
Genetic
Programming
8
Created By Priti Srinivas Sajja

9. Genetic Algorithms
Introduction
Fundamentals
G A Cycle
Algorithm The most common method of encoding is with a binary
string. Each individual represents one binary string. Each
Function bit in this string can represent some characteristic of the
Optimization - 1
solution. Then the individual (or chromosome) can be
Function
represented as shown below:
Optimization - 2
Ordering Problems
Edge Individual 1 0 1 0 1 0 1 0 0 1 0 0 1 0 1 1 1
Recombination Individual 2 1 1 0 0 0 1 1 0 0 0 0 1 0 0 1 0
Schema
Genetic
Programming
9
Created By Priti Srinivas Sajja

10. Genetic Algorithms
Introduction Individual 1 0 1 0 1 0 1 0 0 1 0 0 1 0 1 1 1
New
Fundamentals Individual 1
0 1 0 1 1 1 0 1 1 0 0 1 0 0 1 1
Individual 2 1 1 0 0 0 1 1 0 0 0 0 1 0 0 1 0
G A Cycle
Algorithm
New
Individual 2
0 0 0 1 0 1 1 0 0 1 0 1 0 0 1 0
Function
Optimization - 1
Function Individual 1 0 1 0 1 0 0 0 0 1 0 0 1 0 1 1 1
Optimization - 2
Individual 2 1 1 0 0 0 1 Crossover point0
1 0 0 0 1 0 0 1 0
Ordering Problems
Substrings of same length
Edge New
Recombination Individual 1 1 1 0 0 0 0 0 0 1 0 0 1 0 1 1 1
New
Schema Individual 2 0 1 0 1 0 1 1 0 0 0 0 1 0 0 1 0
Genetic
Programming
10
Created By Priti Srinivas Sajja Images from wpclipart.com

11. Genetic Algorithms
Introduction
Fundamentals
G A Cycle
Algorithm E
F
D A
Function
Optimization - 1 C
Function B
Optimization - 2
Ordering Problems Figure: Roulette wheel selection
Edge
Recombination
Schema
Genetic
Programming
11
Created By Priti Srinivas Sajja

12. Genetic Algorithms
Introduction
Consider a simple function f(x)=x*x on the integer
Fundamentals interval [0,30].
1000
Algorithm f(x)
500
Function
Function
Optimization - 1
Optimization - 1
Function 0 x 30
Optimization - 2 • Consider the following four strings in initial population:
01101
Ordering Problems
11000
Edge
01000
Recombination
10011
• Use a simple genetic algorithm composed of three
Schema operators(Reproduction, Crossover and Mutation) to
optimize the aforementioned function in given interval.
Genetic
Programming
12
Created By Priti Srinivas Sajja

13. Genetic Algorithms
Introduction
Sr. Value of x Decimal Fitness f(x) = x*x Roulette wheel
No. of value of x (Value in selection count
Fundamentals indivi decimal)
dual
1 01101 13 169 1
Algorithm
2 11000 24 576 2
Function
Function 3 01000 08 064 0
Optimization - 1
Optimization - 1 4 10011 19 361 1
Function
Optimization - 2 Table : First Generation of f(x)=x*x
Ordering Problems
1
Edge Figure : Roulette wheel 2
Recombination 3
presentation for the first
4
generation shown in the
Schema f(x)=x*x example
Genetic
Programming
13
Created By Priti Srinivas Sajja

14. Genetic Algorithms
Introduction Sr. No. of Value of x New individual Operation Fitness f(x) =
selected after mutation site x*x
individual and cross-over (Value in
Fundamentals decimal)
1 01101 01100 5 12*12=144
2 11000 11001 5 25*25=625
Algorithm
3 11000 11011 3 27*27= 729
Function
Function 4 10011 10000 3 16*16= 256
Optimization - 1
Optimization - 1 Table : Mutation and crossover after reproduction for f(x)=x*x
Function
Optimization - 2
Sr. No. of Value of x Decimal value of y Fitness f(x) = x*x
Individual (Value in Decimal)
Ordering Problems
1 01100 12 144
Edge 2 11001 25 625
Recombination 3 11011 27 729
4 10000 16 256
Schema
Table: Second generation of the function f(x)=x*x
Genetic
Programming
14
Created By Priti Srinivas Sajja

15. Genetic Algorithms
Introduction
Sr. No. of Value of x Value of y Bit string Fitness f(X) = Roulette
individual x+y wheel
Fundamentals (Value in selection
decimal) count
Algorithm 1 01011 01010 0101101010 10+11= 21 1
2 11000 01100 1100001100 24+12=36 2
Function 3 01001 00101 0100100101 9+5=14 0
Optimization - 1 4 01011 00111 0101100111 11+7=18 1
Function
Function
Optimization - 2
Optimization - 2 Table: First generation of f(x,y)= x+y
1
Ordering Problems 2
• Maximize f(x,y) = x + y in the interval of [0, 3
Edge 30] with binary encoding strategy of size 10, 4
Recombination with the first 5 bits representing a value of the
first variable x, and the remaining bits
Schema representing a value of the second variable y. Figure : Roulette wheel
presentation for the generation
shown in the f(x,y)= x+y example
Genetic
Programming
15
Created By Priti Srinivas Sajja

16. Genetic Algorithms
Introduction
Fundamentals
Algorithm
Function
Optimization - 1
Function
Function
Optimization - 3
Optimization - 2
Ordering Problems
Edge
Recombination
Schema
Genetic
Programming
16
Created By Priti Srinivas Sajja

17. Genetic Algorithms
Introduction
Initial Population for TSP
Fundamentals covering any five city from given six cities
Algorithm (5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)
Function
Optimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)
Function
Optimization - 2
(3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)
OrderingProblems
Ordering Problems
Edge (4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)
Recombination
Schema
Example Courtesy: Jim Cohoon and Kimberly Hanks
Genetic
Programming
17
Created By Priti Srinivas Sajja

18. Genetic Algorithms
Introduction
Fundamentals Generation of Individuals
Algorithm
(5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)
Function
Optimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)
Function
Optimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)
OrderingProblems
Ordering Problems
(4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)
Edge
Recombination
Schema
(3,4,5,6,2)
Genetic
Programming
18
Created By Priti Srinivas Sajja

19. Genetic Algorithms
Introduction
Fundamentals Generation of more Individuals
Algorithm
(5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)
Function
Optimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)
Function
Optimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)
OrderingProblems
Ordering Problems
(4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)
Edge
Recombination
Schema
(3,4,5,6,2) (5,4,2,6,3)
Genetic
Programming
19
Created By Priti Srinivas Sajja

20. Genetic Algorithms
Introduction
Fundamentals Mutation
Algorithm
(5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)
Function
Optimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)
Function
Optimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)
OrderingProblems
Ordering Problems
(4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)
Edge
Recombination
Schema
(3,4,5,6,2) (5,4,2,6,3)
Genetic
Programming
20
Created By Priti Srinivas Sajja

21. Genetic Algorithms
Introduction
Elimination of the worst tours
Fundamentals
(5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)
Algorithm
Function (2,3,4,6,5) (2,3,6,4,5) (3,4,5,2,6)
Optimization - 1
Function
Optimization - 2
(3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)
OrderingProblems
Ordering Problems
(4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)
Edge
Recombination
Schema (3,4,5,6,2) (5,4,2,6,3)
Genetic
Programming
21
Created By Priti Srinivas Sajja

22. Genetic Algorithms
Introduction
• Finding an optimal ordering for a sequence of N items.
• For example, in a traveling salesman problem, consider there
Fundamentals are four cities 1,2,3 and 4.
• Each city is then, labeled by a unique bit string.
Algorithm
• A common fitness function for the problem is length of the
Function
Optimization - 1 candidate tour.
Function • A natural way is the permutation, so that 3214 is one
Optimization - 2 candidate’s tour and 4123 is another.
OrderingProblems
Ordering Problems • This representation is problematic for genetic algorithm
because Mutation and crossover do not produce necessarily
Edge
Recombination leagal tours.
Schema • For example, cross over between position 2 and 3 produces (in
the example 3214 and 4123) produces the individuals 3223 and
Genetic 4114, both of which are illegal tours.
Programming
22
Created By Priti Srinivas Sajja

23. Genetic Algorithms
Introduction
Fundamentals
• Adopting a different representations
Algorithm
Function • Designing a special crossover operators
Optimization - 1
Function
Optimization - 2 • Penalizing the illegal solution with the proper
OrderingProblems
Ordering Problems fitness function.
Edge
Recombination
Schema
Genetic
Programming
23
Created By Priti Srinivas Sajja

24. Genetic Algorithms
Introduction
362145
Fundamentals 364125
521364
Algorithm Original Individuals
Function
Optimization - 1
Key Adjacent Keys
Function
Optimization - 2 1 2, 2, 3, 4
2 1, 1, 5, 6
Ordering Problems 3 1, 6, 6
Edge
Edge 4 1, 5, 6
Recombination
Recombination 5 2, 4
6 2, 3, 3, 4
Schema
Genetic
Programming
24
Created By Priti Srinivas Sajja

25. Genetic Algorithms
Introduction
• For the problems, which are complex, noisy or dynamic, it is
virtually impossible t to predict the performance of a genetic
algorithm.
Fundamentals
• Holland[1975], introduced the notion of schema to explain
Algorithm
how genetic algorithms search for region of high fitness.
Function
• A schema is a template, defined over alphabet{0,1,*} which
Optimization - 1
describes a pattern of bit strings in the search space {0,1}l,
Function where l is length of a bit string.
Optimization - 2
• For example, the two strings A and B have several bits in
Ordering Problems
common. We can use schemas to describe the patterns these
Edge two strings A=100111 and B=010011 share:
Recombination **0*11
Schema
Schema ****11
**0***
Genetic **0**1
Programming
25
Created By Priti Srinivas Sajja

26. Genetic Algorithms
Introduction
Fundamentals • A bit string x that matches a schema S’s pattern is said to
be an instance of S. For example, A and B both are
instances of the schemas.
Algorithm
• Order of schema is the number of defined bits in it. For
Function example, order of the first schema **0*11 is 3.
Optimization - 1 • Defining length of a schema is the distance between the
Function left most and right most defined bits in the schema.
Optimization - 2 • For example, the defining length of **0**1 is 3. Other
examples are
Ordering Problems
• for S1 = [01#1#], (S1) = 4 – 1 = 3
Edge • for S2 = [##1#1010], (S2) = 8 – 3 = 5
Recombination
Schema
Schema
Genetic
Programming
26
Created By Priti Srinivas Sajja

27. Genetic Algorithms
• Schema defines hyper plane in the
Introduction search space {0,1}l .
• This figure defines four hyper planes
Fundamentals corresponding to the four different
schemas.
Algorithm
• Any point in the space can be
Function simultaneously an instance of two
Optimization - 1 schemas.
Function • The fitness of any bit string in the
Optimization - 2 population gives some information about
the average fitness of the 2l different
Ordering Problems schemas (class) of which it is an instance.
*0***
Edge • So an explicit evaluation of population0****
of
Recombination M individual strings is also an implicit 1**** *1***
(implied) evaluation of a much larger is an instance of
Schema
Schema number of schemas. 1**** and *0***.
Genetic • This is referred to as implicit parallelism.
Programming
27
Created By Priti Srinivas Sajja

28. Genetic Algorithms
• Programs written in subset of LISP language and
Introduction
• Programs which can be represented in a tree are the candidate of
Fundamentals evolution under genetic fashion/natural selection.
• Population of a random trees are generated and evaluated as in the
Algorithm standard genetic algorithm.
• They have special crossover function.
Function
Optimization - 1 Expression: x2+3xy+y2
Function LISP: (+(*xx)(*xy)(*yy))
Optimization - 2
Ordering Problems
+
Edge
Recombination
* * *
Schema
X x 3 x y y y
Genetic
Genetic
Programming
Programming
28
Created By Priti Srinivas Sajja

29. Genetic Algorithms
Introduction
• Human written programs are design elegant and general
Fundamentals but the genetic generated programs are often needlessly
complicated and not revealing the underplaying
Algorithm algorithm.
Function • For cos2x we write (-1(*2(*sin x)(sin x))))
Optimization - 1 • Genetic program discovers
Function • (sin(-(-2(*x2))(sin(sin(sin(sin(sin(sin(*(sin(sin 1)))))))))))
Optimization - 2
• The evolved programs are inelegant, redundant, inefficient
Ordering Problems and difficult for human read.
• They give adhoc type of solutions that evolve in nature
Edge
Recombination
through gene duplication, mutation and modifying
structures.
Schema • If successful, it leads to the design revolutions.
Genetic
Genetic
Programming
Programming
29
Created By Priti Srinivas Sajja

30. Genetic Algorithms
Introduction
• Optimization, combinatorial, and scheduling problems
Fundamentals • Automatic programming
• Game playing
Algorithm • Self-managing and sorting networks
• Machine and robot learning
Function • Evolving artificial neural network (ANN), rules-based systems and
Optimization - 1 other hybrid architectures of soft computing
Function • Designing and controlling robots
Optimization - 2
• Modeling natural systems (to model processes of innovation)
Ordering Problems • Emergence of economic markets
• Ecological phenomena
Edge
• Study of evolutionary aspects of social systems, such as the
Recombination
evolution of cooperation, evolution of communication, and trail-
Schema following behavior in ants
• Artificial life models (systems that model interactions between
Genetic species evolution and individual learning)
Programming
30
Created By Priti Srinivas Sajja

31. Genetic Algorithms
Introduction
Fundamentals
Algorithm
Function
Optimization - 1
Function
Optimization - 2 References
• “Knowledge-based systems”, Akerkar RA and Priti
Ordering Problems
Srinivas Sajja, Jones & Bartlett Publishers, Sudbury, MA, USA
(2009)
Edge
Recombination • Allen B. Tucker,Jr. The Computer Science and Engineering
handbook, CRC Press, 1997, (pp. 557 to pp.569)
Schema
• Articulte.com
Genetic • Pit.edu
Programming
31
Created By Priti Srinivas Sajja