1. Alan Nochenson
IST 511
Oct 15, 2012

2.  Branch of AI
 Geared towards taking input and yielding
patterns or predictions
 Use data to capture uncertainty in underlying
probability distributions

3.  1959 – Arthur Samuel: "Field of study that
gives computers the ability to learn without
being explicitly programmed".
 Machine learning != data mining

4.  Decision trees
 Association rule learning
 Artificial neural networks
 Genetic programming
 Inductive logic programming
 Support vector machines
 Clustering
 Bayesian networks
 Reinforcement learning
 Representation learning
 Sparse dictionary learning

5.  Machine learning technique
 Used to optimize a program’s ability to
perform some kind of task
 Inspired by biological evolution (i.e.
Darwinian evolution)

6.  Explicit + implicit parallelism
 Gracefulness
 Knowledge about evolutionary processes
(brain processes are harder)

7.  Natural evolution is too slow
 Silly argument – generations are much quicker in
computers than biology and so many generations
have given great process
 Waddington (1967) says GP algorithms are
too simple to get to the same type of
complexity as real evolution
 Answer: GP works for efficiency in a wide range of
problems

8.  E.g.

9.  Initialization
 Selection
 Crossover (breeding)
 Mutation
 Termination

10.  Let us try to find an algorithm, where the
operators are + and – to get to a target
number, which we call x. We will restrict
ourselves to 5 “spots” and single digit
numbers
 E.g. Let x=10
 Solutions would be:
▪ 0+2+8, 1+9-0 , 8+3-1, …

11.  Create a population of individuals
 1+3-4
 ++23-
 0+4+1
 983-2
 …

12.  How far off of our target are we?
 1+3-4 (fitness=10-0=0)
 ++23- (fitness=-100 since invalid)
 0+4+1 (fitness = 10-5=5)
 983-2 = 3-2 (fitness = 1)
 …

13.  Select the most fit individuals to breed (or
some other algorithm)
 1+3-4 (fitness=0)
 ++23- (fitness=-100 since invalid)
 0+4+1 (fitness = 5)
 983-2 = 3-2 = 0+3-2 (fitness = 1)
 …

14.  Crossover (take part from each)
 0+4+1
 0+3-2
 => 0+3+1 = 4 (fitness = 4)
 Mutate with some probability (change one
number of operator
 Put new individual into population
 Remove least fit individual (the invalid one)

15.  1+3-4 = 0
 ++23- = -100 (gets removed)
 0+4+1 = 5
 0+3-2 = 1
 0+3+1 = 4

16.  Keep going until you have a perfectly fit
individual
 Sometimes the trait you need isn’t in the
population
 That’s why mutation exists
 Applications are much wider than this, and
can generate real programming language
code

18.  http://en.wikipedia.org/wiki/Machine_learnin
g
 http://en.wikipedia.org/wiki/Genetic_progra
mming
 D. E. Goldberg and J. H. Holland, “Genetic
Algorithms and Machine Learning,” Machine
Learning, vol. 3, no. 2, pp. 95–99, 1988.