Congress on Evolutionary Computation (CEC 2016) - Plenary Talk

CEC 2016 ((28 Jul 2016)
Is evolutionary computing
evolving fast enough?
Professor Graham Kendall
Vice-Provost (Research and Knowledge Exchange)
University of Nottingham (Malaysia and UK)
School of Computer Science
ASAP Research Group

Evolutionary Computation (EC) has been part of the research agenda for at
least 60 years but if you ask the average EC researcher to name three
examples of EC being used in real world applications they might struggle.
Other technologies have seen much wider adoption. 3D printing is changing
the way that manufacturing is done, moving some of that functionality into
the home. Immersive reality is on the verge of changing society, in ways that
are not totally clear yet. Ubiquitous computing is becoming more prevalent,
enabling users to access computing resources in ways that were
unimaginable even just a few years ago. It might be argued that EC has not
had the same penetration as other technologies. In this talk, we will look
back at what EC promised, see if it has delivered on that promise and
compare its progress with other technologies. Finally, we will suggest some
challenges that might further advance EC and enable its wider adoption.
Abstract
CEC 2016 ((28 Jul 2016)

ask the average EC researcher to name three
examples of EC being used in real world applications
Abstract
CEC 2016 ((28 Jul 2016)
Hands Up!

Straw Poll
CEC 2016 ((28 Jul 2016)
Hands Up if you have registered a patent?

Straw Poll
CEC 2016 ((28 Jul 2016)
Keep your hands up if that patent is EC
related?

Straw Poll
CEC 2016 ((28 Jul 2016)
Keep your hands up if you have received a
ROI?

Real World Problems
• Writing a paper on using an evolutionary approach for a real world problem is
not the same as using an evolutionary approach to address a real world
problem
• Looking at a sample of EC papers, which are labeled as “real world
applications”, shows that the problem they tackle is a real world problem (be
that timetabling, vehicle routing, protein folding etc.) but it is tested on
benchmark datasets, or just labelled “real world” but the model would not be
suitable for the real world
• One of the problems the EC research community is that they do not tackle real,
real world problems
• This is partly (largely?) due to the industrial community not working with
universities
CEC 2016 ((28 Jul 2016)

• IT person turned Computer Scientist
• Used to write/manage large scale software
systems
• The problems that we face today seem to be
the same as I was facing in the
1970s/1980s/1990s
• Interested in how we deploy large scale
systems, on time, on budget, which work to
the satisfaction of the users
• I am not a software engineer so will not
discuss the various methodologies (e.g.
SSADM, Waterfall methods, Agile, DSDM,
Scrum etc.)
• Rather, focus on whether EC can help, if not
why not
About Me
CEC 2016 ((28 Jul 2016)

• A swipe at GP
• Many impressive results
• The outlook looks very positive
for this research domian
The Talk is Not
CEC 2016 ((28 Jul 2016)

• A call to action for better function
optimisers
• We are very good at this
CEC 2016 ((28 Jul 2016)
The Talk is Not

• Asking whether we lack
methodologies to enable us to
achieve automated software
development
• We have many (most/all) of the
tools that we need
CEC 2016 ((28 Jul 2016)
The Talk is Not

• Asking do we need to have better
automation tools?
• We have many good tools, we
are just not addressing the
problem considered here
CEC 2016 ((28 Jul 2016)
The Talk is Not

• About how to commercialise EC
software
• Erik Goodman gave an
excellent tutorial at WCCI 2016
and (even just) the slides are
extremely informative
• There are issues around
• IP
• Whether to publish
• Trust in Universities
• Support for software
• Best vehicle to use for
commercialisation (spin-
out, royalties, licensing
etc.)
CEC 2016 ((28 Jul 2016)
The Talk is Not

• About asking whether we have
the ability to have “Computer
programs that write (large scale)
computer programs”
CEC 2016 ((28 Jul 2016)
The Talk is

My Opinion
CEC 2016 ((28 Jul 2016)
• EC done in universities stays in universities
• EC done in industry stays in industry
• Large industry players (Google, Amazon, Microsoft etc.) mop up good EC
researchers and they don’t tend to publish their results
• Either too busy, no pressure to publish or commercial sensitivity
• Academia works too slowly
• Academia cannot deliver a full product/service
• Academics are “academics” and not businessmen
• There is EC out there, but it is hard to track down where it is and what it is
• Google DeepMind, and the recent AlphaGo success has got everybody
talking about AI as the “next big thing”
• We do not know how many industrial uses of EC there are

What have “we” promised
“In the 1950s, Arthur Samuel identified the goal of getting a
computer to perform a task without being explicitly programmed
as one of the central goals in the fields of computer science and
artificial intelligence. Such automatic programming of a computer
involves merely telling the computer what is to be done , rather
than explicitly telling it, step-by-step, how to perform the desired
task.”
CEC 2016 ((28 Jul 2016)
Koza J.R. Hierarchical genetic algorithms operating on populations of computer programs (1989) Proceedings of the Eleventh International Joint
Conference on Artificial Intelligence IJCAI-89, 768-774

“
”
CEC 2016 ((28 Jul 2016)
• Samuel’s own work
• Fogel/Blondie 24
• DeepMind/AlphaGo

“Conclusions
The examples from the five areas of artificial intelligence,
including sequence induction, automatic programming, function
learning, robotic planning, and pattern recognition support the
view that computational procedures (i.e. computer programs,
LISP S-expressions) can be built up from appropriate small
"building blocks" using hierarhical genetic algorithms.”
CEC 2016 ((28 Jul 2016)
Koza J.R. Hierarchical genetic algorithms operating on populations of computer programs (1989) Proceedings of the Eleventh International Joint
Conference on Artificial Intelligence IJCAI-89, 768-774

Humies (Human Competitive Results)
(A) The result was patented as an invention in the past, is an improvement over a
patented invention, or would qualify today as a patentable new invention.
(B) The result is equal to or better than a result that was accepted as a new scientific
result at the time when it was published in a peer-reviewed scientific journal.
(C) The result is equal to or better than a result that was placed into a database or
archive of results maintained by an internationally recognized panel of scientific
experts.
(D) The result is publishable in its own right as a new scientific result — independent of
the fact that the result was mechanically created.
(E) The result is equal to or better than the most recent human-created solution to a
long-standing problem for which there has been a succession of increasingly better
human-created solutions.
(F) The result is equal to or better than a result that was considered an achievement in
its field at the time it was first discovered.
(G) The result solves a problem of indisputable difficulty in its field.
(H) The result holds its own or wins a regulated competition involving human
contestants (in the form of either live human players or human-written computer
programs). CEC 2016 ((28 Jul 2016)

Humies
Recent Humie Gold Medal Winners
# Year Project Claimed
Categor(ies)
Ref
13th 2016 Automated Software Transplantation C,E,F,G,H 1
11th 2014 Genetic Algorithms for Evolving Computer
Chess Programs
B,G,H TEVC, 18(5):779-789
10th 2013 Evolutionary Design of FreeCell Solvers B,D,F,G,H TCIAG, 4(4):270-281
10th 2013 Search for a grand tour of the Jupiter Galilean
moons
C,F,H,D 4
9th 2012 Evolutionary Game Design C,D,F 5
8th 2011 GA-FreeCell: Evolving Solvers for the Game of
FreeCell
B,D,F,G,H 6
1 Barr, Earl T., Mark Harman, Yue Jia, Alexandru Marginean, and Justyna Petke. "Automated software transplantation." In Proceedings of the 2015
International Symposium on Software Testing and Analysis, pp. 257-269. ACM, 2015
4 Izzo D., SimÃµes, L.F., MÃ¤rtens, M, de Croon G., Heritier A., Hong Yam C.: "Search for a grand tour of the Jupiter Galilean moons" to be published
in Proceedings of the 15th annual conference on Genetic and evolutionary computation, ACM, 2013
5 C. Browne (2011) Evolutionary Game Design, Springer, Berlin, ISBN 978-1-4471-2178-7
6 A. Elyasaf, A. Hauptman and M. Sipper. "GA-FreeCell: Evolving Solvers for the Game of FreeCell", Genetic and Evolutionary Computation
Conference (GECCO 2011), July 2011, Dublin, Ireland CEC 2016 ((28 Jul 2016)

Humies
Categor(ies)
Ref
11th 2014 Genetic Algorithms for Evolving Computer
Chess Programs
B, G, H TEVC, 18(5):779-789
10th 2013 Evolutionary Design of FreeCell Solvers B,D,F,G,H TCIAG, 4(4):270-281
10th 2013 Search for a grand tour of the Jupiter Galilean
moons
C,F,H,D 4
9th 2012 Evolutionary Game Design C,D,F 5
8th 2011 GA-FreeCell: Evolving Solvers for the Game of
FreeCell
B,D,F,G,H 6
CEC 2016 ((28 Jul 2016)

Humies
Categor(ies)
Ref
“Automated transplantation would open many exciting avenues for software
development: suppose we could autotransplant code from one system into another,
entirely unrelated, system. This paper introduces a theory, an algorithm, and a tool that
achieve this. Leveraging lightweight annotation, program analysis identifies an organ
(interesting behavior to transplant); testing validates that the organ exhibits the desired
behavior during its extraction and after its implantation into a host. While we do not claim
automated transplantation is now a solved problem, our results are encouraging: we
report that in 12 of 15 experiments, involving 5 donors and 3 hosts (all popular real-world
systems), we successfully autotransplanted new functionality and passed all
regression tests. Autotransplantation is also already useful: in 26 hours computation time
we successfully autotransplanted the H.264 video encoding functionality from the x264
system to the VLC media player; compare this to upgrading x264 within VLC, a task that we
estimate, from VLC's version history, took human programmers an average of 20 days
of elapsed, as opposed to dedicated, time.”

“
”
CEC 2016 ((28 Jul 2016)
• Many impressive results
• Lots of research effort

“We will try to demonstrate how quick- and cheap-to-implement
knowledge-poor heuristics can be used within a hyper-heuristic
framework to provide a methodology suited to fast and cheap
development of industrial and commercial systems. This will
lead to a prototype hyper-heuristic “toolbox” for the user
community.”
EPSRC Research Proposal, 2000
CEC 2016 ((28 Jul 2016)

Bin Packing Problem
442 252 127 106 37 10 10
252 252 127 106 37 10 9
252 252 127 85 12 10 9
252 127 106 84 12 10
252 127 106 46 12 10
Pack into bins with a capacity of 524
How would you do it?
CEC 2016 ((28 Jul 2016)

Bin Packing Problem
Largest fit, first fit heuristic
Sort the objects in decreasing order of weight , taking them in this
order put each object in the first bin that will accommodate that
object. The bins are also ordered in the order they came into use.
442 252 127 106 37 10 10
252 252 127 106 37 10 9
252 252 127 85 12 10 9
252 127 106 84 12 10
252 127 106 46 12 10
CEC 2016 ((28 Jul 2016)

Bin Packing Problem
442 252 127 106 37 10 10
252 252 127 106 37 10 9
252 252 127 85 12 10 9
252 127 106 84 12 10
252 127 106 46 12 10
CEC 2016 ((28 Jul 2016)
442
252
252 252
252
C = 524
i = 33

CEC 2016 ((28 Jul 2016)
442
46
12
12
12
252
252
10
10
252
252
10
10
252
252
10
10
252
127
127
9
9
127
127
127
106
37
106
106
106
85
84
37
• 33 items
packed
• All Bins, sum of
items = 524
• Optimal solution

Bin 1 Bin 2 Bin 3 Bin 4 Bin 5 Bin 6 Bin 7
442 442
252 252
252 252
252 252
252 252
252 252
252 252
252 252
127 127
127 127
127 127
127 127
127 127
106 106
106 106
106 106
106 106
85 85
84 84
46 46
37 37
37 37
12 12
12 12
12 12
10 10
10 10
10 10
10 10
10 10
10 10
9 9
9 9
524 524 524 524 524 524 524
CEC 2016 ((28 Jul 2016)
• 33 items packed
• All Bins, sum of items = 524
• Optimal solution

Bin Packing Problem
442 252 127 106 37 10 10
252 252 127 106 37 10 9
252 252 127 85 12 10 9
252 127 106 84 12 10
252 127 106 46 12 10
Pack into bins with a capacity of 524
How would you do it?
CEC 2016 ((28 Jul 2016)
X

Remove Item 46
Bin 1 Bin 2 Bin 3 Bin 4 Bin 5 Bin 6 Bin 7 Bin 8
442 442
252 252
252 252
252 252
252 252
252 252
252 252
252 252
127 127
127 127
127 127
127 127
127 127
106 106
106 106
106 106
106 106
85 85
84 84
37 37
37 37
12 12
12 12
12 12
10 10
10 10
10 10
10 10
10 10
10 10
9 9
9 9
516 516 516 516 516 517 516 9
CEC 2016 ((28 Jul 2016)
• 32 items packed
• All Bins, sum of items
<524
• Non-optimal solution

Hyper-heuristics
Domain Barrier
……
Set of low level heuristics
Evaluation Function
Hyper-heuristic
Data flow
Data flow
H1 H2 Hn
CEC 2016 ((28 Jul 2016)

Hyper-heuristics
Domain Barrier
……
Evaluation Function
Hyper-heuristic
Data flow
Data flow
H1 H2 Hn
Generate these?
Heuristic Selection
CEC 2016 ((28 Jul 2016)
Evolve Acceptance
Function?

Hyper-heuristics
Domain Barrier
……
Evaluation Function
Hyper-heuristic
Data flow
Data flow
H1 H2 Hn
Burke, E. K; Gendreau, M; Hyde, M; Kendall, G;
Ochoa, G; Özcan, E and Qu, R Hyper-heuristics: a
survey of the state of the art. Journal of the
Operational Research Society, 64 (12): 1695-1724,
2013
CEC 2016 ((28 Jul 2016)

“
provide a methodology suited to fast and cheap
development of industrial and commercial systems
”
CEC 2016 ((28 Jul 2016)

Hyper-heuristics
1. Li, J and Kendall, G A hyper-heuristic methodology to generate adaptive strategies for
games. IEEE Transactions on Computational Intelligence and AI in Games, In Press
2. Zamil, Z. Z; Alkazemi, B. Y and Kendall, G A Tabu Search hyper-heuristic strategy for t-
way test suite generation. Applied Soft Computing, 44: 57-74, 2016.
3. Grobler, J; Engelbrecht, A. P; Kendall, G and Yadavalli, V.S.S Heuristic Space Diversity
Control for Improved Meta-Hyper-Heuristic Performance. Information Sciences, 300: 49-
62, 2015
4. Maashi, M; Kendall, G and Özcan, E Choice Function based Hyper-heuristics for Multi-
objective Optimization. Applied Soft Computing, 28: 312-326, 2015
5. Sabar, N. R; Ayob, M; Kendall, G and Qu, R A Dynamic Multiarmed Bandit-Gene
Expression Programming Hyper-Heuristic for Combinatorial Optimization Problems.
IEEE Transactions on Cybernetics, 45 (2): 217-228, 2015.
6. Sabar, N. R and Kendall, G Population based Monte Carlo tree search hyper-heuristic for
combinatorial optimization problems. Information Sciences, 314: 225-239, 2015
7. Sabar, N. R; Ayob, M; Kendall, G and Qu, R Grammatical Evolution Hyper-Heuristic for
Combinatorial Optimization Problems. IEEE Transactions on Evolutionary Computation,
17 (6): 840-861, 2013
CEC 2016 ((28 Jul 2016)

“
”
CEC 2016 ((28 Jul 2016)

Is software development that hard?
CEC 2016 ((28 Jul 2016)

• The term hacker was coined during
the pioneering days of computing,
largely at MIT
• People skilled at freaking the phone
system gradually moved over to
computing
• Their motivation was not to cause
destruction but just to work out how
things worked and to do them well
• Their motivation was not financial
• They led their life by an (unwritten)
hacker code
Hackers: Heroes of the Computer Revolution - 25th Anniversary Edition, 2010, O'Reilly Media, ISBN-13: 978-1449388393CEC 2016 ((28 Jul 2016)

• Chandler is the efforts of Mitch
Kapor (creator of Lotus 1-2-3) to
create a personal information
manager (based on Agenda)
• Released 08 Aug 2008
• How do software development
teams work (or not)?
• Why is it so difficult to reuse
software effectively or efficiently?
• See https://en.wikipedia.org/wiki/Chandler_(software)
Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software, Crown
Business; Reprint edition (February 26, 2008), ISBN-13: 978-1400082476
CEC 2016 ((28 Jul 2016)

Personal Perspective
CEC 2016 ((28 Jul 2016)
• As an IT Manager, I was often tasked
with project managing large (and even
not so large) projects
• Number of lines of code?
• Number of software elements?
• Man days/weeks/months/years?
• Within budget
• Alpha testing
• Beta Testing
• QA testing
• Deployment

Brooks's law: adding manpower to a late software project makes it later.
Software Development: Throw
money at it?
Frederick P. Brooks Jr., The Mythical Man-month: Essays on Software Engineering, 1975
CEC 2016 ((28 Jul 2016)

“The U.S. Air Force has decided to scrap a major ERP (enterprise resource
planning) software project after spending US$1 billion, concluding that
finishing it would cost far too much more money for too little gain.”
Software Failures
http://www.cio.com/article/2390341/cio-role/air-force-scraps-massive-erp-project-after-racking-up--1-billion-in-costs.html, accessed 26 Jul 2016
CEC 2016 ((28 Jul 2016)

“In 2003, Levi Strauss, was a global corporation, with operations in more
than 110 countries but with an IT system that was an antiquated,
"Balkanised" mix of incompatible country-specific systems. So its bosses
decided to migrate to a single SAP system and hired a team of fancy
consultants (from Deloitte) to lead the effort. "The risks seemed small,"
wrote the researchers. "The proposed budget was less than $5m." But
very quickly things fell apart. One major customer, Walmart, required that
the system interface with its supply chain management system, creating
additional work. During the switchover to the new system, Levi Strauss was
unable to fulfil orders and had to close its three US distribution centres for a
week. In 2008, the company took a $192.5m charge against earnings to
compensate for the botched project — and fired its chief information officer.”
Software Failures
https://www.theguardian.com/technology/2013/apr/21/fred-brooks-complex-software-projects, last accessed 26 July 2016
CEC 2016 ((28 Jul 2016)

“We reached this bleak conclusion after conducting the largest global study
ever of IT change initiatives. We examined 1,471 projects, comparing their
budgets and estimated performance benefits with the actual costs and
results. They ran the gamut from enterprise resource planning to
management information and customer relationship management systems.
Most, like the Levi Strauss project, incurred high expenses—the average
cost was $167 million, the largest $33 billion—and many were expected
to take several years. Our sample drew heavily on public agencies (92%)
and U.S.-based projects (83%), but we found little difference between them
and projects at the government agencies, private companies, and European
organizations that made up the rest of our sample.”
Software Failures
https://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/ar/1, accessed 26 July 2016
CEC 2016 ((28 Jul 2016)

http://www.computerworld.com.sg/resource/applications/top-software-failures-of-20152016/, accessed 26 July 2016
“Less than a week into 2016, HSBC suffered a major IT outage. Millions of the
bank's customers were unable to access online accounts. Services only
returned to normal after a two-day outage.
The bank's chief operating officer Jack Hackett blamed a "complex technical issue"
with its internal systems.”
“In August 2015 a reported 275,000 individual payments failed to be
processed by HSBC, which left many potentially without pay before the Bank
Holiday weekend. The cause of this major failure was a problem with its electronic
payment system for its business banking users which affected salary payments.”
CEC 2016 ((28 Jul 2016)
Software Failures

“In December 2015 a glitch caused more than 3,200 US prisoners to be
released early. The software calculates a prisoner's sentence depending on
good/bad behaviour and was introduced in 2002.
According to reports, the problem has been ongoing for 13 years until a new IT
boss was appointed and informed the governor's office. It is estimated that on
average prisoners were released 49 days early.”
“The government's online calculator for calculating spouses' financial worth was hit
with a Form E fault meaning that those calculations were wrong for thousands
of couples that have been divorced in the past 20 months.
According to reports this error has been inflating spouses' finances since April
2014 but was only brought to like in December 2015. According to the Office of
National Statistics there were 114,720 divorces in England and Wales in 2013 so
the damage that this fault has caused is yet to be fully revealed.”
CEC 2016 ((28 Jul 2016)
Software Failures

“Bloomberg's London office suffered a software glitch resulting in their trading
terminals going down for two hours.
This came at a very bad time as the UK's Debt Management Office (DMO) was set
to auction a series of short-term Treasury bills (these bills are short-term
Government borrowing). In a statement Bloomberg said: ‘Service has been fully
restored. We experienced a combination of hardware and software failures in the
network, which caused an excessive volume of network traffic.’”
“In June 2015 about 600,000 payments failed to enter the accounts of RBS
overnight - including wages and benefit payments. Many took several days to
come through. The bank's chief admin officer said a "technology fault meant we
could not ingest a file from a third-party provider". In 2012 6.5 million RBS
customers experienced an outage due to batch scheduling software, a glitch for
which the bank was subsequently fined £56 million.”
CEC 2016 ((28 Jul 2016)
Software Failures

“Over the past two years Nissan has been recalling airbags adding up to over
1 million cars. The software failure was due to a glitch in the airbag's sensory
detectors.
In short, the affected cars could not detect whether an adult was sat in the car's
passenger seat and as a result the airbags would not inflate. There has been a
reported two accidents due to this software failure.”
“Starbucks witnessed a register malfunction which according to Starbucks, was
caused by an 'internal failure' during its routine refresh. This resulted in 60
percent of stores in the US and Canada being forced to close. The affected
stores were unable to process payment transactions and at one point many stores
were giving the coffee away for free.”
CEC 2016 ((28 Jul 2016)
Software Failures

“In March 2015 the government was forced to delay launch of its £154 million
rural payments system, a new online service for farmers to use to apply for
Common Agricultural Policy payments. The service was originally supposed to be
up and running by May 2015 but integration problems between the portal and the
rules engine software proved ‘too difficult to overcome’. It is now expected to be
available for the 2016 deadline instead.”
“A 'one-off technical glitch' with processing software caused thousands of The
Co-operative Food customers to be charged twice while paying for goods in
July 2015. The Co-op apologised and promised to reimburse people within 24
hours.”
CEC 2016 ((28 Jul 2016)
Software Failures

“Just two weeks before Christmas, some of Amazon's third-party retailers in the
UK saw their wares reduced to just 1p each thanks to an hour-long pricing
software glitch. Eagle-eyed shoppers had a field day, but scores of small
businesses were left having to absorb heavy losses.”
“Over 150 flights from London airports were cancelled and hundreds more
delayed on 12 December last year. A server in Swanwick running the UK's
National Air Traffic Services (NATS) flight data processing system went down as a
result of a software flaw. NATS has experienced a number of technical failures
since the Swanwick base opened. Worryingly, software experts have warned its
systems could collapse again.”
CEC 2016 ((28 Jul 2016)
Software Failures

“Emergency services were unavailable for six hours across seven US states
last April. The incident affected 81 call centres, meaning about 6,000 people
made 911 calls that were unable to connect in Washington and parts of six other
states. A study from Federal Communications Commission found an entirely
preventable software error was responsible for causing the service to drop.”
“Despite being the world's most profitable company, Apple had a major
embarrassment in September 2014 when it had to pull the update for its new
iOS 8 operating system a mere hour after release. Users complained of lost
phone signal, frozen updates and unlocking problems. A study by Bloomberg
claimed the latest operating system crashes 67 percent more often than its
predecessor.”
CEC 2016 ((28 Jul 2016)
Software Failures

Cost of failures
CEC 2016 ((28 Jul 2016)
“Most, like the Levi Strauss project, incurred high expenses—the average
cost was $167 million, the largest $33 billion—and many were expected
to take several years.”
How many people develop software?
https://www.infoq.com/news/2014/01/IDC-software-developers, last accessed 27 July 2016

Agile Software Development
“Agile software development describes a set of principles for software
development under which requirements and solutions evolve through the
collaborative effort of self-organizing cross-functional teams. It promotes
adaptive planning, evolutionary development, early delivery, and continuous
improvement, and it encourages rapid and flexible response to change. These
principles support the definition and continuing evolution of many software
development methods.”
https://en.wikipedia.org/wiki/Agile_software_development, last accessed 27 Jul 2016
CEC 2016 ((28 Jul 2016)
This would seem to suggest that EC would be well
suited for large scale software development?

Arthur Samuel
• Samuel’s challenge: “Can we design a program
that would invent its own features in a game of
checkers and learn how to play, even up to the
level of an expert?”
• Newell’s Challenge: “Could the program learn
just by playing games against itself and receiving
feedback, not after each game, but only after a
series of games, even to the point where the
program wouldn’t even know which programs
had been won or lost?”
• Newell (and Minsky) believed that this was not
possible, arguing that the way forward was to
solve the credit assignment problem.
1. Samuel, A. L. 1959. Some studies in machine learning using the game of checkers. IBM Journal of Research and Development 3(3) 210-229
2. Samuel, A. L. 1967. Some studies in machine learning using the game of checkers ii - recent progress. IBM Journal of Research and
Development 11(6) 610-617
CEC 2016 ((28 Jul 2016)

Arthur Samuel
• Working in the late 50’s/early 60’s, Arthur Samuel
developed an algorithm that learnt to play checkers,
by playing against itself
• Bearing in mind the computing power that was
available, the experiment was a success, although
the matches against Robert Nealy were
controversial
• I can show you the relevant games available, if
interested
• Newell and Minksy would argue that you had to
solve the credit assignment problem to create an
effective checkers program
Allen Newell
Marvin Minksy
CEC 2016 ((28 Jul 2016)

David Fogel
The Gedanken Experiment
• I offer to sit down and play a game with you. We sit across an
8x8 board and I tell you the legal moves
• We play five games, only then do I say “You got 7 points”. I
don’t tell you if you win or lost
• We play another five games and I say “You got 5 points”
• You only know “higher is better”
• How long would it take you to become an expert at this game?
• We cannot conduct this experiment but we can get a computer
to do it
1. Fogel, D. B. 2002. Blondie24: Playing at the Edge of AI . Morgan Kaufmann Publishers, Inc., San Francisco, CA
2. Fogel, D. B., K. Chellapilla. 2002. Verifying anaconda's expert rating by competing against Chinook: experiments in co-evolving a neural checkers
player. Neurocomputing 42(1-4) 69-86
3. Fogel, D. B., T. J. Hays, S. L. Hahn, J. Quon. 2004. A self-learning evolutionary chess program. Proceedings of the IEEE 92(12) 19471954
CEC 2016 ((28 Jul 2016)

David Fogel
• Motivated by the defeat of Garry Kasparov in May
1997, Fogel set out to meet the challenge set by
Samuel
• Using Artificial Neural Networks as a function
evaluator, he used it at the bottom of a mini-max
search tree to evaluate board positions
• No optimization of weights and no evaluation
function
• Population of 30 players , played against each other
• After various experimental setups a player was
evolved that was rated over 2000 (expert level) and
that bear over 99% of players of zone.com
• Samuel’s challenge had been met
CEC 2016 ((28 Jul 2016)

Work Continues
1. Al-Khateeb, B and Kendall, G Introducing Individual and Social Learning Into
Evolutionary Checkers. IEEE Transactions on Computational Intelligence and AI in
Games, 4 (4): 258-269, 2012
2. Kendall, G and Su, Y Imperfect Evolutionary Systems. IEEE Transactions on
Evolutionary Computation, 11 (3): 294-307, 2007
3. Kendall, G; Yaakob, R and Hingston, P An Investigation of an Evolutionary Approach to
the Opening of Go. In Proceedings of the 2004 IEEE Congress on Evolutionary
Computation (CEC'04), pages 2052-2059, Portland, Oregon, 2004
4. Davis, J.E and Kendall, G An Investigation, using Co-Evolution, to Evolve an Awari
Player. In Proceedings of the 2002 Congress on Evolutionary Computation (CEC 2002),
pages 1408-1413, Hilton Hawaiian Village Hotel, Honolulu, Hawaii, May 12-17, 2002
5. Kendall, G and Smith, C The evolution of blackjack strategies. In Proceedings of the
The IEEE 2003 Congress on Evolutionary Computation (CEC2003), pages 2474-2481,
Canberra, Australia, 2003
CEC 2016 ((28 Jul 2016)

Digression
• Johnathan Schaeffer
• Chinook
• Marian Tinsley
• Checkers is Solved
• Checkers has roughly 500 billion billion
possible positions (5 × 1020)
• Perfect play by both sides leads to a draw
• DOI: 10.1126/science.1144079
CEC 2016 ((28 Jul 2016)

Genetic Algorithms and Programming
• Motivated by Darwin’s principles of natural
evolution (Survival of the Fittest)
• Evolve solutions to problem, rather than applying a
more traditional algorithmic design approach
• GA’s use a chromosome representation, GP uses
a tree based representation
• GAs/GPs are some of the best known examples of
many evolutionary algorithms. Others include
PSO, ACO,HBO etc.
• The proliferation of these algorithms is not without
criticism
• Sörensen K. 2013. Metaheuristics – the metaphor exposed.
International Transactions on Operational Research,
22(1):3-18
CEC 2016 ((28 Jul 2016)

Opportunities
CEC 2016 ((28 Jul 2016)

Dynamic Adaptive Automated Software Engineering
(DAASE)
“Current software development processes are expensive, laborious and error
prone. They achieve adaptivity at only a glacial pace, largely through enormous
human effort, forcing highly skilled engineers to waste significant time adapting
many tedious implementation details. Often, the resulting software is equally
inflexible, forcing users to also rely on their innate human adaptivity to find
"workarounds". Yet software is one of the most inherently flexible engineering
materials with which we have worked, DAASE seeks to use computational search
as an overall approach to achieve the software's full potential for flexibility and
adaptivity. In so-doing we will be creating new ways to develop and deploy
software. DAASE will also create an array of new processes, methods, techniques
and tools for a new kind of software engineering, radically transforming the theory
and practice of software engineering.”
CEC 2016 ((28 Jul 2016)
http://daase.cs.ucl.ac.uk/about/, last accessed 28 July 2016

Home users
• How can we package up software development for the novice user, such that
an easy to use user interface enables anybody to develop software for any
purpose
• How can we make developing software as accessible as 3D printing
• Hyper-heuristics (this was certainly our intent)?
• Genetic Programming?
CEC 2016 ((28 Jul 2016)

Commercial Systems - 1
• There will always be a need to do the mundane stuff (e.g. database updates)
• Development effort should be spent on defining the objective function (ref Xin
Yao’s plenary)
• Requirements could be adapted in real time by users, to guide an evolutionary
process
• Could include test cases that are hard constraints (so, if not satisfied the
program is not a valid solution)
• Well defined APIs are essential, probably evolved
• Evolve small building blocks that could connect together, similar to “Mashups”
• Terrazas, G; Siepmann, P; Kendall, G and Krasnogor, N An
Evolutionary Methodology for the Automated Design of Cellular
Automaton-based Complex Systems. Journal of Cellular Automata, 2
(1): 77-102, 2007
CEC 2016 ((28 Jul 2016)

Commercial Systems - 2
• Utilise Cloud computing (ref Una-May O’Reilly’s plenary)
• Evolution could be run 24/7 and the objective function could be changed at any
time
• A Quality Assurance system could be employed comprising humans,
automated testers, or even an arms race to test evolved software before
deployment
CEC 2016 ((28 Jul 2016)

Concluding Remarks
• A challenge to Computer Science is to make large scale software easier to
develop
• Make software development as easy as 3D printing
• The cost to industry is immense, especially, if things go wrong
• Evolutionary Computation is well placed to make advances in this area
• Most (all) of the automated software development methodologies that I know of
are search, or can be defined as search
• Search for a program
• Search for a heuristic selection algorithm
• Search for new heuristics
• But that is an Operations Research bias
• I believe that this is one of the biggest challenges that faces Computer
Science, due to the expense, timescales and problems that are currently
experienced
CEC 2016 ((28 Jul 2016)

Thank you
Q&A
CEC 2016 ((28 Jul 2016)

Congress on Evolutionary Computation (CEC 2016) - Plenary Talk

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (14)

Similaire à Congress on Evolutionary Computation (CEC 2016) - Plenary Talk

Similaire à Congress on Evolutionary Computation (CEC 2016) - Plenary Talk (20)

Dernier

Dernier (20)

Congress on Evolutionary Computation (CEC 2016) - Plenary Talk