TenYearsCPOptimizer

© 2018 IBM Corporation
Ten Years of CP Optimizer
Paul Shaw, IBM Analytics
CPAIOR 2018, Delft, 26-29 June, 2018

What is CP Optimizer?
 CP Optimizer is a constraint programming engine with a focus on scheduling problems
 Version 1.0 released in 2007
 Version 2.0 released in 2008 integrating scheduling
 Most recent version is 12.8, released at the end of 2017
– Jumped 10 versions in 2010!
 CP Optimizer is part of CPLEX Optimization Studio
– CPLEX mathematical programming engine
– OPL development environment

What is CP Optimizer?
 CP Optimizer is a constraint programming engine with a focus on scheduling problems
 Version 1.0 released in 2007
 Version 2.0 released in 2008 integrating scheduling
 Most recent version is 12.8, released at the end of 2017
– Jumped 10 versions in 2010!
 CP Optimizer is part of CPLEX Optimization Studio
– CPLEX mathemetical programming engine
– OPL development environment
CPLEX Optimization
Studio is free for both
students and academics
Google
“Student CPLEX 12.8”

Where is it used?
 Mostly for industrial scheduling. e.g.
– Port management
– Aircraft assembly
– Manufacturing. e.g. Plastics and
textiles
– Electronics and chip fabrication
– Automated laboratories
– Integrated facility management
(workforce scheduling)

Performance
CP Optimizer timeline
201320122011201020092008 2014 2015 2016 2017 2018
Automatic
solution
search
Interval-based
scheduling
language
Scheduling
relaxations
with CPLEX
Lexicographic
multi-criterion
optimization
Infeasibility explainer
Starting solution
Presolve
“Strong” constraints
Modeling
assistance
Objective
bounds
Determ.
parallel
solving
Backtrack explainer

How we work
 Our preoccupations:
– Solve more quickly, or produce good solutions earlier
– Make model development easier, faster, or clearer
– Allow you to do something (useful!) you couldn't do before
 Examples:
– Failure-directed search for scheduling (faster solve)
– Modeling assistance (eases model development)
– Model file format (allows you to store and examine model instances)

How we work
 Quality
– We run CP Optimizer on thousands of models a day, both pre-existing and randomly
generated. We favor robustness over potentially spectacular but mercurial behavior
 Flexibility
– Make sure CP Optimizer can be integrated into optimization workflows: you can
provide indications on search strategy, customize parts of the solver, inject initial
solutions, abort the solver cleanly, combine easily with CPLEX, etc.
 Access
– Multiple languages or APIs, solve locally or on IBM cloud, use from DSX Local
 Support
– Help our users succeed in what they want to do. Contact us on the forums!

A short history
 Evolved from the ILOG products Solver and Scheduler
 ILOG Scheduler in particular had grown unwieldy
– Reference manual was over 900 pages
– Very difficult to master

A short history
 We created a new product which would be easier to use. Three-way approach:
– Simpler modeling with fewer constructs which can be more freely mixed
– Automatic search to allow the user to concentrate on modeling
– Associated tools to aid in model creation and debugging

A short history
IBM ILOG CP optimizer for scheduling:
20+ years of scheduling with constraints at IBM/ILOG
Constraints, March 2018
Reasoning with conditional time intervals, Flairs 2008
Reasoning with conditional time intervals part II:
An algebraical model for resources, Flairs 2009

A short history
 Conditional interval variable is basic decision object
 Time relations between intervals (precedences)
 Logical constraints between interval presence
 Resource usage is represented as functions of time
which are decided on the position of intervals
 Resource capacities are constraints on these fns.
 Constraints to handle alternatives and hierarchies

Automatic Search
Complete
Can prove optimality
or infeasibility
Anytime
Produces improving
solutions as they are found
Parallel
Use multiple cores to
speed up solving
Randomized
Internally, ties are broken
pseudo-randomly
Deterministic
Multiple runs will produce
the same result
Configurable
Parameters
Limits, inference levels,
control search, display
Starting points
Give an initial solution
that you want to improve
Search Phases
Specify variable and
value choice strategy

Native
Model
Worker 1 Worker n
Java
Python
C#
C++
OPL
Search Controller
CPO
Minizinc
FZN
CP Optimizer workflow
Presolved Model
Solutions
No-goods
Search statuses
…..
create
import
export
config / action
results

Interfaces: creating an “integer” model
 Same modeling power in Python, OPL, C++, Java, C#
 In C++, customize the engine (search strategy, custom constraints)
from docplex.cp.model import CpoModel
n = 5
mdl = CpoModel()
x = [ mdl.integer_var(0, n-1, "X{}".format(i)) for i in range(n) ]
mdl.add(mdl.all_diff(x))
for d in range(1, n-1) :
mdl.add(mdl.all_diff(x[k+d] - x[k] for k in range(n-d)))
res = mdl.solve()
Costas Array
All vectors between pairs of
points (i, x[i]) are different

Interfaces: creating a scheduling model
Job Shop Scheduling
jobs = [[(0, 2), (1, 3), (2, 1)],
[(1, 3), (0, 4), (2, 2)],
[(2, 1), (1, 4), (0, 2)]]
nmach = 1 + max(o[0] for j in jobs for o in j)
mach = [ [] for _ in range(nmach) ]
mdl = CpoModel()
op = [ [ mdl.interval_var(size=o[1],name='J_{}_{}'.format(jid, oid))
for oid,o in enumerate(j) ] for jid, j in enumerate(jobs) ]
for jid,j in enumerate(jobs) :
for oid,o in enumerate(j) :
mach[o[0]].append(op[jid][oid])
if oid > 0 :
mdl.add(mdl.end_before_start(op[jid][oid-1], op[jid][oid]))
for m in mach:
mdl.add(mdl.no_overlap(m))
mdl.add(mdl.minimize(mdl.max(mdl.end_of(j[-1]) for j in op)))
res = mdl.solve()

Engine log: Costas Array
! ----------------------------------------------------------------------------
! Satisfiability problem - 13 variables, 12 constraints
! Initial process time : 0.00s (0.00s extraction + 0.00s propagation)
! . Log search space : 48.1 (before), 48.1 (after)
! . Memory usage : 339.5 kB (before), 339.5 kB (after)
! Using parallel search with 4 workers.
! ----------------------------------------------------------------------------
! Branches Non-fixed W Branch decision
1000 4 1 9 != X4
1000 4 2 8 != X1
(abridged)
2000 4 4 F 4 = X9
* 2881 0.04s 1 11 != X1
! ----------------------------------------------------------------------------
! Search completed, 1 solution found.
! Number of branches : 9881
! Number of fails : 4750
! Total memory usage : 2.5 MB (2.4 MB CP Optimizer + 0.0 MB Concert)
! Time spent in solve : 0.04s (0.04s engine + 0.00s extraction)
! Search speed (br. / s) : 247025.0
! ----------------------------------------------------------------------------

File format
///////////////////////////////////////////////////////////////////////////////
// CPO file generated at 2018.06.22-17:02:26 for model: jobshop
// Source file: jobshop.py
///////////////////////////////////////////////////////////////////////////////
//--- Internals ---
internals {
version(12.8.0.0);
}
//--- Constants ---
//--- Variables ---
J_0_0 = intervalVar(size=2);
//--- Expressions ---
#line 20 "jobshop.py"
endBeforeStart(J_0_0, J_0_1);
#line 22
noOverlap([J_0_0, J_1_1, J_2_2]);
noOverlap([J_0_1, J_1_0, J_2_1]);
noOverlap([J_0_2, J_1_2, J_2_0]);
#line 23
minimize(max([endOf(J_0_2),endOf(J_1_2), endOf(J_2_2)]));
Job Shop Scheduling

File format: extensions for configuring CP Optimizer
Job Shop Scheduling
//--- Search phases ---
search {
#line 23
searchPhase([J_0_0, J_0_1, J_0_2]);
}
//--- Parameters ---
#line off
parameters {
TimeLimit = 10;
Workers = 2;
}
//--- Starting point ---
#line off
startingPoint {
J_0_0 = (present, start=0);
}

Engine log: Job shop with parameterization
! ----------------------------------------------------------------------------
! Minimization problem - 12 variables, 9 constraints, 1 phase
! Using starting point solution
! TimeLimit = 10
! Workers = 2
! Initial process time : 0.00s (0.00s extraction + 0.00s propagation)
! . Log search space : 28.5 (before), 28.5 (after)
! . Memory usage : 442.2 kB (before), 442.2 kB (after)
! Using parallel search with 2 workers.
! ----------------------------------------------------------------------------
! Best Branches Non-fixed W Branch decision
0 12 -
+ New bound is 9
! Starting point is complete, restoration succeeded.
* 42 1 0.00s 1 (gap is 78.57%)
* 12 19 0.00s 1 (gap is 25.00%)
* 11 24 0.00s 1 (gap is 18.18%)
11 25 12 1 F -
+ New bound is 11 (gap is 0%)
(abridged)

Presolve
 Model analysis before search begins. The hope is to improve the formulation by replacing
constructs with others that will be more efficient
 Transforms the initial model into a new one. The transformed model is fed to the workers
 CP Optimizer does different types of operations inside presolve
– Basic simplifications. e.g. constant propagation, linear simplification
– Aggregation / combination. e.g. common sub-expression elimination
– Higher level transformations

Presolve: some examples
 Constant propagation
– 7 * (2*x + 1) + 2 → 14 * x + 9
 Application of De Morgan's laws
– !(z  2 && max(x,y)  5) → (z  1) || (max(x, y)  4)
 Aggregation of difference constraints
– (x != y) && (x != z) && (y != z) → allDiff([x, y, z])
 Rewriting of scheduling expressions
– startOf(y)  endOf(x) → endBeforeStart(x, y)
– endOf(x) – startOf(x) → lengthOf(x)

Presolve: some more examples
 Square detection
– (x + y) * (x + y)  100→ square(x+y)  100
 Neutral element removal
– x  10..100 && y  -100..100 && min(x, y)  0 → y  -100..0
 Tautology removal
– x  0..1 && element([2, 5], x)  0 → true
 Clause recognition and grouping
– x + y - z  0 → clause([1, 1, -1], x, y, z)

Presolve: what difference does it make?
 Compare CP Optimizer on a set of models with and without presolve
– Parameter Presolve=On vs Presolve=Off
 Set of 77 families of assorted (non-scheduling) problems, including Minizinc problems
 Performance is measured by adjusting the speed of a solver to get the same Minizinc score
as the other on a family of problems. Geometric mean of speeds is taken over families

Presolve: what difference does it make?
 Compare CP Optimizer on a set of models with and without presolve
– Parameter Presolve=On vs Presolve=Off
as the other on a family of problems. Geometric mean of speeds is taken over families
Presolve Speed
change
# families
faster
# families
slower
# families
similar
On +33% 37 11 29

Native
Model
Java
Python
C#
C++
OPL
Search Controller
CPO
Minizinc
FZN
Presolved Model
Solutions
No-goods
Search statuses
…..
create
import
export
config / action
results
Worker 1 Worker n

What is a worker?
 A worker is a backtracking search engine with constraint propagation
 Although DFS is the basic mechanism, full DFS on the problem is not used
– DFS is a building block in a larger scheme (as in restarts)
 Propagation - all the “standard” stuff, plus:
– Level is adjustable using parameters, and can be locally adapted by the worker
– Constraint combinations for better propagation (e.g. temporal and logical networks)
– CPLEX is used as a relaxation for scheduling problems
– User can use “strong” constraints (typically for integer problems)

What do workers do?
 The workers mostly do a quite similar job to each other. They:
– Try to find solutions better than the current incumbent
– Try to find a proof of optimality or infeasibility
 To do this they interleave two sets of techniques designed to do either one job or the other
– To improve solutions, essentially local techniques are used (e.g. find a solution close
to the current incumbent)
– To find proofs, essentially global techniques are used (use an adaptive tree-based
search looking to traverse the search space quickly)
– This is the case for both integer and scheduling problems

What do workers do? - Case of scheduling

reoptimize

Use of CPLEX
 CPLEX is used to provide a relaxation to the scheduling (sub-)problem
 What is relaxed?
– Irregular cost functions
– Precedences, logical constraints between intervals
– Alternatives, spans
– Linear constraints in the formulation
– etc. P. Laborie, J. Rogerie. Temporal Linear
Relaxation in IBM ILOG CP Optimizer.
Journal of Scheduling 19(4), 391–400 (2016)

Use of CPLEX: the effect
 In the paper, 15 different “irregular cost” benchmarks were studied

Failure-directed Search
 LNS has been used to solve scheduling problems since 2008 in CP Optimizer, but FDS was
introduced later in 2013
 Before this, it was rare for CP Optimizer to produce proofs on non-trival problems
 As measured on our benchmarks, FDS gave a global 2x speed increase
 FDS allowed to improve hundreds of bounds (lower and upper) and close a significant
number of classic instances:
http://vilim.eu/petr/cpaior2015-results.pdf

Failure-directed Search
http://vilim.eu/petr/cpaior2015-results.pdf

SearchController
reoptimize

Native
Model
Worker 1 Worker n
Java
Python
C#
C++
OPL
CPO
Minizinc
FZN
Presolved Model
Solutions
No-goods
Search statuses
…..
create
import
export
config / action
results
Search Controller

Performance: Search controller no-good communications
 Compare CP Optimizer on a set of models with and without no-good communication
– No-good communication On or Off
as the other on a family of problems. Geometric mean is taken over families

Performance: Search controller no-good communications
 Compare CP Optimizer on a set of models with and without no-good communication
– No-good communication On or Off
as the other on a family of problems. Geometric mean is taken over families
Comms. Speed
change
# families
faster
# families
slower
# families
similar
On +30% 53 4 20

Performance evolution
Performance on Scheduling

Performance evolution
Integer performance
evolution is more subdued
But over 3X since 2014
Performance on Scheduling

A look at the Minizinc 2017 challenge
 We use numerous problem families from the Minizinc problem base to test CP Optimizer
– Mostly integer (not scheduling) problems
– I pulled these data from some standard CP Optimizer 12.8 runs on our system
 The data are not produced under competition conditions and indicative only!
– We use our own machines (quite old now, from around 2011)
– We are reading CPO, not FZN files (previously converted)
– (We artificially adjust times by adding mzn2fzn and FZN→CPO conversion times)
 We ran CP Optimizer 12.8 10 times on each instance and took the median
 Default settings are used. The specified Minizinc strategy is ignored

A look at the Minizinc 2017 challenge - “free” category
Solver Score
CP Optimizer 1495
Chuffed 1376
iZplus 1316
or-tools-lcg 1249
mzn-gurobi 1227
1st 2nd 3rd 4-5 6-10 11-15 16-22
0
1
2
3
4
5
6
7
8
9
Distribution of ranks
CP Optimizer
Chuffed
izplus
or-tools-lcg
mzn-gurobi
Rank
Numberoffamilies

A look at the Minizinc 2017 challenge - “par” category
Solver Score
CP Optimizer 1491
lgc-glucose 1464
Gecode 1389
Chuffed (“free”) 1302
Choco4 1277
1st 2nd 3rd 4-5 6-10 11-15 16-22
0
2
4
6
8
10
12
Distribution of ranks
CP Optimizer
lcg-glucose
Gecode
Chuffed ("free')
Choco4
Rank
Numberoffamilies

CP Optimizer's Tools
 Engine log, file format - Already seen these. Useful for understanding and debugging
 CP Optimizer Interactive
– Load and solve models, change parameters
– Perform multiple runs with different seeds to evaluate model changes
 Modeling Assistance
 Infeasibility Explainer (Conflict Refiner)
 Backtrack Explainer
 Objective Bound

Modeling assistance
 Warnings are emitted when potentially suspicious modeling patterns are seen
 These may be more or less worrying and have three priority levels
– Parameter WarningLevel can go from zero (no warnings) to 3 (all warnings)
 These warnings include the line of the code which generated the model and the part of the
model in the CPO file format
 CP Optimizer can issue today around 80 usage warnings

Modeling assistance
mdl = CpoModel()
x,y,z = mdl.integer_var(0, 1, "x"), mdl.integer_var(0, 99, “y”), mdl.integer_var(0, 99, “z')
mdl.add(y + z >= 0)
mdl.add(x == (0.2 * y >= z / 3))
mdl.add(mdl.minimize(x))
mdl.solve()

Modeling assistance
mdl = CpoModel()
x,y,z = mdl.integer_var(0, 1, "x"), mdl.integer_var(0, 99, “y”), mdl.integer_var(0, 99, “z')
mdl.add(y + z >= 0)
mdl.add(x == (0.2 * y >= z / 3))
mdl.add(mdl.minimize(x))
mdl.solve()
mod_ass.py:8(stream:24:15): Warning: Comparison of floating point expressions may result in true or false
for very small changes in expression values.
0.2 * y >= z / 3
mod_ass.py:7(stream:22:7): Warning: The constraint is always true, it will be removed.
y + z >= 0

Modeling assistance

Explaining infeasibility (Conflict Refiner)
 Commonly when developing an optimization model, one comes across an infeasible model
– A bug in the model
– Bad data
– Some real-world aspect over-constraining the problem
 You may also wish to create an infeasible model to ask a question
– If it is impossible to place Tom on the Monday shift, then why?
 You would like a compact explanation for the infeasibility
 The “conflict refiner” produces a (locally) minimal conflict explaining the infeasibility

Satellite
scheduling
model in OPL

!----------------------------------------------------------------------------
! Satisfiability problem - 2,980 variables, 851 constraints
! Problem found infeasible at the root node
! ----------------------------------------------------------------------------
...
! ----------------------------------------------------------------------------
! Conflict refining - 851 constraints
! ----------------------------------------------------------------------------
! Iteration Number of constraints
* 1 851
* 2 426
...
* 58 5
* 59 5
! Conflict refining terminated
! ----------------------------------------------------------------------------
! Conflict status : Terminated normally, conflict found
! Conflict size : 5 constraints
! Number of iterations : 59
! Total memory usage : 13.3 MB
! Conflict computation time : 0.51s
! ----------------------------------------------------------------------------

 The conflict refiner is highly configurable
– Say which constraints can be considered (some constraints are “background”)
– Consider variable domains or not
– Group constraints (either all or none of each group in the conflict)
1232 1266
134A
12721238
144
1228 1260
146
1230 1262
146A

Explaining backtrack
 Explaining failure #3 in a simple facility location problem
- Failure #3
-- Possible conflict explaining failure
// Model constraints
#line 120 "../../../examples/src/cpp/facility_explanations.cpp"
element(loc_0, [open_0, open_1, open_2, open_3, open_4]) == 1;
#line 124
count([loc_0, loc_1, loc_2, loc_3, loc_4, loc_6, loc_7, loc_8], 0) <= 3;
// Branch constraints
open_1 == 0;
open_2 == 0;
open_4 == 0;

- Failure #3
#line 124
open_1 == 0;
open_2 == 0;
open_4 == 0;
capacity

- Failure #3
#line 124
open_1 == 0;
open_2 == 0;
open_4 == 0;
capacity
Why not add ∑i
Ci
oi
≥ N ?

Objective bounds
 CP is not known for having very good objective bounds
– Yet for some scheduling problems, notably those minimizing makespan, the bound
can be quite reasonable
– We decided to give this information to the user
 Today, we don't do much to actively improve the bound. Help us improve!
– Discovery of new unary no-goods can increase the bound
– FDS performs some “strong branching” (one level lookahead on some variables)
●
Weakest bound from the sub-problems may dominate the current bound

Objective bounds

Objective bounds
 Scheduling bound is
useful in a number of
cases (over 35% of
unsolved problems have
a bound under 10%)
 On integer problems, the
bound is rarely useful

Some topics that didn't make it...
 Scheduling language
 Lexicographic multi-objective
 Impact-based search
 Dynamic probing
 Multipoint search (evolutionary algorithm / CP hybrid)
 CP Optimizer interactive optimizer
 “Strong” constraints
 Custom constraints and search
 Expression “compilation”
 Objective landscapes
 Combinations with CPLEX
 etc.

Performance
CP Optimizer timeline
201320122011201020092008 2014 2015 2016 2017 2018
Automatic
solution
search
Interval-based
scheduling
language
Scheduling
relaxations
with CPLEX
Lexicographic
multi-criterion
optimization
Infeasibility explainer
Starting solution
“Strong” constraints
Modeling
assistance
Objective
bounds
Determ.
parallel
solving
Backtrack explainer
Presolve

TenYearsCPOptimizer

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to TenYearsCPOptimizer

Similar to TenYearsCPOptimizer (20)

Recently uploaded

Recently uploaded (20)

TenYearsCPOptimizer