This presentation discusses advances in CPLEX 12.5.1 with special attention to using the remote object facility for building advanced distributed memory optimization algorithms. Performance improvements in CPLEX 12.5.1 complete the presentation.

1. © 2013 IBM Corporation
®
Using
CPLEX 12.5.1
Remote Objects
Roland Wunderling
roland.wunderling@at.ibm.com

2. IBM Software Group
© 2013 IBM Corporation2
Agenda
 CPLEX 12.5.1 updates
 CPLEX remote object
• Synchronous use
• Asynchronous use
 Examples
– Distributed Decomposition Algorithm
• Decomposition
• Distributed Parallel Benders Decomposition
– Distributed MIP solver
• Simple
• Advanced
 CPLEX 12.5.1 performance

3. IBM Software Group
© 2013 IBM Corporation
CPLEX 12.5.1 Updates
3

4. IBM Software Group
© 2013 IBM Corporation
CPLEX 12.5.1 updates
 Ports
– Support for Microsoft Visual Studio 2012
– Support for Windows 8 and Server 2012
– Support for Xcode and 64-bit Python on Mac OS X
 Various fixes
4

5. IBM Software Group
© 2013 IBM Corporation
CPLEX 12.5.1 updates
 Added functionality to OPL
– Extended scripting
• New relaxation iterator IloOplRelaxationIterator
• New conflict iterator IloOplConflictIterator
• Support of multiple MIP starts
– Extended IloOplConflictIterator for Java and .NET
 Added functionality to CPLEX
– Duals for QCPs
• Parameter CPX_PARAM_CALCQCPDUALS
– Remote Object support for C++ and Java
5

6. IBM Software Group
© 2013 IBM Corporation
CPLEX Remote Objects
6

7. IBM Software Group
© 2013 IBM Corporation7
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

8. IBM Software Group
© 2013 IBM Corporation8
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
– Slow performance on local hardware
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

9. IBM Software Group
© 2013 IBM Corporation9
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
– Slow performance on local hardware
– High Performance Hardware available
• Port application to HPC
 Needs full environment / licenses on Server
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

10. IBM Software Group
© 2013 IBM Corporation10
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
– Slow performance on local hardware
– High Performance Hardware available
• Port application to HPC
 Needs full environment / licenses on Server
• Use CPLEX as Remote Object
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

11. IBM Software Group
© 2013 IBM Corporation11
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
– Change single line of code to establish connection
• Constructor takes arguments to describe how to
access remote CPLEX
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

12. IBM Software Group
© 2013 IBM Corporation12
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
– Change single line of code to establish connection
• Constructor takes arguments to describe how to
access remote CPLEX
– All methods are mapped to messages to remote
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

13. IBM Software Group
© 2013 IBM Corporation13
CPLEX remote object
 Available in C since CPLEX 12.5.0 – C++ & Java 12.5.1
– Transparent use of remote hardware
– Change single line of code to establish connection
• Constructor takes arguments to describe how to
access remote CPLEX
– All methods are mapped to messages to remote
– solve() method used high performance hardware
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}

14. IBM Software Group
© 2013 IBM Corporation14
CPLEX remote object - Transports
 Transports
– Describe how to potentially initiate and connect to remote
CPLEX object
 3(+1) transports available
– ssh
• Start CPLEX on remote hardware via
ssh <host> cplex –worker=process
– TCP/IP
• Connect to CPLEX that has been started on remote hardware via
cplex –worker=tcpip –address=<host:port>
– MPI
• Message Passing Interface
• Established standard for HPC since 1992
– (NULL, i.e. use local object)

15. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
15
Local CPLEX CPLEX remote object

16. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
 IloCplex cplex(env, ?);
16
cplex cplex (proxy) cplex
Local CPLEX CPLEX remote object

17. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
17
cplex (proxy)
Local CPLEX CPLEX remote object
cplex cplex

18. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 cplex.solve();
 Synchronous: Blocks until completed
18
cplex
cplex (proxy)
cplex
Local CPLEX CPLEX remote object

19. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 cplex.solve();
 Synchronous: Blocks until completed
19
cplex
cplex (proxy)
cplex
Local CPLEX CPLEX remote object
solution solution

20. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 cplex.solve();
 Synchronous: Blocks until completed
 useSolution(cplex);
20
cplex
cplex (proxy)
cplex
Local CPLEX CPLEX remote object
solution solution

21. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - architecture
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 cplex.solve();
 Synchronous: Blocks until completed
 useSolution(cplex);
 cplex.end();
21
Local CPLEX CPLEX remote object

22. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Why idle during cplex.solve()?
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 handle = cplex.solve(true);
 Asynchronous: non-blocking
22
cplex (proxy)
cplex
CPLEX remote object

23. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Why idle during cplex.solve()?
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 handle = cplex.solve(true);
 Asynchronous: non-blocking
 doOtherThings();
23
cplex (proxy)
cplex
CPLEX remote object

24. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Why idle during cplex.solve()?
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 handle = cplex.solve(true);
 Asynchronous: non-blocking
 doOtherThings();
 handle.join();
24
cplex (proxy)
cplex
CPLEX remote object

25. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Why idle during cplex.solve()?
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 handle = cplex.solve(true);
 Asynchronous: non-blocking
 doOtherThings();
 handle.join();
 Blocking until solution available
25
cplex (proxy)
cplex
CPLEX remote object
solution

26. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Why idle during cplex.solve()?
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 handle = cplex.solve(true);
 Asynchronous: non-blocking
 doOtherThings();
 handle.join();
 Blocking until solution available
 useSolution(cplex);
26
cplex (proxy)
cplex
CPLEX remote object
solution

27. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Why idle during cplex.solve()?
 IloCplex cplex(env, ?);
 buildModel(cplex, data);
 handle = cplex.solve(true);
 Asynchronous: non-blocking
 doOtherThings();
 handle.join();
 Blocking until solution available
 useSolution(cplex);
 cplex.end();
27
CPLEX remote object

28. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 Asynchronous calls support a master/worker distributed
parallel programming paradigm for building distributed
parallel optimization algorithms
 Asynchronous calls return AsyncHandle objects to join
back
 Different subclasses for different asynchronous calls to
handle different return data in specific join*() method,
e.g.
– SolveHandle IloCplex::solve()
– bool SolveHandle::joinSolve()
28

29. IBM Software Group
© 2013 IBM Corporation
Example
-
Distributed Decomposition
Algorithms
29

30. IBM Software Group
© 2013 IBM Corporation
Decomposition
 Decomposition
– Solve larger problem by partitioning into smaller parts and
merging the results
 Sequential workflow
30
Main
Problem
Split
Part 1
Part 2
Part N
Soln 1
Soln 2
Soln N
Solve
Solve
Solve
Combine
Main
Solution

31. IBM Software Group
© 2013 IBM Corporation
Decomposition
 Decomposition
– Solve larger problem by partitioning into smaller parts and
merging the results
 Parallel workflow
31
Main
Problem
Split
Part 1
Part 2
Part N
Soln 1
Soln 2
Soln N
Solve
Solve
Solve
Combine
Main
Solution

32. IBM Software Group
© 2013 IBM Corporation
Decomposition
 Decomposable matrix
min c’x
s.t. Ax = b, with A =
x ≥ 0
 Solve each block i independently
min ci’xi
s.t. Aixi = bi
xi ≥ 0
32
A1
A2
An

33. IBM Software Group
© 2013 IBM Corporation
Decomposition
 Sequential code
solveParts (int N, IloModel *part, IloNumVarArray *vars) {
IloNumArray solution(part[0].getEnv());
IloCplex cplex[P];
for (int p = 0; p < N; ++p) {
// solve each part
cplex[p] = IloCplex(part[p]);
if ( cplex[p].solve() ) {
// collect and combine results
IloNumArray partsol(part[p].getEnv());
cplex[p].getValues(parsol, vars[p]);
solution.add(partsol);
partsol.end();
cplex[p].end();
}
}
return solution;
}
33

34. IBM Software Group
© 2013 IBM Corporation
Decomposition
 Distributed parallel code
solveParts (int N, IloModel *part, IloNumVarArray *vars) {
IloNumArray solution(part[0].getEnv());
IloCplex cplex[P];
IloCplex::SolveHandle handle[P];
for (int p = 0; p < N; ++p) {
// solve each part
cplex[p] = IloCplex(part[p], transport, argc, argv);
handle[p] = cplex.solve(true);
}
for (int p = 0; p < N; ++p) {
if ( handle[p].joinSolve() ) {
// collect and combine results
IloNumArray partsol(part[p].getEnv());
cplex[p].getValues(parsol, vars[p]);
solution.add(partsol);
partsol.end();
cplex[p].end();
}
}
return solution;
}
34

35. IBM Software Group
© 2013 IBM Corporation
Benders Decomposition
 Block-structured Problem
min c’x
s.t. Ax = b, with A =
x ≥ 0
 Transform into 2-stage problem
min {c0’x0 + min{d’y : Dy = b-Ax0, y ≥ 0} : solvable, x0 ≥ 0},
where d’ = (c’1, …, c’n),
y’ = (x’1, …, x’n), and
D = diag(A1, …, An).
35
A1
A2
An
A0

36. IBM Software Group
© 2013 IBM Corporation
Benders Decomposition
 Sketch of Algorithm
1. Solve restricted master problem to produce x0
2. Solve subproblem on y
a) If subproblem solves to objective limit goto 3
b) Otherwise add cut to master problem and goto 1
3. (x0,y) is optimal solution to problem
 Implementation (see example iloparbenders.cpp)
– Step 2 implemented as lazy constraint callback for solver of
step 1
– Step 2 implemented using distributed decomposition
algorithm adding cuts separately per part
36

37. IBM Software Group
© 2013 IBM Corporation
Example
-
Distributed MIP Solver
37

38. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object – asynchronous calls
 All methods of class IloCplex::AsyncHandle
– join()
• Blocks until asynchronous call has completed
– test()
• Tests if asynchronous call has completed
– kill()
• Interrupt asynchronous call (still need to call join())
 Support more types of parallel algorithms
38

39. IBM Software Group
© 2013 IBM Corporation
Simple Example: Distributed Concurrent MIP
 Idea
– Execute several solvers concurrently on remote hardware
each with different parameter settings
– Stop when first solver is done
 Master (simplified)
distributedConcurrentMIPopt(const char* modelfile)
{
IloEnv env;
IloModel model(env);
IloCplex cplex(env);
cplex.importModel(model);
// create and start all worker solvers
Worker worker[N];
for (int i = 0; i < N; ++i) {
// setup transport, argc, argv according to chosen transport
worker[i] = new Worker(env, model, i, transport, argc, argv);
}
39

40. IBM Software Group
© 2013 IBM Corporation
Simple Example: Distributed Concurrent MIP
 Worker class (simplified)
class Worker {
IloCplex cplex;
IloCplex::SolveHandle handle;
public:
Worker(IloEnv env, IloModel model, int idx,
const char *transport, int argc, const char *argv)
{
cplex = IloCplex(model, transport, argc, argv);
cplex.setParam(IloCplex::RandomSeed, idx);
handle = cplex.solve(true);
}
40
Exploit
Performance
variability

41. IBM Software Group
© 2013 IBM Corporation
Simple Example: Distributed Concurrent MIP
 Master (continued…)
distributedConcurrentMIPopt(const char* modelfile)
{
// create and start all worker solvers
// wait for first worker to terminate and kill others
int winner = -1;
while (winner < 0) {
for (int i = 0; i < N; ++i) {
if ( !worker[i]->test() ) {
winner = i;
for (int j = 0; j < N; ++j) {
if (i != j)
delete worker[j];
}
break;
}
}
}
// Solution can be found in winning worker
bool success = worker[winner]->join();
}
41

42. IBM Software Group
© 2013 IBM Corporation
Simple Example: Distributed Concurrent MIP
 Worker class (continued…)
class Worker {
IloCplex cplex;
IloCplex::SolveHandle handle;
public:
Worker(IloEnv env, const char *modelfile, int idx,
const char *transport, int argc, const char *argv;
bool test() { return (handle ? handle.test() : false); }
~Worker() {
if ( handle ) {
handle.kill();
handle.join();
}
cplex.end();
}
bool join() {
bool status = (handle ? handle.joinSolve() : false);
handle = NULL;
return status;
}
}
42

43. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Cooperate rather than Compete
– Primal bound
• Best solution found so far
– Dual bound
• Proven bound for possible solution values
– Optimal solution
• Primal bound == Dual bound
 Idea
– Parameterize some workers to focus on primal bound
– Parameterize some workers to focus on dual bound
– Stop when best known primal and dual bound match
43

44. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Configure and launch primal and dual worker(s)
44
cplex
cplex
primal.solve(true)
dual.solve(true)
Primal worker
Dual worker
Master
MIP
Params
solve(true)

45. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Monitor primal dual bounds from workers
45
cplex
cplex
primal.solve(true)
dual.solve(true)
Primal worker
Dual worker
Master
Worker
primal/dual
bound
Best primal
?
Best dual

46. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Monitor primal dual bounds from workers
 Sending bounds implemented as user callback on remote
– Installed with user function from .so on remote computers
46
cplex
cplex
primal.solve(true)
dual.solve(true)
Primal worker
Dual worker
Master
Worker
primal/dual
bound
User callback
User callback
user
function
from
.so
user
function
from
.so
Best primal
?
Best dual

47. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Monitor primal dual bounds from workers
 Receiving bounds implemented as Info Handlers
– Callbacks on worker handles process incoming messages
47
cplex
cplex
primal.solve(true)
dual.solve(true)
Primal worker
Dual worker
Master
Worker
primal/dual
bound
User callback
User callback
user
function
from
.so
user
function
from
.so
Best primal
?
Best dual
InfoHandler
InfoHandler

48. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Terminate when best dual == best primal bound
48
cplex
cplex
primal.solve(true)
dual.solve(true)
Primal worker
Dual worker
Master
Kill
User callback
User callback
user
function
from
.so
user
function
from
.so
Best primal
==
Best dual
InfoHandler
InfoHandler

49. IBM Software Group
© 2013 IBM Corporation
Advanced Example: Distributed Parallel MIP
 Requires monitoring of primal/dual bounds by master in
order to stop progress ASAP
 Advanced functionality: user functions
– Call C user function available in shared library on workers
– Used to install info callback that sends bound information to
master using CPXXsendinfodouble()
 Advanced functionality: IloCplex::RemoteInfoHandler
– Reacts to info sent from callback via function
CPXXsendinfodouble()
 Master monitors best primal and dual bound reported by
info handlers of workers to stop all workers
– See iloparmipopt.cpp
49

50. IBM Software Group
© 2013 IBM Corporation
Performance experiments with parmipopt example
 User reported performance of his adapted parmipopt
example on unit commitment problems
50

51. IBM Software Group
© 2013 IBM Corporation
CPLEX remote object - summary
 Synchronous remote function calls to transparently use
remote hardware for optimization tasks in your program
 Asynchronous remote function calls support
master/worker parallel programming paradigm
– Handle Objects to test/join/kill asynchronous call and collect
return values
– User functions (C only) to extend remote functionality
– InfoHandler Objects to handle data sent during
asynchronous optimization
51

52. IBM Software Group
© 2013 IBM Corporation
CPLEX 12.5.1
Performance Improvements
52

53. IBM Software Group
© 2013 IBM Corporation
CPLEX 12.5.1 performance
 CPLEX Performance Improvements
– MILP
• Lift-and-Project Cuts
 Parameter CPX_PARAM_LANDPCUTS
• Parallel cut loop
• Probing
 Implied Literals Detection and Probing
 Orbital Probing
 Lifting after Probing
 Parallel Probing
• Previously unsolved MIPLIB 2010 problems ns111636 and tw-
myciel4 have been solved with CPLEX 12.5.1
– MIQP
• New presolve reduction for linearizing products xy for binary
variables x and y
53

54. IBM Software Group
© 2013 IBM Corporation
 Unbiased performance evolution (internal testset)
MIP Performance Improvements
54
0
200
400
600
800
1000
1200
1400
6.0
6.5
7.0
8.0
9.0
10.0
11.0
12.1
12.2
12.3
12.4
12.5
12.5.1
numberoftimeouts
0
50
100
150
200
250
totalspeedup

55. IBM Software Group
© 2013 IBM Corporation
 Unbiased performance evolution (internal testset)
MIP Performance Improvements
55
0
200
400
600
800
1000
1200
1400
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
numberoftimeouts
0
50
100
150
200
250
totalspeedup