Sudoku solving with CVXOPT - London Python Dojo

1. Sudoku solving with CVXOPT
Sudoku solving with CVXOPT
Ben Moran
http://transnite.wordpress.com
09 December 2009

2. Sudoku solving with CVXOPT
Background
1 Background
2 Sudoku as matrix and vectors
3 Sparsity and solving
4 Conclusion

3. Sudoku solving with CVXOPT
Background

4. Sudoku solving with CVXOPT
Background
Based on. . .
a paper by Babu, Pelckmans, Stoica:
Linear Systems, Sparse Solutions, and Sudoku 1
Solve Sudoku by turning it into a linear programming problem,
inspired by new signal processing paradigm: compressed
sensing 2
I thought it was interesting and reimplemented it in Python
with CVXOPT
1
http://www.it.uu.se/katalog/praba420/Sudoku.pdf
2
http://nuit-blanche.blogspot.com/

5. Sudoku solving with CVXOPT
Background
Why?
Ben Laurie - Sudoku is a denial of service attack on human
intellect
It's a solved problem!

6. Sudoku solving with CVXOPT
Background
Existing solutions
Knuth's Dancing Links (DLX) 3
Norvig's constraint propagation (also Python) 4
Sinkhorn method 5
And many others
3
http://en.wikipedia.org/wiki/Dancing_Links
4
http://norvig.com/sudoku.html
5
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=4802290arnumber=4804

7. Sudoku solving with CVXOPT
Background
But. . .
This is still an interesting way into the brand new eld of
compressed sensing
And what have Sudoku got to do with. . .
signal processing
sensor networks
single-pixel cameras
satellite imaging?
Our solution will be interesting, as the solver won't know the
rules of Sudoku at all!

8. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
What are linear systems?
Matrix - Vector multiplication
1 3 2 11
=
4 −1 3 5
Ax =b
In Python:
import numpy
numpy . m a t r i x ( ' 1 3 ; 4 −1 ' ) ∗
numpy . m a t r i x ( ' 2 ; 3 ' )
matrix ( [ [ 1 1 ] ,
[ 5]])

9. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
Solving a linear system
Given
1 3 11
A = and b = , what is x?
4 −1 5
In Python:
import numpy
numpy . l i n a l g . s o l v e ( numpy . m a t r i x ( ' 1 3 ; 4 −1 ' ) ,
numpy . m a t r i x ( ' 1 1 ; 5 ' ) )
matrix ( [ [ 2 . ] ,
[ 3.]])

10. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
Sudoku board as an indicator vector
We can turn a 9x9 Sudoku board into a single vector with
9x9x9 = 729 elements
Each 9 entries corresponds to one cell, most are zero
Put a 1 in the rst place for 1, in the second place for 2. . .
p = Problem ( 1 2 . . ,N=2)
numpy . m a t r i x ( p . t o _ i n d i c a t o r _ v e c t o r ( ) , ' i ' )
matrix ( [ [ 1 ] ,
[ 0 ],
[ 0 ],
[ 1 ],
[ 0 ],
[ 0 ],
[ 0 ],
[ 0 ]])

11. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
Sudoku rules as a matrix system
Now we can set up a special matrix to enforce the rules of
Sudoku
It will have one row for each constraint, and 9x9x9 = 729
columns
When you multiply this with the indicator vector, you
get all 1's if the board is valid
Here are some of the rules for 2x2 Sudoku:
p = Problem ( 1 2 . . ,N=2)
numpy . m a t r i x ( p . m a t r i x ( ) )
matrix ( [ [ 1 , 1 , 0 , 0 , 0 , 0, 0, 0.] ,
[ 0, 0, 1, 1, 0, 0, 0, 0.] ,
[ 0, 0, 0, 0, 1, 1, 0, 0.] ,
[ 0, 0, 0, 0, 0, 0, 1, 1.] ,
[ 1, 0, 1, 0, 0, 0, 0, 0.] ,
...
[ 0, 0, 0, 1, 0, 0, 0, 0.]])

12. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
Rules for 2x2 Sudoku
1 2
_ _

13. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
_ _ _ 1 5 _ _ 7 _
 
 1 _ 6 _ _ _ 8 2 _ 
3 _ _ 8 6 _ _ 4 _
 
 
9 _ _ 4 _ _ 5 6 7
 
 
_ _ 4 7 _ 8 3 _ _
 
 
7 3 2 _ _ 6 _ _ 4
 
 
_ 4 _ _ 8 1 _ _ 9
 
 
 
 _ 1 7 _ _ _ 2 _ 8 
_ 5 _ _ 3 7 _ _ _
Figure: Full size 9x9 board

14. Sudoku solving with CVXOPT
Sudoku as matrix and vectors
Figure: Full size 9x9 rules matrix

15. Sudoku solving with CVXOPT
Sparsity and solving
What do we gain by writing Sudoku like that?
We can check solutions with a matrix-vector multiply
Turn the proposed solution into an indicator vector
Multiply it with the the rules matrix
If the solution is valid, the answer will be all ones
Solve Ax = [1 1 1 1 1] to nd the answer?
. . . No - there is an innite number of answers, most won't
give valid boards
Need something else to nd our true correct answer

16. Sudoku solving with CVXOPT
Sparsity and solving
Sparsity
The true solution if it exists will be the sparsest
(Sparse vector == mostly zeros)
0 3
   
 1   1 
0 0.1
   
   
Sparse  0  vs non-sparse  4
   

0 −2
   
   
1 1
   
   
0 .5
That still doesn't help. . . no ecient method to solve that
directly

17. Sudoku solving with CVXOPT
Sparsity and solving
Dierent kinds of distance: L-2 norm, Pythagoras
You measure the length of a vector with a norm
This is an L-p norm (for a 3d vector):
(|x1 |p + |x2 |p + |x3 |p )1/p
If you plug in p=2, you get normal Euclidean distance
(Pythagoras' theorem):
(|x1 |2 + |x2 |2 + |x3 |2 )
We can choose the solution with the smallest L2-norm. . .

18. Sudoku solving with CVXOPT
Sparsity and solving
Dierent kinds of distance: L-2 norm, Pythagoras
But we don't get a valid answer
# Least squares s o l u t i o n
M = numpy . m a t r i x ( s a m p l e ( ) . m a t r i x ( ) )
o n e s =numpy . o n e s ( (M. s h a p e [ 0 ] , 1 ) )
v = numpy . l i n a l g . p i n v (M) ∗ o n e s
(M∗ v ) [ : 4 ]
matrix ( [ [ 1 . ] ,
[ 1.] ,
[ 1.] ,
[ 1.]])
v [ : 4 ]
matrix ([[ −0.0735114 ] ,
[ 0.24178992] ,
[ 0.00310795] ,
[ 0.28139045]])

19. Sudoku solving with CVXOPT
Sparsity and solving
Dierent kinds of distance: L-0 norm, sparsity
Plug in p=0 and you get L0, sparsity (number of non-zeros):
|x1 |0 + |x2 |0 + |x3 |0
The right solution will have the lowest L0 norm
But we've no way of nding it, besides solving the Sudoku!

20. Sudoku solving with CVXOPT
Sparsity and solving
Dierent kinds of distance: L-1 norm
With p=1 you get the absolute sum, taxicab distance
|x1 |1 + |x2 |1 + |x3 |1

21. Sudoku solving with CVXOPT
Sparsity and solving
Why is L-1 important?
There are ecient methods of solving it (linear programming)
And for many matrices, the minimum L1 norm solution turns
out to also minimize L0
(Smallest L1 then nds the sparsest solution)
This insight is central to the compressed sensing revolution

22. Sudoku solving with CVXOPT
Sparsity and solving
Python - CVXOPT
Now we can install python-cvxopt
6 and solve the problem!
CVXOPT is a GPL library for optimization
linear programming
Other kinds of convex optimization
We can use it to nd the solution x of Ax=b, with the smallest
L1-norm
The result comes out as a binary vector of 0s and 1s (without
being constrained to do so!)
6
http://abel.ee.ucla.edu/cvxopt/

23. Sudoku solving with CVXOPT
Sparsity and solving
Python - CVXOPT
After all that work to set up the question, the answer is one
line!
h e l p c v x o p t . s o l v e r s . l p
Help on f u n c t i o n l p i n module c v x o p t . c o n e p r o g :
l p ( c , G , h , A=None , b=None , s o l v e r=None , . . . )
S o l v e s a p a i r o f p r i m a l and d u a l LPs
minimize c ' ∗x
s u b j e c t t o G∗ x + s = h
A∗ x = b
s = 0
...
# S o l v e w i t h CVXOPT
c v x o p t . s o l v e r s . l p ( c0 , G , hh )

24. Sudoku solving with CVXOPT
Conclusion
Caveats
This won't solve every Sudoku - it tends to solve the easier
ones
It fails when the L1 minimum doesn't nd the L0 minimum
There are other methods, from the compressed sensing
literature, like iterative reweighting, that solve more
For Sudoku, which puzzles are hard why is still an open
problem at the frontiers of research

25. Sudoku solving with CVXOPT
Conclusion
Compressed sensing
Very many important, real world signals are sparse: images,
sounds, gene expressions
Instead of capturing the raw signal and compressing. . .
. . . compressed sensing means you can capture them
pre-compressed, with fewer measurements
(below the Nyquist rate: 44.1kHz CDs to play back 22kHz
frequencies)
Less hardware needed will mean better images or cheaper kit
Single-pixel camera 7
7
http://dsp.rice.edu/cscamera

26. Sudoku solving with CVXOPT
Conclusion
Code, links and slides
Code is at http://github.com/benmoran/L1-Sudoku/
Sporadic blog: http://transnite.wordpress.com
Find CVXOPT at
http://abel.ee.ucla.edu/cvxopt/examples/index.html
And the nice CVXMOD wrapper at http://cvxmod.net