2. Course programme
9.30-10.00 Introduction (Andrew Blake)
10.00-11.00 Discrete Models in Computer Vision (Carsten Rother)
15min Coffee break
11.15-12.30 Message Passing: DP, TRW, LP relaxation (Pawan Kumar)
12.30-13.00 Quadratic pseudo-boolean optimization (Pushmeet Kohli)
1 hour Lunch break
14:00-15.00 Transformation and move-making methods (Pushmeet Kohli)
15:00-15.30 Speed and Efficiency (Pushmeet Kohli)
15min Coffee break
15:45-16.15 Comparison of Methods (Carsten Rother)
16:30-17.30 Recent Advances: Dual-decomposition, higher-order, etc.
(Carsten Rother + Pawan Kumar)
All online material will be online (after conference):
http://research.microsoft.com/en-us/um/cambridge/projects/tutorial/
3. E(x) = ∑ fi (xi) + ∑ gij (xi,xj) + ∑ hc(xc)
i ij c
Unary Pairwise Higher Order
∑ ci xi + ∑ dij |xi-xj| E: {0,1}n → R
i i,j n = number of pixels
Image Segmentation
4. n = Number of Variables
Segmentation Energy
NP-Hard
MAXCUT
CSP Pair-wise
O(n3)
Submodular
Functions
Tree
Structured O(n6)
Space of Problems
7. Pseudo-boolean function f{0,1}n ℝ is submodular if
f(A) + f(B) f(A˅B) + f(A˄B) for all A,B ϵ {0,1}n
(OR) (AND)
Example: n = 2, A = [1,0] , B = [0,1]
f([1,0]) + f([0,1]) f([1,1]) + f([0,0])
Property : Sum of submodular functions is submodular
Binary Image Segmentation Energy is submodular
E(x) = ∑ ci xi + ∑ dij |xi-xj|
i i,j
8. Discrete Analogues of Concave Functions
[Lovasz, ’83]
Widely applied in Operations Research
Applications in Machine Learning
MAP Inference in Markov Random Fields
Clustering [Narasimhan , Jojic, & Bilmes, NIPS 2005]
Structure Learning [Narasimhan & Bilmes, NIPS 2006]
Maximizing the spread of influence through a social
network [Kempe, Kleinberg & Tardos, KDD 2003]
9. Polynomial time algorithms
Ellipsoid Algorithm: [Grotschel, Lovasz & Schrijver ‘81]
First strongly polynomial algorithm: [Iwata et al. ’00] [A. Schrijver ’00]
Current Best: O(n5 Q + n6) [Q is function evaluation time] [Orlin ‘07]
Symmetric functions: E(x) = E(1-x)
Can be minimized in O(n3)
Minimizing Pairwise submodular functions
Can be transformed to st-mincut/max-flow [Hammer , 1965]
Very low empirical running time ~ O(n)
E(X) = ∑ fi (xi) + ∑ gij (xi,xj)
i ij
10. Source
2 9 Graph (V, E, C)
2 Vertices V = {v1, v2 ... vn}
v1 v2 Edges E = {(v1, v2) ....}
1 Costs C = {c(1, 2) ....}
5 4
Sink
11. What is a st-cut?
Source
2 9
2
v1 v2
1
5 4
Sink
12. What is a st-cut?
An st-cut (S,T) divides the nodes
between source and sink.
Source
2 9 What is the cost of a st-cut?
2 Sum of cost of all edges going
v1 v2 from S to T
1
5 4
Sink
5 + 1 + 9 = 15
13. What is a st-cut?
An st-cut (S,T) divides the nodes
between source and sink.
Source
2 9 What is the cost of a st-cut?
2 Sum of cost of all edges going
v1 v2 from S to T
1
5 4
What is the st-mincut?
Sink st-cut with the
minimum cost
2 + 2 + 4 = 8
14. Construct a graph such that:
1. Any st-cut corresponds to an assignment of x
2. The cost of the cut is equal to the energy of x : E(x)
S st-mincut
E(x)
T
Solution
[Hammer, 1965] [Kolmogorov and Zabih, 2002]
15. E(x) = ∑ θi (xi) + ∑ θij (xi,xj)
i i,j
For all ij θij(0,1) + θij (1,0) θij (0,0) + θij (1,1)
Equivalent (transformable)
E(x) = ∑ ci xi + ∑ cij xi(1-xj) cij≥0
i i,j
25. Solve the dual maximum flow problem
Compute the maximum flow between
Source
Source and Sink s.t.
2 9
Edges: Flow < Capacity
1
Nodes: Flow in = Flow out
v1 v2
2
5 4
Min-cutMax-flow Theorem
Sink In every network, the maximum flow
equals the cost of the st-mincut
Assuming non-negative capacity
27. Flow = 0
Augmenting Path Based
Algorithms
Source
2 9 1. Find path from source to sink
with positive capacity
1
v1 v2
2
5 4
Sink
28. Flow = 0 + 2
Augmenting Path Based
Algorithms
Source
2-2 9 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
4
5-2
Sink
29. Flow = 2
Augmenting Path Based
Algorithms
Source
0 9 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
3 4
Sink
30. Flow = 2
Augmenting Path Based
Algorithms
Source
0 9 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
3 4
3. Repeat until no path can be
Sink
found
31. Flow = 2
Augmenting Path Based
Algorithms
Source
0 9 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
3 4
3. Repeat until no path can be
Sink
found
32. Flow = 2 + 4
Augmenting Path Based
Algorithms
Source
0 5 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
3 0
3. Repeat until no path can be
Sink
found
33. Flow = 6
Augmenting Path Based
Algorithms
Source
0 5 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
3 0
3. Repeat until no path can be
Sink
found
34. Flow = 6
Augmenting Path Based
Algorithms
Source
0 5 1. Find path from source to sink
with positive capacity
1
v1 v2
2. Push maximum possible flow
2 through this path
3 0
3. Repeat until no path can be
Sink
found
35. Flow = 6 + 2
Augmenting Path Based
Algorithms
Source
0 3 1. Find path from source to sink
with positive capacity
1+2
v1 v2
2. Push maximum possible flow
2-2 through this path
1 0
3. Repeat until no path can be
Sink
found
36. Flow = 8
Augmenting Path Based
Algorithms
Source
0 4 1. Find path from source to sink
with positive capacity
3
v1 v2
2. Push maximum possible flow
0 through this path
2 0
3. Repeat until no path can be
Sink
found
37. Flow = 8
Augmenting Path Based
Algorithms
Source
0 4 1. Find path from source to sink
with positive capacity
3
v1 v2
2. Push maximum possible flow
0 through this path
2 0
3. Repeat until no path can be
Sink
found
56. n: #nodes
Ford Fulkerson: Choose any augmenting path m: #edges
Source
999 1000
0
a1 a2
1
1000 999
Sink
We will have to perform 2000 augmentations!
Worst case complexity: O (m x Total_Flow)
(Pseudo-polynomial bound: depends on flow)
58. Specialized algorithms for vision problems
Grid graphs
Low connectivity (m ~ O(n))
Dual search tree augmenting path algorithm
[Boykov and Kolmogorov PAMI 2004]
• Finds approximate shortest augmenting paths efficiently
• High worst-case time complexity
• Empirically outperforms other algorithms on vision problems
Efficient code available on
the web
http://www.adastral.ucl.ac.uk/~vladkolm/software.html
59. E(x) = ∑ ci xi + ∑ dij |xi-xj| E: {0,1}n → R
0 → fg
i i,j
1 → bg
n = number of
pixels
x* = arg min E(x)
x
How to minimize E(x)?
60. Graph *g;
For all pixels p
Source (0)
/* Add a node to the graph */
nodeID(p) = g->add_node();
/* Set cost of terminal edges */
set_weights(nodeID(p), fgCost(p), bgCost(p));
end
for all adjacent pixels p,q
add_weights(nodeID(p), nodeID(q), cost(p,q));
end
g->compute_maxflow(); Sink (1)
label_p = g->is_connected_to_source(nodeID(p));
// is the label of pixel p (0 or 1)
61. Graph *g;
For all pixels p
Source (0)
/* Add a node to the graph */
nodeID(p) = g->add_node();
bgCost(a1) bgCost(a2)
/* Set cost of terminal edges */
set_weights(nodeID(p), fgCost(p), bgCost(p));
a1 a2
end
for all adjacent pixels p,q
add_weights(nodeID(p), nodeID(q), cost(p,q)); fgCost(a1) fgCost(a2)
end
g->compute_maxflow(); Sink (1)
label_p = g->is_connected_to_source(nodeID(p));
// is the label of pixel p (0 or 1)
62. Graph *g;
For all pixels p
Source (0)
/* Add a node to the graph */
nodeID(p) = g->add_node();
bgCost(a1) bgCost(a2)
/* Set cost of terminal edges */
set_weights(nodeID(p), fgCost(p), bgCost(p)); cost(p,q)
a1 a2
end
for all adjacent pixels p,q
add_weights(nodeID(p), nodeID(q), cost(p,q)); fgCost(a1) fgCost(a2)
end
g->compute_maxflow(); Sink (1)
label_p = g->is_connected_to_source(nodeID(p));
// is the label of pixel p (0 or 1)
63. Graph *g;
For all pixels p
Source (0)
/* Add a node to the graph */
nodeID(p) = g->add_node();
bgCost(a1) bgCost(a2)
/* Set cost of terminal edges */
set_weights(nodeID(p), fgCost(p), bgCost(p)); cost(p,q)
a1 a2
end
for all adjacent pixels p,q
add_weights(nodeID(p), nodeID(q), cost(p,q)); fgCost(a1) fgCost(a2)
end
g->compute_maxflow(); Sink (1)
label_p = g->is_connected_to_source(nodeID(p));
// is the label of pixel p (0 or 1) a1 = bg a2 = fg
65. Advances in Markov Random Fields for Computer Vision
MIT Press, summer 2010
Topics of this course and much, much more
Contributors: usual suspects – lecturers on this course + Boykov,
Kolmogorov, Weiss, Freeman, ....
one for the office and one for home
www.research.microsoft.com/vision/MRFbook
69. E(x) = ∑ θi (xi) + ∑ θij (xi,xj)
i i,j
θij(0,1) + θij (1,0) ≤θ ij (0,0) + θij (1,1) for some ij
Minimizing general non-submodular functions is NP-
hard.
Commonly used method is to solve a relaxation of
the problem
[Boros and Hammer, ‘02]
75. QPBO: 0 ? ? ? ? ?
p q r s t
Probe Node p:
0 0 0 0 ? ? 0 1 0 1 0 ?
p q r s t p q r s t
What can we say about variables?
•r -> is always 0
•s -> is always equal to q
•t -> is 0 when q = 1
76. Probe nodes in an order until energy
unchanged
Simplified energy preserves global optimality
and (sometimes) gives the global minimum
Result depends slightly on the order
79. Need for a human
like segmentation
colour appearance
based Segmentation
Image Segmentation
Result
[Kumar et al, 05] [Kohli et al, 06,08]
80. x – binary image segmentation (xi ∊ {0,1})
ω – non-local parameter (lives in some large set Ω)
E(x,ω) = C(ω) + ∑ θi (ω, xi) + ∑ θij (ω,xi,xj)
i i,j
unary pairwise
constant
potentials potentials
≥0
ω θi (ω, xi)
Pose Shape Prior
Stickman Rough Shape
Model Prior
[Kohli et al, 06,08]
81. x – binary image segmentation (xi ∊ {0,1})
ω – non-local parameter (lives in some large set Ω)
E(x,ω) = C(ω) + ∑ θi (ω, xi) + ∑ θij (ω,xi,xj)
i i,j
unary pairwise
constant
potentials potentials
≥0
ω
Template
Position
Scale
Orientation
[Kohli et al, 06,08] [Lempitsky et al, 08]
82. x – binary image segmentation (xi ∊ {0,1})
ω – non-local parameter (lives in some large set Ω)
E(x,ω) = C(ω) + ∑ θi (ω, xi) + ∑ θij (ω,xi,xj)
i i,j
unary pairwise
constant
potentials potentials
≥0
{x*,ω*} = arg min E(x,ω)
x,ω
• Standard “graph cut” energy if ω is fixed
[Kohli et al, 06,08] [Lempitsky et al, 08]
83. Local Method: Gradient Descent over ω
ω * = arg min min E (x,ω) Submodular
ω x
ω*
[Kohli et al, 06,08]
84. Local Method: Gradient Descent over ω
ω * = arg min min E (x,ω) Submodular
ω x
E (x,ω1)
Dynamic Graph
Similar Energy Cuts
Functions
15- 20 time
E (x,ω2) speedup!
[Kohli et al, 06,08]
89. 30,000,000 shapes
Exhaustive search: 30,000,000 mincuts
Branch-and-Mincut: 12,000 mincuts
Speed-up: 2500 times
(30 seconds per 312x272 image)
[Lempitsky et al, 08]
90. Left ventricle epicardium tracking (work in progress)
Original sequence Branch & Bound segmentation No shape prior
Shape prior from other sequences
5,200,000 templates
≈20 seconds per frame
Speed-up 1150
Data courtesy: Dr Harald Becher, Department of Cardiovascular Medicine, University of Oxford
[Lempitsky et al, 08]
92. Pairwise functions have limited expressive power
Inability to incorporate region based likelihoods and priors
Field of Experts Model
[Roth & Black CVPR 2005 ]
[Potetz, CVPR 2007]
Minimize Curvature
[Woodford et al. CVPR 2008 ]
Other Examples:
[Rother, Kolmogorov, Minka & Blake, CVPR 2006]
[Komodakis and Paragios, CVPR 2009]
[Rother, Kohli, Feng, Jia, CVPR 2009]
[Ishikawa, CVPR 2009]
And many others ...
93. n = number of pixels
E: {0,1}n → R
0 →fg, 1→bg
E(X) = ∑ ci xi + ∑ dij |xi-xj|
i i,j
Image Unary Cost Segmentation
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
94. [Kohli et al. ‘07]
Patch Dictionary
(Tree)
h(Xp) = { C1 if xi = 0, i ϵ p
Cmax otherwise
p
Cmax C1
95. n = number of pixels
E: {0,1}n → R
0 →fg, 1→bg
E(X) = ∑ ci xi + ∑ dij |xi-xj| + ∑ hp (Xp)
i i,j p
h(Xp) = { C1 if xi = 0, i ϵ p
Cmax otherwise
p
[Kohli et al. ‘07]
96. n = number of pixels
E: {0,1}n → R
0 →fg, 1→bg
E(X) = ∑ ci xi + ∑ dij |xi-xj| + ∑ hp (Xp)
i i,j p
Image Pairwise Segmentation Final Segmentation
[Kohli et al. ‘07]
97. Exact
Higher Order Transformation Pairwise Submodular
Submodular
Function
Functions
?
Billionnet and M. Minoux [DAM 1985] st-mincut
Kolmogorov & Zabih [PAMI 2004] S
Freedman & Drineas [CVPR2005]
Kohli Kumar Torr [CVPR2007, PAMI 2008]
Kohli Ladicky Torr [CVPR 2008, IJCV 2009]
Ramalingam Kohli Alahari Torr [CVPR 2008]
Zivny et al. [CP 2008] T
98. Higher Order Pairwise
Function Submodular
Function
Identified transformable families of higher order function s.t.
1. Constant or polynomial number of auxiliary variables (a) added
2. All pairwise functions (g) are submodular
99. Example: H (X) = F ( ∑ xi )
concave
H (X)
0 ∑ xi
[Kohli et al. ‘08]
100. Simple Example using Auxiliary variables
f(x) = { 0 if all xi = 0
C1 otherwise
x ϵ L = {0,1}n
min f(x) = min C1a + C1 ∑ ā xi
x x,a ϵ {0,1}
Higher Order Quadratic Submodular
Submodular Function Function
∑xi < 1 a=0 (ā=1) f(x) = 0
∑xi ≥ 1 a=1 (ā=0) f(x) = C1
101. min f(x) = min C1a + C1 ∑ ā xi
x x,a ϵ {0,1}
Higher Order Quadratic Submodular
Submodular Function Function
C1∑xi
C1
1 2 3
∑xi
102. min f(x) = min C1a + C1 ∑ ā xi
x x,a ϵ {0,1}
Higher Order Quadratic Submodular
Submodular Function Function
C1∑xi
a=0 a=1
Lower envelop
of concave
C1 functions is
concave
1 2 3
∑xi
103. min f(x)
x
= min f1 (x)a + f2(x)ā
x,a ϵ {0,1}
Higher Order Quadratic Submodular
Submodular Function Function
f2(x)
f1(x) Lower envelop
of concave
functions is
concave
1 2 3
∑xi
104. min f(x)
x
= min f1 (x)a + f2(x)ā
x,a ϵ {0,1}
Higher Order Quadratic Submodular
Submodular Function Function
f2(x)
a=0 a=1
f1(x) Lower envelop
of concave
functions is
concave
1 2 3
∑xi
105. Transforming Potentials with 3 variables
[Woodford, Fitzgibbon, Reid, Torr, CVPR 2008]
Transforming general “sparse” higher order functions
[Rother, Kohli, Feng, Jia, CVPR 2009]
[Ishikawa, CVPR 2009]
[Komodakis and Paragios, CVPR 2009]
106. Training
Image
Pairwise
Energy
P(x)
Test Image Test Image Minimized using Result
(60% Noise) st-mincut or
max-product
message passing
107. Training
Image
Higher Order Structure
not Preserved
Pairwise
Energy
P(x)
Test Image Test Image Minimized using Result
(60% Noise) st-mincut or
max-product
message passing
108. Minimize: E(X) = P(X) + ∑ hc (Xc)
c
Where: hc: {0,1}|c| → R
Higher Order Function (|c| = 10x10 = 100)
Assigns cost to 2100 possible labellings!
Exploit function structure to transform
it to a Pairwise function
p1 p2 p 3
109. Training Learned
Image Patterns
Test Image Test Image Pairwise Higher-Order
(60% Noise) Result Result
[Joint work with Carsten Rother ]
111. Min
y
E(y) = ∑ fi (yi) + ∑ gij (yi,yj)
i i,j
y ϵ Labels L = {l1, l2, … , lk}
Exact Transformation to QPBF
[Roy and Cox ’98] [Ishikawa ’03] [Schlesinger & Flach ’06]
[Ramalingam, Alahari, Kohli, and Torr ’08]
Move making algorithms
112. So what is the problem?
Em (y1,y2, ..., yn) Eb (x1,x2, ..., xm)
yi ϵ L = {l1, l2, … , lk} xi ϵ L = {0,1}
Multi-label Problem Binary label Problem
such that:
Let Y and X be the set of feasible solutions, then
1. One-One encoding function T:X->Y
2. arg min Em(y) = T(arg min Eb(x))
113. • Popular encoding scheme
[Roy and Cox ’98, Ishikawa ’03, Schlesinger & Flach ’06]
# Nodes = n * k
# Pairwise = m * k2
114. • Popular encoding scheme
[Roy and Cox ’98, Ishikawa ’03, Schlesinger & Flach ’06]
Ishikawa’s result:
E(y) = ∑ θi (yi) + ∑ θij (yi,yj)
i i,j
y ϵ Labels L = {l1, l2, … , lk}
Convex
θij (yi,yj) = g(|yi-yj|) Function
# Nodes = n * k
# Pairwise = m * k2
g(|yi-yj|)
|yi-yj|
115. • Popular encoding scheme
[Roy and Cox ’98, Ishikawa ’03, Schlesinger & Flach ’06]
Schlesinger & Flach ’06:
E(y) = ∑ θi (yi) + ∑ θij (yi,yj)
i i,j
y ϵ Labels L = {l1, l2, … , lk}
θij(li+1,lj) + θij (li,lj+1) θij (li,lj) + θij (li+1,lj+1)
# Nodes = n * k
lj +1
# Pairwise = m * k2 li +1
lj
li
116. Image MAP Scanline
Solution algorithm
[Roy and Cox, 98]
117. θij (yi,yj) = g(|yi-yj|)
Applicability
Cannot handle truncated costs
(non-robust)
|yi-yj|
discontinuity preserving potentials
Computational Cost Blake&Zisserman’83,87
Very high computational cost
Problem size = |Variables| x |Labels|
Gray level image denoising (1 Mpixel image)
(~2.5 x 108 graph nodes)
118. Other “less known” algorithms
Unary Pair-wise Complexity
Potentials Potentials
Ishikawa Arbitrary Convex and T(nk, mk2)
Transformation Symmetric
[03]
Schlesinger Arbitrary Submodular T(nk, mk2)
Transformation
[06]
Hochbaum Linear Convex and T(n, m) + n log k
[01] Symmetric
Hochbaum Convex Convex and O(mn log n log nk)
[01] Symmetric
T(a,b) = complexity of maxflow with a nodes and b edges
119. Min
y
E(y) = ∑ fi (yi) + ∑ gij (yi,yj)
i i,j
y ϵ Labels L = {l1, l2, … , lk}
Exact Transformation to QPBF
Move making algorithms
[Boykov , Veksler and Zabih 2001] [Woodford, Fitzgibbon, Reid, Torr, 2008]
[Lempitsky, Rother, Blake, 2008] [Veksler, 2008] [Kohli, Ladicky, Torr 2008]
121. Current Solution
Search
Neighbourhood
Optimal Move
Energy
Solution Space
122. Current Solution
Search
Neighbourhood
Optimal Move
xc
Key Property
Energy
(t)
Move Space
Solution Space
Bigger move • Better solutions
space • Finding the optimal move hard
123. Minimizing Pairwise Functions
[Boykov Veksler and Zabih, PAMI 2001]
• Series of locally optimal moves
• Each move reduces energy
• Optimal move by minimizing submodular function
Current Solution
Search Neighbourhood
Move Space (t) : 2n
n Number of Variables
L Number of Labels
Space of Solutions (x) : Ln
Extend to minimize Higher order Functions
Kohli et al. ‘07, ‘08, ‘09
124. x = t x1 + (1-t) x2
New Current Second
solution Solution solution
Em(t) = E(t x1 + (1-t) x2)
Minimize over move variables t
For certain x1 and x2, the move energy is sub-modular QPBF
[Boykov , Veksler and Zabih 2001]
125. • Variables labeled α, β can swap their labels
[Boykov , Veksler and Zabih 2001]
126. • Variables labeled α, β can swap their labels
Tree
Ground
Swap Sky, House House
Sky
[Boykov , Veksler and Zabih 2001]
127. • Variables labeled α, β can swap their labels
Move energy is submodular if:
Unary Potentials: Arbitrary
Pairwise potentials: Semi-metric
θij (la,lb) ≥ 0
θij (la,lb) = 0 a = b
Examples: Potts model, Truncated Convex
[Boykov , Veksler and Zabih 2001]
128. • Variables take label a or retain current label
[Boykov , Veksler and Zabih 2001]
[Boykov, Veksler, Zabih]
129. • Variables take label a or retain current label
Tree
Ground
House
Status: Expand Ground
Initialize with Tree
Sky
House Sky
[Boykov , Veksler and Zabih 2001]
[Boykov, Veksler, Zabih]
130. • Variables take label a or retain current label
Semi metric
Move energy is submodular if: +
Unary Potentials: Arbitrary Triangle
Inequality
Pairwise potentials: Metric
θij (la,lb) + θij (lb,lc) ≥ θij (la,lc)
Examples: Potts model, Truncated linear
Cannot solve truncated quadratic
[Boykov , Veksler and Zabih 2001]
[Boykov, Veksler, Zabih]
131. Expansion and Swap can be derived as a primal dual scheme
Get solution of the dual problem which is a lower bound on
the energy of solution
Weak guarantee on the solution E(x) < 2(dmax /dmin) E(x*)
θij (li,lj) = g(|li-lj|)
dmax
dmin
|yi-yj|
[Komodakis et al 05, 07]
132. x = t x1 + (1-t) x2
New First Second
solution solution solution
Minimize over move variables t
Move Type First Second Guarantee
Solution Solution
Expansion Old solution All alpha Metric
Fusion Any solution Any solution
Move functions can be non-submodular!!
133. x = t x1 + (1-t) x2
x1, x2 can be continuous Optical Flow
Example
Solution
from
Method 2 x2
Solution
from F Final
Method 1 x1 x Solution
[Woodford, Fitzgibbon, Reid, Torr, 2008] [Lempitsky, Rother, Blake, 2008]
134. Move variables can be multi-label
x = (t ==1) x1 + (t==2) x2 +… +(t==k) xk
Optimal move found out by using the Ishikawa Transform
Useful for minimizing energies with truncated convex
pairwise potentials
T
θij (yi,yj) = min(|yi-yj|2,T)
θij (yi,yj)
|yi-yj|
[Veksler, 2007]
135. Noisy
Image
Image
Range
Expansion
Moves
Move
[Veksler, 2008]
142. minimize
Frame 1 EA SA
Reuse
Computation
differences
between EB* Simpler
A and B
fast
minimize
Frame 2 EB SB 3–100000
time speedup!
Reparametrization
[Kohli & Torr, ICCV05 PAMI07] [Komodakis & Paragios, CVPR07]
143. Original Energy
E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 + ā1a2
Reparametrized Energy
E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2
New Energy
E(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 7a1ā2 + ā1a2
New Reparametrized Energy
E(a1,a2) = 8 + ā1+ 3a2 + 3ā1a2 + 5a1ā2
[Kohli & Torr, ICCV05 PAMI07]& [Komodakis & Paragios, CVPR07]
Kohli Torr (ICCV05, PAMI07)
144. Approximation
Original
algorithm Approximate
Problem
(Large) Solution
(Slow)
Fast partially
optimal
algorithm
[ Alahari Kohli & Torr CVPR ‘08]
145. Approximation
Original
algorithm Approximate
Problem
(Large) Solution
(Slow)
Fast partially
optimal algorithm
[Kovtun ‘03]
Fast partially
[ Kohli et al. ‘09]
optimal
algorithm Reduced Problem Approximation
algorithm
Solved Problem
Approximate
(Global Optima) (Fast) Solution
[ Alahari Kohli & Torr CVPR ‘08]
146. 3- 100
Times Speed up
sky
Tree Reweighted
Original Building
Message Passing Approximate
Problem Airplane
(Large) Solution
(9.89 sec)
Grass
Fast partially
optimal algorithm
[Kovtun ‘03]
Fast partially
[ Kohli et al. ‘09]
optimal Tree Reweighted sky
algorithm sky
Reduced Problem Message Passing Building
Building Approximate
Solved Problem Airplane
Solution
Airplane Total Time
(Global Optima)
(0.30 sec) Grass
Grass
[ Alahari Kohli & Torr CVPR ‘08]
147. Minimization with Complex Higher Order
Functions
Connectivity
Counting Constraints
Hybrid algorithms
Connections between Messages Passing
algorithms BP, TRW, and graph cuts
148. Advances in Markov Random Fields for Computer Vision
MIT Press, summer 2010
Topics of this course and much, much more
Contributors: usual suspects – lecturers on this course + Boykov,
Kolmogorov, Weiss, Freeman, ....
one for the office and one for home
www.research.microsoft.com/vision/MRFbook
