13. Chap. 2. Centroidal Voronoi Tesselation
The classical method:
Lloyd algorithm = gradient descent
Which objective function ?
14. Chap. 2. Centroidal Voronoi Tesselation
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
F=
∫Vor(i)
2
dxxi - x
i
15. Chap. 2. Centroidal Voronoi Tesselation
F=
∫Vor(i)
2
dxxi - x
i
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
Why Lloyd works ? What is the gradient of F ?
16. Chap. 2. Centroidal Voronoi Tesselation
F=
∫Vor(i)
2
dxxi - x
i
=
∫Inf i
2
dxxi - x
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
Why Lloyd works ? What is the gradient of F ?
17. Chap. 2. Centroidal Voronoi Tesselation
F=
∫Vor(i)
2
dxxi - x
i
=
∫Inf i
2
dxxi - x
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
Why Lloyd works ? What is the gradient of F ?
18. Chap. 2. Centroidal Voronoi Tesselation
F|xi = 2 mi (xi - gi) [Iri et.al], [Du et.al]
F=
∫Vor(i)
2
dxxi - x
i
=
∫Inf i
2
dxxi - x
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
Why Lloyd works ? What is the gradient of F ?
19. Chap. 2. Centroidal Voronoi Tesselation
F|xi = 2 mi (xi - gi) [Iri et.al], [Du et.al]
F=
∫Vor(i)
2
dxxi - x
i
=
∫Inf i
2
dxxi - x
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
Why Lloyd works ? What is the gradient of F ?
Volume of Vor(i)
20. Chap. 2. Centroidal Voronoi Tesselation
F|xi = 2 mi (xi - gi) [Iri et.al], [Du et.al]
F=
∫Vor(i)
2
dxxi - x
i
=
∫Inf i
2
dxxi - x
The classical method:
Lloyd algorithm = gradient descent
Which objective function ? This one (quantization noise power)
Why Lloyd works ? What is the gradient of F ?
Volume of Vor(i) Centroid of Vor(i)
21. Chap. 2. Centroidal Voronoi Tesselation
The classical method:
Lloyd algorithm = gradient descent
F=
∫Vor(i)
2
dxxi - x
i
F is smooth (C2) [Liu, Wang, L, Sun, Yan, Lu, Yang 2009]
22. Chap. 2. Centroidal Voronoi Tesselation
The classical method:
Lloyd algorithm = gradient descent
F=
∫Vor(i)
2
dxxi - x
i
F is smooth (C2) [Liu, Wang, L, Sun, Yan, Lu, Yang 2009]
30. Yann Brenier
The polar factorization theorem
(Brenier Transport)
Each time the Laplace operator is used in a PDE, it can be replaced with the
Monte-Ampère operator, and then interesting things occur
31. Euler
Lagrange
The Least Action Principle
Hamilton,
Legendre,
Maupertuis
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
Short summary of the 1st chapter of Landau,Lifshitz Course of Theoretical Physics
32. Euler
Lagrange
The Least Action Principle
Hamilton,
Legendre,
Maupertuis
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
position
33. Euler
Lagrange
The Least Action Principle
Hamilton,
Legendre,
Maupertuis
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
position
speed
34. Euler
Lagrange
The Least Action Principle
Hamilton,
Legendre,
Maupertuis
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
position
speed
time
35. Euler
Lagrange
∫t1
t2
L(x,x,t) dt
The Least Action Principle
Hamilton,
Legendre,
Maupertuis
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
Axiom 2: The movement (time
evolution) of the physical system
minimizes the following integral
36. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
Axiom 2: The movement (time
evolution) of the physical system
minimizes the following integral
37. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists a function L(x,x,t) that describes the state
of a physical system
Axiom 2: The movement (time
evolution) of the physical system
minimizes the following integral
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
t=0
t=1
38. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
39. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
40. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
41. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
Homogeneity of space =>
Preservation of momentum
42. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
Homogeneity of space =>
Preservation of momentum
Isotropy of space =>
Preservation of angular momentum
43. ∫t1
t2
L(x,x,t) dt
The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
Homogeneity of space =>
Preservation of momentum
Isotropy of space =>
Preservation of angular momentum
Preserved quantities
Integrals of Motion
Noeter theorem
44. The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
Homogeneity of space =>
Preservation of momentum
Isotropy of space =>
Preservation of angular momentum
Free particle:
Theorem 3: v = cte (Newton law I)
45. The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
Homogeneity of space =>
Preservation of momentum
Isotropy of space =>
Preservation of angular momentum
Free particle:
Theorem 3: v = cte (Newton law I)
Expression of the Lagrangian:
L = ½ m v2
46. The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Theorem 2:
∂L
∂x
x - L = cte
Homogeneity of time =>
Preservation of energy
Homogeneity of space =>
Preservation of momentum
Isotropy of space =>
Preservation of angular momentum
Free particle:
Theorem 3: v = cte (Newton law I)
Expression of the Lagrangian:
L = ½ m v2
Expression of the Energy:
E = ½ m v2
47. The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Particle in a field:
Expression of the Lagrangian:
L = ½ m v2 – U(x)
Free particle:
Theorem 3: v = cte (Newton law I)
Expression of the Lagrangian:
L = ½ m v2
Expression of the Energy:
E = ½ m v2
48. The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Particle in a field:
Expression of the Lagrangian:
L = ½ m v2 – U(x)
Expression of the Energy:
E = ½ m v2 + U(x)
Free particle:
Theorem 3: v = cte (Newton law I)
Expression of the Lagrangian:
L = ½ m v2
Expression of the Energy:
E = ½ m v2
49. The Least Action Principle
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. change of
Gallileo frame + hom. + isotrop. :
x’
t’ =
x+vt
t
Particle in a field:
Expression of the Lagrangian:
L = ½ m v2 – U(x)
Expression of the Energy:
E = ½ m v2 + U(x)
Theorem 4:
mx = - U (Newton law II)
Free particle:
Theorem 3: v = cte (Newton law I)
Expression of the Lagrangian:
L = ½ m v2
Expression of the Energy:
E = ½ m v2
50. Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. Lorentz change of
frame
x’
t’ =
(x-vt) x γ
(t – vx/c2) x γ
γ = 1 / ( 1 – v2 / c2)
The Least Action Principle
(relativistic setting, just for fun)
√
51. The Least Action Principle
(relativistic setting, just for fun)
Axiom 1: There exists L
Axiom 2: The movement minimizes ∫ L
Theorem 1: (Lagrange equation):
∂L
∂x
d
dt
∂L
∂x
=
Axiom 3:
Invariance w.r.t. Lorentz change of
frame
x’
t’ =
(x-vt) x γ
(t – vx/c2) x γ
Theorem 5:
E = ½ γ m v2 + mc2
γ = 1 / ( 1 – v2 / c2)√
52. The Least Action Principle
(quantum physics setting, just for fun)
56. FluidsLagrange point of view
ρ(x,y,t) nb particles per square
Euler point of view
v(x,y,t) speed of the particle under
grid point (x,y) at time t
57. Fluids
ρ(x,y,t) nb particles per square
Euler point of view
v(x,y,t) speed of the particle under
grid point (x,y) at time t
Q1: how to compute the
acceleration of the particles
from v(x,y,t) ?
dv
dt
=
∂v v. v
∂t
+
Q2: incompressible fluids ?
div(v) = 0
Q3: mass preservation ?
d ρ
dt = - div(ρv)
(Continuity equation)
58. Fluids
Start with Lagrange coordinates:
particle trajectories: X(t,x)
Minimize
Action:
∫t1
t2
1/2
∫Ω
∂X (t,x)
∂t
dxdt
(ρ = cte)
s.t. X satisfies mass preservation
(X is measure-preserving, more on
this later)
2
59. Fluids
Start with Lagrange coordinates:
particle trajectories: X(t,x)
Minimize
Action:
∫t1
t2
1/2
∫Ω
∂X (t,x)
∂t
dxdt
(ρ = cte)
s.t. X satisfies mass preservation
(X is measure-preserving, more on
this later)
Acceleration of the
particle under the grid
2
60. Fluids
Start with Lagrange coordinates:
particle trajectories: X(t,x)
Minimize
Action:
∫t1
t2
1/2
∫Ω
∂X (t,x)
∂t
dxdt
(ρ = cte)
s.t. X satisfies mass preservation
(X is measure-preserving, more on
this later)
The Lagrange multiplier
for the constraint = pressure
2
63. ?
T
Fluids – Benamou Brenier
Minimize
A(ρ,v) =
∫t1
t2
(t2-t1)
∫Ω
ρ(x,t) ||v(t,x)||2
dxdt
s.t. ρ(t1,.) = ρ1 ; ρ(t2,.) = ρ2 ; d ρ
dt
= - div(ρv)
ρ1 ρ2
Minimize C(T) =
∫Ω
|| x – T(x) ||2 dx
s.t. T is measure-preserving
ρ1(x)
64. Part. 3 Optimal Transport – Monge problem
A map T is a transport map between μ and ν if
μ(T-1(B)) = ν(B) for any Borel subset B of Y
(X;μ) (Y;ν)
65. Part. 3 Optimal Transport – Monge problem
A map T is a transport map between μ and ν if
μ(T-1(B)) = ν(B) for any Borel subset B
B
(X;μ) (Y;ν)
66. Part. 3 Optimal Transport – Monge problem
A map T is a transport map between μ and ν if
μ(T-1(B)) = ν(B) for any Borel subset B
B
T-1(B)
(X;μ) (Y;ν)
67. Part. 3 Optimal Transport – Monge problem
A map T is a transport map between μ and ν if
μ(T-1(B)) = ν(B) for any Borel subset B
(or ν = T#μ the pushforward of μ)
(X;μ) (Y;ν)
68. Part. 3 Optimal Transport – Monge problem
Monge problem (1787):
Find a transport map T that minimizes C(T) = ∫X || x – T(x) ||2 dμ(x)
(X;μ) (Y;ν)
69. Part. 3 Optimal Transport – Kantorovich
Monge problem:
Find a transport map T that minimizes C(T) = ∫X || x – T(x) ||2 dμ(x)
Kantorovich problem (1942):
Find a measure γ defined on X x Y
such that ∫x in X dγ(x,y) = dν(y)
and ∫y in Y dγ(x,y) = dμ(x)
that minimizes ∫∫X x Y || x – y ||2 dγ(x,y)
70. Part. 3 Optimal Transport – Kantorovich
Transport plan – example 1/2 : translation of a segment
71. Part. 3 Optimal Transport – Kantorovich
Transport plan – example 1/2 : translation of a segment
73. Part. 3 Optimal Transport – Dual of Kantorovich
Dual formulation of Kantorovich problem:
Find a c-concave function ψ
that maximizes ∫X ψ(x)dμ + ∫Y ψc(y)dν
ψc(y) = inf x [ c(x,y) - ψ(x) ]where: (Legendre transform)
74. Part. 3 Optimal Transport – Dual of Kantorovich
Dual formulation of Kantorovich problem:
Find a c-concave function ψ
that maximizes ∫X ψ(x)dμ + ∫Y ψc(y)dν
ψc(y) = inf x [ c(x,y) - ψ(x) ]where: (Legendre transform)
What about our initial problem ?
T(x) = x – grad ψ(x) = grad (½ x2- ψ(x) )
{{grad ψ(x) with ψ(x) := (½ x2- ψ(x))
[Brenier]
75. Part. 3 Optimal Transport – Dual of Kantorovich
Dual formulation of Kantorovich problem:
Find a c-concave function ψ
that maximizes ∫X ψ(x)dμ + ∫Y ψc(y)dν
ψc(y) = inf x [ c(x,y) - ψ(x) ]where: (Legendre transform)
What about our initial problem ?
T(x) = x – grad ψ(x) = grad (½ x2- ψ(x) )
{{grad ψ(x) with ψ(x) := (½ x2- ψ(x))
When μ and ν have a density u and v, (H ψ(x)). v(grad ψ(x)) = u(x)
Monge-Ampère
equation
for all borel set A, ∫A dμ = ∫T(A) (|JT|) dν = ∫T(A) (H ψ ) dν
[Brenier]
76. Part. 3 Optimal Transport – Dual of Kantorovich
Dual formulation of Kantorovich problem:
Find a c-concave function ψ
that maximizes ∫X ψ(x)dμ + ∫Y ψc(y)dν
ψc(y) = inf x [ c(x,y) - ψ(x) ]where: (Legendre transform)
What about our initial problem ?
T(x) = x – grad ψ(x) = grad (½ x2- ψ(x) )
{{grad ψ(x) with ψ(x) := (½ x2- ψ(x))
When μ and ν have a density u and v, (H ψ(x)). v(grad ψ(x)) = u(x)
Monge-Ampère
equation
for all borel set A, ∫A dμ = ∫T(A) (|JT|) dν = ∫T(A) (H ψ ) dν
[Brenier]
77. Part. 3 Optimal Transport – semi-discrete
∫X ψc (x)dμ + ∫Y ψ(y)dν
Sup
ψ Є ψc
(DMK)
(X;μ) (Y;ν)
84. Part. 3 Optimal Transport
Theorem: (direct consequence of MK duality
alternative proof in [Aurenhammer, Hoffmann, Aronov 98] ):
Given a measure μ with density, a set of points (yj), a set of positive coefficients vj
such that ∑ vj =∫ dμ(x), it is possible to find the weights W = [ψ(y1) ψ(y2) … ψ(ym)]
such that the map TS
W is the unique optimal transport map
between μ and ν = ∑ vj δ(yj)
85. Part. 3 Optimal Transport
Optimal transport Centroidal Voronoi Tesselation
[L - A numerical Algorithm for L2 semi-discrete OT (ESAIM M2AN 2015)]
98. Part. 3 Optimal Transport – 2D examples
Numerical Experiment: A disk becomes two disks
99. Part. 3 Optimal Transport – 3D examples
Numerical Experiment: A sphere becomes two spheres
100. Part. 3 Optimal Transport – 3D examples
Numerical Experiment: Armadillo to sphere
101. Part. 3 Optimal Transport – 3D examples
Numerical Experiment: Other examples
102. Part. 3 Optimal Transport – 3D examples
Numerical Experiment: Varying density
103. Part. 1 Optimal Transport
[Schwartzburg et.al, SIGGRAPH 2014]
104. Part. 1 Optimal Transport
Optimal transport E.U.R.
let there be light !
105. Part. 1 Optimal Transport
Optimal transport E.U.R.
let there be light !
The millenium simulation project,
Max Planck Institute fur Astrophysik
pc/h : parsec (= 3.2 light years)
106. Part. 4 Optimal Transport – 3D examples
Numerical Experiment: Performances
2002: several hours of supercomputer time were needed
for computing OT with a few thousand Dirac masses, with a combinatorial
algorithm in O(n2log(n))
107. Part. 4 Optimal Transport – 3D examples
Numerical Experiment: Performances
2002: several hours of supercomputer time were needed
for computing OT with a few thousand Dirac masses, with a combinatorial
algorithm in O(n2log(n))
2015:
3D version of [Mérigot] (multilevel + BFGS) + several tricks [L 2015]
108. Part. 4 Optimal Transport – 3D examples
Numerical Experiment: Performances
2002: several hours of supercomputer time were needed
for computing OT with a few thousand Dirac masses, with a combinatorial
algorithm in O(n2log(n))
2015:
3D version of [Mérigot] (multilevel + BFGS) + several tricks [L 2015]
109. Part. 4 Optimal Transport – 3D examples
Numerical Experiment: Performances
2002: several hours of supercomputer time were needed
for computing OT with a few thousand Dirac masses, with a combinatorial
algorithm in O(n2log(n))
2015:
3D version of [Mérigot] (multilevel + BFGS) + several tricks [L 2015]
2016: Damped Newton [Mérigot, Thibert] + several tricks [L] for 3D:
1 million Dirac masses in 240 seconds
110. Part. 4 Optimal Transport – 3D examples
Numerical Experiment: Performances
2002: several hours of supercomputer time were needed
for computing OT with a few thousand Dirac masses, with a combinatorial
algorithm in O(n2log(n))
2015:
3D version of [Mérigot] (multilevel + BFGS) + several tricks [L 2015]
2016: Damped Newton [Mérigot, Thibert] + several tricks [L] for 3D:
1 million Dirac masses in 240 seconds
10 million Dirac masses in 90 minutes
111. Part. 4 Optimal Transport – 3D examples
Numerical Experiment: Performances
2002: several hours of supercomputer time were needed
for computing OT with a few thousand Dirac masses, with a combinatorial
algorithm in O(n2log(n))
2015:
3D version of [Mérigot] (multilevel + BFGS) + several tricks [L 2015]
2016: Damped Newton [Mérigot, Thibert] + several tricks [L] for 3D:
1 million Dirac masses in 240 seconds
10 million Dirac masses in 90 minutes
Semi-discrete OT is now scalable ! (new tool in Num. Ana. Toolbox)
112. Other topics
•Euler equation in more complicated setting:
[Merigot & Mirebeau, Merigot & Galouet]
•Using semi-discrete OT to solve other PDEs
[Benamou, Carlier, Merigot , Oudet]
•Faster solvers
113. Online resources
Source code: alice.loria.fr/software/geogram
Demos: www.loria.fr/~levy/GEOGRAM/
www.loria.fr/~levy/GLUP/
L., A numerical algorithm for semi-discrete L2 OT in 3D,
ESAIM Math. Modeling and Analysis, 2015
Video (presentation with the proofs)
www.loria.fr/~levy (Ecole d été calcul des variations 2015)
New projects:
MAGA Q. Mérigot - ANR
EXPLORAGRAM (Inria) with E. Schwindt, Q. Mérigot and J.-D. Benamou