Practising Fourier Analysis with Digital Images

PRACTICING FOURIER
ANALYSIS WITH DIGITAL
IMAGES
MASTERS IN COMPUTER VISION
Frédéric Morain-Nicolier
frederic.nicolier@univ-reims.fr
2014

1. INTRODUCTION 1.1. WHO IS IT ?
CONTENTS
1. INTRODUCTION
1.1 Who is it ?
1.2 Outline
1.3 References
2. FOURIER AND ITS
REPRESENTATIONS
3. UNDERSTANDING THE
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
2 / 139

WHO IS IT ?
3 / 139

CONTACT INFORMATIONS
Frédéric Morain-Nicolier
I http://pixel-shaker.fr
I frederic.nicolier@univ-reims.fr
I Dept Geii, IUT Troyes, 9 rue de Québec, 10026 Troyes
Cedex
I Phone : 03 25 42 71 68
4 / 139

1. INTRODUCTION 1.2. OUTLINE
CONTENTS
1. INTRODUCTION
1.1 Who is it ?
1.2 Outline
1.3 References
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
5 / 139

1. INTRODUCTION 1.2. OUTLINE
OUTLINE
I Fourier and its representations
I Understanding the Fourier Analysis
I Fourier Analysis Applications
(Focusing on Magnitude and Phase)
6 / 139

1. INTRODUCTION 1.3. REFERENCES
CONTENTS
1. INTRODUCTION
1.1 Who is it ?
1.2 Outline
1.3 References
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
7 / 139

WEB
I Earl F. Glynn - Research Notes
http://research.stowers-institute.org/efg
I Nicolas Thome - Introduction to Image Processing
http://webia.lip6.fr/~thomen/Teaching/BIMA.html
8 / 139

BOOKS
I D.W. Kammler, A first course in Fourier analysis,
Cambridge University Press, 2008.
I Jean Dhombres et Jean-Bernard Robert, Fourier, créateur de
la physique mathématique, collection “Un savant, une
époque”, Belin, 1998.
9 / 139

ARTICLES
I Q. Chen, M. Defrise and F. Deconinck, "Symmetric phase-only
matched filtering of Fourier-Mellin transforms for image
registration and recognition", IEEE pattern analysis and machine
intelligence, vol. 16, 1994, p. 1156-1168.
I Van des Schaaf A., Van Hateren J., "Modelling the Power Spectra
of Natural Images : Statistics and Information", Vision Research,
vol. 36, n°17, p. 2759-2770, 1996
I Y. Shapiro, and M. Porat, “Image Representation and
Reconstruction from Spectral Amplitude or Phase,” in IEEE
International Conference on Electronics, Circuits and Systems
1998, Lisboa, Portugal, 1998, pp. 461-464
I F. Nicolier, O. Laligant, F. Truchetet. Discrete wavelet transform
implementation in fourier domain. Journal of Electronic
Imaging, 11(3) :338–346, jul 2002
I N. Skarbnik, The Importance of Phase in Image Processing,
CCIT Report, 2010
10 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.1. FOURIER
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
2.1 Fourier
2.2 Fourier Series and Transform
2.3 Discrete Fourier Transform
2.4 Fast Fourier Transform
2.5 2D DFT
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
11 / 139

Joseph Fourier 12
WHO IS FOURIER ?
1768 (né à Auxerre) - 1830
Participe à la révolution
1798 : campagne d’Égypte
1802 : préfet de l’Isère (destitué à la
restauration)
1817 : élu membre de l’Académie des
Sciences
1822 : secrétaire perpétuel de l’AS
1826 : membre de l’Académie française
1822 : publication de la «théorie analytique
de la chaleur»
Jean Dhombres et Jean-Bernard Robert,
Fourier, créateur de la physique mathématique,
collection « Un savant, une époque », Belin
1998), ISBN 2-7011-1213-3.
septembre 2010
Jean Baptiste Joseph Fourier (1768-1830)
I 1768 (born in Auxerre) - 1830
I Active in French revolution
I 1798 : Napoleon’s Egypt Campaign
12 / 139

WHO IS FOURIER ?
Where is Auxerre ?
13 / 139

WHO IS FOURIER ?
I 1802 : préfet de l’Isère (dismissed during restauration)
I On the Propagation of Heat in Solid Bodies, was read to Paris
Institute on 21 dec 1807. Laplace and Lagrange objected to
what is now Fourier series : “... his analysis ... leaves
something to be desired on the score of generality and even
rigour...” (from report awarding Fourier math prize in 1811)
14 / 139

WHO IS FOURIER ?
I 1802 : préfet de l’Isère (dismissed during restauration)
I 1817 : member of Académie des Sciences
I 1822 : perpetual secretary of A.S.
I 1826 : membre de Académie Française
I 1822 : publication of La théorie analytique de la chaleur
(Analytic Theory of Heat)
15 / 139

WHO IS FOURIER ?
I In La Théorie Analytique de la Chaleur (Analytic Theory of
Heat) (1822), Fourier
I developed the theory of the series known by his name,
I and applied it to the solution of boundary-value problems
in partial differential equations.
Good Book (in french !) : Jean Dhombres et Jean-Bernard
Robert, Fourier, créateur de la physique mathématique, collection
“Un savant, une époque”, Belin, 1998.
16 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.2. FOURIER SERIES AND TRANSFORM
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
2.1 Fourier
2.5 2D DFT
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
17 / 139

FOURIER 3c)T RPArNoSFpOaRgMa:tWioHYn? de la chaleur 40
18 / 139

HEAT PROPAGATION
I The temperature u(x, t) at time t 0 and coordinate x is a
solution of the partial differential equation (PDE) :
¶u
¶t
(x, t) = a2 ¶2u
¶x2 (x, y) (2.1)
(a2 is the thermal diffusivity of the material).
I Fourier observed that
e2pisx.e4p2a2s2t (2.2)
satisfies the PDE for every choice of s. Its idea was to combine
such elementary solutions.
19 / 139

HEAT PROPAGATION : SOLUTION
I Fourier wrote the solution as
u(x, t) =
Z ¥
¥
A(s)e2pisxe4p2a2s2tds. (2.3)
The function A(s) is needed : as the initial temperature is
known,
u(x, 0) =
Z ¥
¥
A(s)e2pisxds (we recognize the synthesis equation).
(2.4)
I A(s) can be computed from
A(s) =
Z ¥
¥
u(x, 0)e2pisxdx. (2.5)
20 / 139

FOURIER SERIES
I Is only defined on periodic signals :
x(t + T) = x(t)
f (t + T) = f (t). (2.6)
I The fundamental period T0 is the smallest T satisfying (2.6).
Fundamental frequency f0 and angular frequency w0 are :
= 2pf0. (2.7)
6
Periodic Signals
-2 -1 0 1 2
“Biological” Time Series
T0
0 π 2 π 3π 2 2π 3π 4π
t
x(t)
(Source : Earl F. Glynn - Research Notes)
w0 =
2p
T0
Biological time series can be quite complex, and will contain noise.
21 / 139

FOURIER SERIES : REAL COEFFICIENTS
Expansion of continuous function into weighted sum of sines
and cosines.
I A P0-periodic function f , defined on R can be written as
f (t) = a0 +
¥å k=1
(ak cos(kw0t) + bk sin(kw0t)) (2.8)
with
a0 =
1
P0
Z
P0
f (t)dt, (2.9)
ak =
2
P0
Z
P0
f (t) cos(kwt)dt, (2.10)
bk =
2
P0
Z
P0
f (t) sin(kwt)dt. (2.11)
22 / 139

FOURIER SERIES : AN EXAMPLE
(Source : http://www.science.org.au/nova/029/029img/wave1.gif)
23 / 139

FOURIER SERIES : COMPLEX COEFFICIENTS
IWith complex coefficients :
f (t) =
¥å
k=¥
ckeikw0t (2.12)
where
ck =
1
P0
Z
P0
f (t)eikw0tdt. (2.13)
I If f (t) is real, ck = ck .
I For k = 0, ck = average value of f (t) over one period.
I a0/2 = c0 ; ak = ck + ck ; bk = i(ck ck)
24 / 139

FOURIER SERIES : COMPLEX COEFFICIENTS
f (t) =
¥å
k=¥
ckeikw0t
I Coefficients can be written as
ck = jckjeifk (keep this in mind). (2.14)
I ck are the spectral coefficients of f .
I Plot of jckj vs angular frequency w is the Magnitude
spectrum.
I Plot of fk vs w is the phase spectrum.
I With discrete Fourier frequencies (kw0), both are discrete
spectra.
25 / 139

FOURIER SERIES : OTHER EXAMPLES
Given a p-periodic x(t) = t, its Fourier serie is
x(t) = 2

sin t
sin 2t
2
sin 3
=  − + − ...
+
sin 3t
3 . . .

. (2.15)



sin 2
Given: x(t) = t Fourier Series:
14
Fourier Series
sin 3
=  − + − ...
sin 3
=  − + − ...
selected


 sin 2
sin 3
= − + − 
...

sin 2
sin 3
=  − + − ...
sin 2
1
2
3
1
2
3
4
5 6
sin 2
4
5 6
Approximate any function as truncated Fourier series

3
2
( ) 2 sin
t t
x t t


3
2
( ) 2 sin
t t
x t t
-1.0 -0.5 0.0 0.5 1.0 1.5 2.0
Fourier Terms in Expansion of x(t) = t
t
Fourier Terms
0 π 4 π 2 3π 4 π
Fourier Series Approximation
t
x(t)
First Six Series Terms

0 π 4 π 2 3π 4 π
0 1 2 3
14
Fourier Series
selected

3
2
( ) 2 sin
t t
x t t


3
2
( ) 2 sin
t t
x t t
-1.0 -0.5 0.0 0.5 1.0 1.5 2.0
Fourier Terms in Expansion of x(t) = t
t
Fourier Terms
0 π 4 π 2 3π 4 π
t
x(t)
First Six Series Terms
0 π 4 π 2 3π 4 π
0 1 2 3
Fourier Series
=  − + − 3
selected
sin 2

3
2
( ) 2 sin
t t
x t t

sin 3
2
( ) 2 sin
t t
x t t
t
x(t)
0 π 4 π 2 3π 4 π
0 1 2 3
100 terms
200 terms
(show animations)
26 / 139

(CONTINUOUS) FOURIER TRANSFORM
I A non-periodic function, defined on R, can be synthesized
with
f (t) =
1
2p
Z +¥
¥
F(w)eiwtdw. (2.16)
I The analysis equation being
F(w) =
Z +¥
¥
f (t)eiwtdt. (2.17)
(Beware to convergence conditions - Gibbs - see Dirichlet theorem)
27 / 139

(CONTINUOUS) FOURIER TRANSFORM : MAIN
PROPERTIES
18
Fourier Transform
Properties of the Fourier Transform
(Source : Wikipedia)
From http://en.wikipedia.org/wiki/Continuous_Fourier_transform
Also see Schaum’s Theory and Problems: Signals and Systems, Hwei P. Hsu, 1995, pp. 219-223
28 / 139

(CONTINUOUS) FOURIER TRANSFORM : MAIN
PROPERTIES
Important properties (for this course) :
I Shift theorem :
g(t a) eiawG(w). (2.18)
I Scaling :
g(at) 1
jaj
G(
w
a
). (2.19)
I Convolution theorem :
(g h)(t) G(w)H(w). (2.20)
29 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.3. DISCRETE FOURIER TRANSFORM
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
2.1 Fourier
2.5 2D DFT
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
30 / 139

DISCRETE FOURIER TRANSFORM
As a digital image is a discrete 2D signal, a discrete version of
FT is needed. The previous definitions are adapted.
I A discrete signal s[n] is N-periodic if s[n + N] = s[n].
I Fundamental period N0 is the smallest N satisfying above
equation.
I Fundamental angular frequency is W0 = 2p
N0
.
31 / 139

DISCRETE FOURIER TRANSFORM (DFT) : DEFINITION
I A discrete signal s[n], n = 0, 1, . . . ,N 1, can be analyzed
with
S[k] = DFTfs[n]g =
N1
å
n=0
s[n]ei2pkn/N. (2.21)
with k = 0, 1, . . . ,N 1
I The inverse DFT (IDFT = DFT1 = synthesis equation) is
s[n] = IDFTfS[k]g =
1
N
N1
å
k=0
S[k]ei2pkn/N. (2.22)
32 / 139

DISCRETE FOURIER TRANSFORM (DFT)
I One-to-one correspondence between s[n] and S[k]
I DFT closely related to discrete Fourier series and the
Fourier Transform
I DFT is ideal for computer manipulation
I Share many of the same properties as Fourier Transform
I Multiplier 1N
can be used in DFT or IDFT. Sometimes 1 pN
used in both.
I Remember that FT (and therefore DFT) is defined on
periodic signals.
33 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.4. FAST FOURIER TRANSFORM
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
2.1 Fourier
2.5 2D DFT
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
34 / 139

FAST FOURIER TRANSFORM
X[k] =
N1
å
n=0
x[n]ei2pkn/N. (2.23)
I The FFT is a computationally efficient algorithm to
compute the Discrete Fourier Transform and its inverse.
I Evaluating the sum above directly would take O(N2)
arithmetic operations.
35 / 139

X[k] =
N1
å
n=0
x[n]ei2pkn/N, (2.24)
N = ei kwn
Wkn
N ) X[k] =
N1
å
n=0
x[n]Wkn
N . (2.25)
Butterfly algorithm
36 / 139

I The FFT algorithm reduces the computational burden to
O(NlogN) arithmetic operations.
I FFT requires the number of data points to be a power of 2
(usually 0 padding is used to make this true)
I FFT requires evenly-spaced time series
I Even faster FFT with sparse signals (SFFT : Sparse FFT)
37 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.5. 2D DFT
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
2.1 Fourier
2.5 2D DFT
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
38 / 139

Spatial 2. FOURIER Frequency in Images
AND ITS REPRESENTATIONS 2.5. 2D DFT
SPATIAL FREQUENCIES
33
Frequency = 1 Frequency = 2
1 Cycle
2 Cycles
39 / 139

DFT ON IMAGE
F[u, v] =
1
MN
M1
å
m=0
N1
å
n=0
I[m, n]ei2p(umM
+v nN
) (2.26)
34
2D Discrete Fourier Transform
−
−
1
1
M
N
F u v π
Σ Σ
1
I m n e
= ⋅
=
=
0
0
[ , ]
n
Fourier
Transform
um
−  +

vn
N
M
(0,N/2)
(0,0)


2
i
(-M/2,0) (M/2,0)
[ , ]
m
MN
M pixels
SM units
I[m,n] F[u,v]
(0,-N/2)
(M,N)
Spatial Domain Frequency Domain
Source: Seul et al, Practical Algorithms for Image Analysis, 2000, p. 249, 262.
(0,0)
N pixels
SN units
2D FFT can be computed as two discrete Fourier transforms in 1 dimension
40 / 139

DFT ON IMAGE
I Separable implementation :
2D DFT (or FFT) is computed as two stages of 1D discrete
Fourier transforms (matlab : fft2).
I Ix Ixy
(process on columns) (process on lines)
41 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.1. READING THE 2D-DFT
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
3.1 Reading the 2D-DFT
3.2 Magnitude and Phase Spectra
3.3 Translation and Rotation in
Fourier Domain
3.4 Magnitude and Phase
Information
Reconstruction
4. FOURIER ANALYSIS
APPLICATIONS
42 / 139

WHERE IS THE INFORMATION ?
35
2D Discrete Fourier Transform
Fourier
Transform
(-M/2,0) (M/2,0)
I[m,n] F[u,v]
(0,0)
(0,0)
(0,N/2)
(0,-N/2)
(M,N)
M pixels
SM units
N pixels
SN units
Edge represents highest frequency,
smallest resolvable length (2 pixels)
Center represents lowest frequency,
which represents average pixel value
43 / 139

EXAMPLE 1
36
2D FFT Example
FFTs Using ImageJ
ImageJ Steps: (1) File | Open, (2) Process | FFT | FFT
(0,0) Origin (0,0) Origin
Image 2D-DFT (Magnitude)
44 / 139

EXAMPLE - SWAPPING THE QUADRANTS
37
2D FFT Example
FFTs Using ImageJ
ImageJ Steps: Process | FFT | Swap Quadrants
(0,0) Origin
(0,0) Origin
Default d (Source : Earl F. Glynn - Research Notesis)play is to swap quadrants
(matlab : fftshift)
Regularity in image manifests itself in the degree of order or randomness in FFT pattern.
45 / 139

TOY EXAMPLES
170 The Fourier transform
Figure 7.1 Examples of Fourier magnitude images (right column) of images containing only
sinusoids (left and middle column). Axes have been added for clarity. See text for details.
(Source : Introduction to Image Processing - Univ. Utrecht)
46 / 139

REAL EXAMPLE
38
2D FFT Example
FFTs Using ImageJ
Overland Park Arboretum and Botanical Gardens, June 2006
47 / 139

REAL EXAMPLES
7.1 The relation between digital images and sinusoids 171
Figure 7.2 Examples of Fourier magnitude images (right column) of real images (left column).
The top example is of a binary image, the other images are grey-valued. See text for details.
(Source : Introduction to Image Processing - Univ. Utrecht)
48 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.2. MAGNITUDE AND PHASE SPECTRA
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
Fourier Domain
Information
Reconstruction
4. FOURIER ANALYSIS
APPLICATIONS
49 / 139

MAGNITUDE AND PHASE SPECTRA
The DFT coefficients are complex numbers :
I Magnitude spectrum is generally considered the most
readable
I Phase spectrum is intricated
50 / 139

AN EXAMPLE
1
0.8
0.6
0.4
0.2
0
f
50 100 150 200 250
50
100
150
200
250
x 104
3
2.5
2
1.5
1
0.5
50 100 150 200 250
50
100
150
200
3
2
1
0
−1
−2
250 −3
Magnitude : kFk Phase : f(F)
51 / 139

ANOTHER EXAMPLE
1
0.8
0.6
0.4
0.2
0
f
50 100 150 200 250
50
100
150
200
250
x 104
3.5
3
2.5
2
1.5
1
0.5
50 100 150 200 250
50
100
150
200
3
2
1
0
−1
−2
250 −3
Magnitude : kFk Phase : f(F)
52 / 139

MAGNITUDE SPECTRUM
1
0.8
0.6
0.4
0.2
0
50 100 150 200 250
50
100
150
200
250
x 104
3.5
3
2.5
2
1.5
1
0.5
I F[0, 0] (at the center) contains the mean value of the image
F[u, v] =
1
MN
M1
å
m=0
N1
å
n=0
I[m, n]ei2p(umM
+v nN
) (3.1)
) F[0, 0] =
1
MN
M1
å
m=0
N1
å
n=0
I[m, n] (3.2)
53 / 139

MAGNITUDE SPECTRUM
1
0.8
0.6
0.4
0.2
0
50 100 150 200 250
50
100
150
200
250
x 104
3.5
3
2.5
2
1.5
1
0.5
f kFk
I Very high dynamics
I Low frequencies have a greater magnitude than high
frequencies
I It is common to represent log(1 + kFk)
54 / 139

LOG MAGNITUDE SPECTRUM
1
0.8
0.6
0.4
0.2
0
50 100 150 200 250
50
100
150
200
250
x 104
3.5
3
2.5
2
1.5
1
0.5
50 100 150 200 250
50
100
150
200
250
10
9
8
7
6
5
4
3
2
1
kFk log(1 + kFk)
55 / 139

LOG MAGNITUDE SPECTRUM #2
1
0.8
0.6
0.4
0.2
0
50 100 150 200 250
50
100
150
200
250
x 104
3
2.5
2
1.5
1
0.5
50 100 150 200 250
50
100
150
200
250
10
9
8
7
6
5
4
3
2
1
kFk log(1 + kFk)
56 / 139

STRUCTURES IN MAGNITUDE SPECTRUM
1
0.8
0.6
0.4
0.2
0
50 100 150 200 250
50
100
150
200
250
10
9
8
7
6
5
4
3
2
1
f log(1 + kFk)
I Edge in spatial domain , line in Fourier Domain
(orthogonal to the edge)
57 / 139

38
2D FFT Example
FFTs Using ImageJ
I Where are the vertical and horizontal edges ?
58 / 139

38
2D FFT Example
FFTs Using ImageJ
I Where are the vertical and horizontal edges ?
) Remember the implicit periodicity !
59 / 139

I Strong main lines in image are emphasized in Fourier
40
Application of FFT
Pattern/Texture Recognition
Source: Lee and Chen, A New Method for Coarse Classification of Textures and Class Weight Estimation
60 / 139

I Strong main lines in image are emphasized in Fourier
I
(Source : N. Thome)
61 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.3. TRANSLATION AND ROTATION IN FOURIER DOMAIN
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
Fourier Domain
Information
Reconstruction
4. FOURIER ANALYSIS
APPLICATIONS
62 / 139

TRANSLATION EXAMPLE
50 100 150 200 250
50
100
150
200
250
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
50 100 150 200 250
50
100
150
200
250
4
3.5
3
2.5
2
1.5
1
0.5
0
f log(1 + kFk)
50 100 150 200 250
50
100
150
200
250
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
50 100 150 200 250
50
100
150
200
250
4
3.5
3
2.5
2
1.5
1
0.5
0
fT log(1 + kFTk)
63 / 139

TRANSLATION
I Shift in spatial domain :
f [k] ) F[u] ) kF[u]k
f [k a] ) ei2pa uN
F[u] ) kF[u]k. (3.3)
I Magnitude spectrum is invariant to spatial translation.
I Localization information is in phase.
I Remember this for later !
64 / 139

ROTATION EXAMPLE
(Source : N. Thome)
65 / 139

ROTATION
I One can easily shows that
FT [f [x cos q + y sin q,x sin q + y cos q]] =
F[u cos q + v sin q,u sin q + v cos q]. (3.4)
I q-rotation in spatial domain , q-rotation in Fourier
domain (nice !)
66 / 139

ROTATION : ANOTHER EXAMPLE
(Source : N. Thome)
I Pay attention to padding when rotating.
67 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.4. MAGNITUDE AND PHASE INFORMATION
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
Fourier Domain
Information
Reconstruction
4. FOURIER ANALYSIS
APPLICATIONS
68 / 139

MAGNITUDE AND PHASE
1
0.8
0.6
0.4
0.2
0
50 100 150 200 250
50
100
150
200
250
10
9
8
7
6
5
4
3
2
1
50 100 150 200 250
50
100
150
200
3
2
1
0
−1
−2
250 −3
I Localization is in Phase, hard to read
I Frequential content is in Magnitude, easy to read
I But, what spatial domain information is in magnitude (and
phase) ?
69 / 139

MAGNITUDE AND PHASE
f
log(1 + kFk) f(F)
I Take the IFT with only magnitude or phase.
70 / 139

IFT WITH MAGNITUDE AND PHASE ONLY
I Starting from
F = FT[f ] = kFk.eif(F) (3.5)
I two images can be obtained :
fM = FT1[kFk], (3.6)
fP = FT1[eif(F)] = FT1[
F
kFk
]. (3.7)
71 / 139

IFT WITH MAGNITUDE AND PHASE ONLY
fM fP
I Magnitude contains almost no useful spatial information
I Main structures can be retrieved from phase
I Let’s play to mix images !
72 / 139

MAGNITUDE AND PHASE MIXING
I Take two images :
f g
I and their Fourier transforms :
F = kFk.eif(F), (3.8)
G = kGk.eif(G) (3.9)
73 / 139

I Two new images by swapping magnitude and phase :
I1 = kFk.eif(G), (3.10)
I2 = kGk.eif(F). (3.11)
I Guess the result !
74 / 139

f g
kFk.eif(G) kGk.eif(F)
75 / 139

We first wish to examine qualitatively- which of the two, magnitude or phase, carries more visual
information. This can be most vividly demonstrated by the following experiment: the Fourier components-
(phase and magnitude) are generated for the two images of same dimensions, and then swapped (see figure
1), whereby the reconstructed images appears to be more similar to the one whose Fourier phase was used
in the reconstruction. This experiment was previously suggested by Oppenheim in [7] and elsewhere.
MAGNITUDE AND PHASE MIXING : ANOTHER
EXAMPLE
Figure 1: Swapping the Fourier phase and magnitude in images. Top left - original Lena image.
Top right- original monkey image. Bottom left- IFT of Lena phase and monkey magnitude.
Bottom right- IFT of monkey phase and Lena magnitude.
(Source : N. Skarbnik - CCIT Report)
2
76 / 139

pronounced by individuals of different gender were recorded. The signals' phase and magnitude were
swapped. The resulting sentences were played to human listeners- which were able to understand the
meaning of the sentence, as well as to identify the gender of the speaker. Thus, it appears that most of the
signal's information is carried by its phase in 1D case as well. The effect of using a swapped magnitude
resulted in appearance of noise, in a manner similar to the 2D case.
MAGNITUDE AND PHASE MIXING : EVEN FOR
SIGNALS !
The reader is encouraged to review figure 2 and to examine the spectrograms similarity, or to use this link
for the audio files (click images to download the file) in order to evaluate the importance of phase in human
voice signals.
Figure 2: Exchanging the Fourier phase and magnitude in voice. Top left - woman voice
spectrogram. Top right- man voice spectrogram. Bottom left- spectrogram of woman voice phase
and man voice magnitude. Bottom right- spectrogram of man voice phase and woman voice
magnitude. Both reconstructions are primarily dominated by Fourier phase, and not the magnitude.
(Source : N. Skarbnik - CCIT Report)
Next, let us compare the global phase and magnitude by reviewing their distribution in a realistic image
(Lena image in this case).
I Listen the results
77 / 139

MAGNITUDE MODELISATION
I Phase is more informative than magnitude
I The Magnitude spectra is predictable. For natural images
(and signals) :
kF[u, b]k decreases when
p
u2 + v2 increases.
I Some models exists, see [SCH96] 1
1. Van des Schaaf A., Van Hateren J., Modelling the Power Spectra of Na-tural
Images : Statistics and Information, Vision Research, vol. 36, n°17, p.
2759-2770, 1996
78 / 139

POWER SPECTRA OF NATURAL IMAGES
From [SCH96] 2 :
I It has been found that power spectra of natural (ie not
artificial) images tends to depend as 1/f 2.
I From a set of 276 images, taken from a CCD camera
I Different outdoor environments (woods, fields, parks,
residential areas), at various times of the day, in various
seasons, and in various types of weather (sunny, overcast,
foggy, rainy)
I Power Spectra : S = 1
Npixels kFk2
I Take the average Power Spectra over the 276 images
2. Van des Schaaf A., Van Hateren J., Modelling the Power Spectra of Na-tural
Images : Statistics and Information, Vision Research, vol. 36, n°17, p.
2759-2770, 1996
79 / 139

FIGURE 2.1: (A) The average power as a function of spatial frequency. Fat dots show the average over the
complete set of the logarithm of the circularly averaged individual power spectra. Small dots give the
standard deviation from the average of corresponding plots of individual images. (B) The average power as
a function of orientation. Here the power spectra are first averaged over spatial frequency, then the
logarithm is taken, and finally the plots are averaged over the complete set (fat dots). Small dots as in (A).
(Source : Van des Schaaf A., Van Hateren J., Modelling the Power Spectra of Natural Images : Statistics and
because it Information, is absent in a Vision set of images Research, without vol. 36, dominant n°17, p. orientations 2759-2770, (1996)
mostly images
taken from soil covered with leaves and twigs with the camera pointed vertically at
random orientations). The small dots in Fig. 2.1A,B show the standard deviation of
the corresponding plots of individual images in the set.
80 / 139

FIGURE 2.2: (A) Example traces of the power spectra of five individual natural images. Dots show the
logarithm of the circularly averaged power spectrum as a function of spatial frequency. The lines show the
fits of the 1/f α model. The scaling of the vertical axis belongs to the top trace. For clarity, the lower traces
are shifted -2, -6, -8, and -10 log-units, respectively. (B) Distribution of r.m.s.-contrasts for the entire set of
276 natural images. (C) Distribution of 1/f-exponents (α) for the entire set.
(Source : Van des Schaaf A., Van Hateren J., Modelling the Power Spectra of Natural Images : Statistics and
Information, Vision Research, vol. 36, n°17, p. 2759-2770, 1996)
Not only the r.m.s.-contrast, and consequently the total power, vary for individual
images, but also the shape of the power spectrum. As shown in previous studies and
by the almost straight line in Fig. 2.1A, the spectral power, averaged over many
images, varies approximately as 1/f α as a function of spatial frequency, with the 1/f-exponent,
α, close to 2. If we instead inspect the spectra of individual images,
81 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.5. MAGNITUDE AND PHASE RECONSTRUCTION
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
Fourier Domain
Information
Reconstruction
4. FOURIER ANALYSIS
APPLICATIONS
82 / 139

where not all FT information is available (like with SAR images and X-ray crystallography) or when it is
degraded. We wish to demonstrate that the use of local features allows better algorithm performance:
faster convergence, or usage of less a priory known data. We also intend to demonstrate that phase based
algorithms sometimes result in a superior outcome compared to magnitude based ones.
PARTIAL MAGNITUDE RECONSTRUCTION
We will address iterative schemes, as the closed form
solutions demand solving a large set of linear equations,
which in turn involves inversion of appropriate matrices.
Those matrices inversion is impractical for images of
dimensions above 16X16 pixels.
I How much magnitude is it
possible to reconstruct ?
I Iterative scheme proposed
in [Sha98]
I Allow the reconstruction
of 25% of the image
I The authors report a
decent signal
reconstruction after 50
iterations
A Global Magnitude reconstruction scheme presented in
[2] can be seen in the following figure 5. The proposed
methods allow the reconstruction of a signal using at least
25% of the image (half of the signal in each dimension) and
its Fourier magnitude. As the reader can see, the
reconstruction is achieved by an iterative detection of the
unknown part of the signal. The authors of [2] report a
decent signal reconstruction after 50 iterations. In their
next paper [3] the authors propose a Local magnitude-based
Figure 5: A flowchart of image reconstruction
from global magnitude. Diagram adopted from
[2]. [SHA98] Y. Shapiro, and M. Porat, “Image Representation and Reconstruction
from Spectral Amplitude or Phase,” in IEEE International Conference on
Electronics, Circuits and Systems 1998, Lisboa, Portugal, 1998, pp. 461-464.
5
image reconstruction method. While the
reconstruction process converges faster (fever stages for
same quality- see figure 7), it demands more computations.
Out of those stages, the last one, for example, will demand
the same number of iterations as the whole Global
Magnitude based method. On the other hand the number
of spatial points to be known in advance drops to 1 as
opposed to the ~25% needed by the Global Magnitude
based method. The proposed method consists of
application of the Global Magnitude based method to an
increasing part of the original signal, until whole signal
reconstruction is achieved. A graphical description can be
seen in figure 7.
As can be seen from the following figure, the scheme is applied to a sub signal of the dimensions of [2k, 2k],
where k is the iteration number (Xk is the appropriate label on the figure). An N by M image will demand
2 log (max[M, N]) applications of the Global magnitude scheme (which is iterative too) to a sub image of
[2k, 2k] dimensions.
83 / 139

PARTIAL MAGNITUDE RECONSTRUCTION -
ALGORITHM
x
FFT
X Phase
e
iφX
insert
xc
y
FFT
Y
Mag.
||Y ||
IFFT
x cancel
border xc
84 / 139

PARTIAL MAGNITUDE RECONSTRUCTION - EXAMPLE
Initial After 50 iterations
85 / 139

PHASE ONLY RECONSTRUCTION
616 IEEE TRANSACTIONS OAN C OUSTICS ,P EECH, AND SIGNAL PROCESSING, VOLA . SSP-28N , O. 6, DECEMBER 1980
v. NUMERICAALL GORITHMFSO R RECONSTRUCTION
FROM SAMPLES OF A PHASE FUNCTION
In Section 11, we presented two sets of conditions, embodied
in Theorems 1 and 2 , under which a sequence is uniquely
specified to within a positive scale factor by the phase of its
Fourier transform. In this section, we describe two numerical
algorithms which can be used to reconstruct a sequence satis-fying
I Is it possible to reconstruct
an image from phase only ?
I Iterative scheme proposed
in [HAY82]
I Needs an M-point FFT
(with M 2N).
the requirements of Theorem 1 from samples of its phase
function when the location of the first nonzero point of x [ n ]
and the interval outside of which x [ n ] is zero are known.
Although these algorithms will only be discussed in terms of
reconstructing sequences satisfying the conditions of Theorem
1, the reconstruction of sequences meeting the requirements
of Theorem 2 may be accomplished by simply reconstructing
the finite length sequence %In] defined in ( 5 ) using the nega-tive
of the specified phase samples and then computing the
convolutional inverse sequence.
The first algorithm presented below is an iterative technique
in which the estimate of X [ . ] is improved in each iteration.
This algorithm is similar to the iterative algorithms developed
by Gerchberg and Saxton [ 6 ] and Fienup [ 7 ] for reconstruct-ing
a signal from magnitude information and to the iterative
algorithm developed by Quatieri [ 8 ] for reconstructing a
signal from its phase under the assumption that the signal is
minimum phase. The second algorithm is a closed form solu-tion
which is obtained by solving a set of linear equations.
Under the conditions specified in Theorem 1, this algorithm
provides the desired sequence x [ n ] to within a scale factor
when the location of the first nonzero point of x [ n ] and the
interval outside of which x [n] is zero are known.
In the discussions which follow, x [ n ] is used to denote a
sequence which satisfies the conditions of Theorem 1 and is
zero outside the interval 0 d n d N - 1 with x [ O ] # 0. In the
more general case (see footnote 3), a linear phase term may be
added to the given phase to accomplish this.
A. Iterative Algorithm
The M-point discrete Fourier transform (DFT) of x [ n ] will
be denoted as
[HAY82] Hayes, M. The Reconstruction of a Multidimensional Sequence
From the Phase or Magnitude of Its Fourier Transform. Acoustics, Speech
i o (k) and Signal Processing, = I X(kI e x IEEE Transactions (2 1)
on 30, no. 2 (1982) : 140-154
where it is assumed that M 2 2N. Then, an iterative technique
to reconstruct the sequence x [ n ] from the M samples of its
phase e,@), k = 0, 1, - - , M - 1, as illustrated in Fig. 1 and
may be described as follows.
Step 1: We begin with I Xo(k)l, an initial guess of the un-known
DFT magnitude and form the first estimate, X,@), of
X(k) using the specified phase function, i.e.,
Xl(k) = IXO(k)l e ie,(k) (22)
Computing the inverse DFT of X,@) provides the first esti-mate,
x1 [ n ] , of x [ n ] . Since an M-point DFT is used, x1 [ n ] is
I
I
I
I
I
I t r p
I
I
I
I r-l M-POINT DFT
I
II
I A
I
I
I
I
I
I
M-POINT IDFT
I
I
Fig. 1. Block diagram of the iterative algorithm for reconstructing a
signal from its phase.
From this, a new estimate x z [ n ] is obtained from the inverse
DFT of X , (k). Repetitive application of Steps 2 and 3 defines
the iteration.
In this iterative procedure, the total squared error between
x [ n ] and its estimate is nonincreasing with each iteration. To
see this, let x p [ n ]d enote the estimate after the pth iteration
and define the error Ep as
From Parseval's theorem,
86 / 139

PHASE ONLY RECONSTRUCTION - ALGORITHM
x
FFTM
X Phase
e
iφX
y
Y
Mag.
||Y ||
IFFT
N
0
M=3N-1
FFT
initialize with
||Y ||=1
N
87 / 139

PHASE ONLY RECONSTRUCTION - EXAMPLE
Initial After 10 iterations and
M = 2N 1
88 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.1. CLASSIC APPLICATIONS
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
4.1 Classic Applications
4.2 Fourier Shape Descriptors
4.3 Filter Banks in Fourier Domain
4.4 FMI-SPOMF Image Matching
4.5 Some final words
89 / 139

NOISE REMOVAL
Application of FFT in Image Processing
Four Noise
Spikes Removed
39
Noise Removal
FFT Inverse
FFT
Edit FFT
Noise Pattern
Stands Out as
Four Spikes
Source: www.mediacy.com/apps/fft.htm, Image Pro Plus FFT Example. Last seen online in 2004.
90 / 139

TEXTURE RECOGNITION
40
Application of FFT
Source: Lee and Chen, A New Method for Coarse Classification of Textures and Class Weight Estimation
91 / 139

TEXTURE RECOGNITION
The Drosophila eye is a great example a cellular crystal with its
hexagonally closed-packed structure. The absolute value of the
Fourier transform (right) shows its hexagonal structure.
41
Application of FFT
The Drosophila eye (Source is a : great Earl F. Glynn example - Research a Notes)
cellular crystal with
its hexagonally closed-packed structure. The absolute
value of the Fourier transform (right) shows its hexagonal
structure.
Source: http://www.rpgroup.caltech.edu/courses/PBL/size.htm
Could FFT of Drosophila eye be used to identify/quantify subtle phenotypes?
92 / 139

DEBLURRING - DECONVOLUTION
Application of FFT
Deblurring: Deconvolution
The Point Spread Function (PSF) is the Fourier transform of a filter.
(the PSP says how much blurring there will be in trying to image a point).
Hubble image and measured PSF
Dividing the Fourier transform of the PSF into
the transform of the blurred image, and
performing an inverse FFT, recovers the
unblurred image.
FFT(Unblurred Image) * FFT(Point Spread Function) = FFT(Blurred Image)
Unblurred Image = FFT-1[ FFT(Blurred Image) / FFT(Point Spread Function) ]
45
Source: http://www.reindeergraphics.com/index.php?option=com_contenttask=viewid=179Itemid=127
93 / 139

Application of FFT
DEBLURRING - DECONVOLUTION
Deblurring: Deconvolution
The Point Spread Function (PSF) is the Fourier transform of a filter.
(the PSP says how much blurring there will be in trying to image a point).
Hubble image and measured PSF
Dividing the Fourier transform of the PSF into
the transform of the blurred image, and
performing an inverse FFT, recovers the
unblurred image.
46
Deblurred image
Source: http://www.reindeergraphics.com/index.php?option=com_contenttask=viewid=179Itemid=127
94 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.2. FOURIER SHAPE DESCRIPTORS
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
95 / 139

Fourier DDeessccrriippttoorrss:: OOvveerrvviieeww
OVERVIEW
... ...
(Source : Computer Vision Remote Sensing - Univ. Berlin)
● Concise and description of (object) contours
➔ Contours are represented by vectors
I Description ● Numerous of application
(object) contours represented as vectors.
➔ Contour Processing (filtering, interpolation, morphing)
I Applications ➔ Image analysis: :
Characterising and recognising the shapes of object
I Contour Processing (filtering, interpolation, morphing)
I Image analysis : characterizing and recognizing the shapes
of object
I Shape = closed contour !
96 / 139

REPRESENTING RReepprreesseennttAiinnCggO Naa TCCOooUnnRttWoouIuTrrH uuDssFiinnTgg tthhee DDFFTT
... ...
(xN,yN): Coordinates of
the Nth point along the
circumference
Pixels on the contour are
assumed to be ordered (e.g.
clockwise)!
1st Step
Define a complex vector
using coordinates (x,y).
2nd Step
Apply the 1D DFT
97 / 139

AApppplliiccaattiioonn:: RReeccooggnniissiinngg aanndd ccllaassssiiffyyiinngg
EXAMPLE
lleeaavveess
Database
Two types of leaves are to be
recognised and classified
98 / 139

EXAMPLE
lleeaavveess
Image with unclassified objects
99 / 139

EXAMPLE
lleeaavveess
Segmented Objects
(Thresholding)
100 / 139

EXAMPLE
lleeaavveess
Leaves detected and classified
101 / 139

TRANSLATION
TTrraannssllaattiioonn
t
t
Translating U by t:
I Only on F[0].
102 / 139

SCALING CChhaannggeess iinn SSccaallee
Magnification by factor s:
103 / 139

ROTATION
RRoottaattiioonn
Rotation by an angle θ:
(Derivation identical to scale change: Multiplication by constant)
I in Phase
104 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.3. FILTER BANKS IN FOURIER DOMAIN
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
105 / 139

Examples of Fourier’s representation 17
LINEAR TRANSLATION INVARIANT SYSTEMS
fi
Input!
System
A
fo
Output!
Figure 1.14. Schematic representation of a system A.
I Linear System :
In practice we often deal with systems that are homogeneous and additive, i.e.,
A(cf ) = c(Af ) and (4.1)
A(cf) = c(Af)
A(f + g) = (Af) + (Ag)
A(f + g) = (Af ) + (Af ). (4.2)
I Shift (or translation) invariance :
when f, g are arbitrary inputs and c is an arbitrary scalar. Such systems are said be linear. Many common systems also have the property of translation invariance.
We say that a system is translation invariant if the output
gi(t) = fi(t + t) (4.3)
) gi(t) = fo(t + t). (4.4)
go = Agi
(the output is shifted by the same amount than the input).
an arbitrary ! -translate
gi(t) := fi(t + ! ), −# t #,
106 / 139

LINEAR TRANSLATION INVARIANT SYSTEMS
I An LTI system responds sinusoidally when it is shaken
sinusoidally
I The output can be obtained
I from the impulse response g[n]
I by a convolution product :
fo[n] = (fi g)[n] =
¥å
k=¥
f [k]g[n k] (4.5)
I or in Fourier Domain
Fo[u] = Fi[u]G[u] (Convolution Theorem) (4.6)
107 / 139

CONVOLUTION IN FOURIER DOMAIN
Enhanced Image
Pre-processing
Post-processing
FFT{ I[u,v] } FFT-1{ H[u,v] · F[u,v] }
Fourier Transform
42
Application of FFT
Filtering in the Frequency Domain: Convolution
I[m,n]
Raw Image
I’[m,n]
Fourier Transform
F[u,v]
Filter Function
H[u,v]
Inverse
F[u,v] H[u,v] · F[u,v]
Source: Gonzalez and Woods, Digital Image Processing (2nd ed), 2002, p. 159
108 / 139

FILTER BANKS
I A filter bank is an array of band-pass filters that spans the
entire frequency spectrum.
I The bank serves to isolate different frequency components
in a signal
(Source : http://www.aamusings.com/project-documentation/wavs/filterBank.html)
109 / 139

FILTER BANKS
(Source : http://www.aamusings.com/project-documentation/wavs/filterBank.html)
Frequent scheme :
I DCT : Discrete Cosine Transform (a special case of DFT),
I DWT : DiscretWavelet Transform.
110 / 139

FILTER BANKS - FWT
A common implementation of the DiscreteWavelet Transform
is the Mallat Algorithm (the Fast WT) :
I h is a low-pass filter
I g is a high-pass filter
111 / 139

FWT - B-SPLINE WAVELETS
Impulse
Responses
Frequential
Responses
Obtained from the autoconvolution of the box function :
112 / 139

B-SPLINE WAVELETS - EXAMPLE
113 / 139

FILTER BANK IN FOURIER DOMAIN
The high-pass branch :
I g is a linear filter (ok in Fourier Domain)
I the sub-sampling can also be obtained in Fourier Domain :
(u : zeroing the even samples)
114 / 139

FILTER BANK IN FOURIER DOMAIN
115 / 139

quinconce.
FWT IN FOURIER DOMAIN
1.7 – Nous Comparaison avons ainsi de proposé la complexitun algorithme é d’algorithmes de calcul des d’analyse coe⇢cients multird’une ésolution. analyse multiréso-lution
Rapport d’opérations où toutes nécessaires les opérations dans le (cas filtrage classique et échantillonnages) par rapport aux sont calculs e⇡ectuées intdans égralement le domaine
réalisL est la profondeur de l’analyse multirésolution. Ces rapports sont calculés en fonction de images. A gauche : cas séparable ; et à droite : cas quinconce.
de Fourier discret. Cet algorithme, présenté en figure 5.6 nécessite une FFT en entrée et une
FFT par signal de détails (coe⇢cients de l’analyse multirésolution).
As FFT
filtrage par
multiplications
complexes
sous-
échantillonnage
Ds+1
filtrage oar
multiplications
complexes
sous-
échantillonnage
Ds+2
...
FFT1 FFT1
1.6 – algorithme de calcul des coe⇥cients d’une analyse multirésolution dans le domaine discret.
5.6 – Algorithme de calcul des coe⇢cients d’une analyse multirésolution dans le do-maine
de Fourier discret.
1,8 fois moins d’opérations. De plus, nous avons montré que la complexité de cet algorithme Figure rapport à une implémentation classique, où seul les filtrages seraient effectués dans le domaine
Par rapport à une implémentation classique, où seul les filtrages seraient e⇡ectués dans le
Fourier, domaine notre algorithme de Fourier, necessite notre algorithme un nombre nécessite plus un faible nombre d’opplus érations. faible d’opérations. Les graphiques Les gra-phiques
de la indiquent que pour une profondeur d’analyse sur quatre échelles (un cas fréquent), notre algorithme
de la figure 5.7 indiquent que pour une profondeur d’analyse sur quatre échelles (un
116 / 139
cas fréquent), notre algorithme nécessite 1,8 fois moins d’opérations. De plus, nous avons mon-tré

des signaux doivent donc être des entiers. Une image X[k] de dimension N⇥N est transformée
en une image 4. FOURIER de ANALYSIS dimension APPLICATIONS (figure 4.3. 5.5).
FILTER BANKS IN FOURIER DOMAIN
Y [k] M ⇥M L’équation (5.34) est donc réécrite de la façon suivante :
2D-FWT IN FOURIER DOMAIN
Even for images :
M Jtm. L’équation peut donc être transformée pour exprimer le sur-échantillonnage
Figure 5.5 – Sur-échantillonnage d’une image : une rotation et une dilatation sont impliquée
dans la transformation.
5.3.2 Notre contribution
Ces deux équations (5.34 et 5.36) permettent d’exprimer les échantillonnages dans le do-maine
de Fourier. Cependant les signaux manipulés sont continus ce qui implique que lors de
l’implémentation, une discrétisation du domaine de Fourier sera nécessaire. Ce passage peut
être évité en écrivant les équations directement dans le domaine de Fourier discret. Les indices
des signaux doivent donc être des entiers. Une image X[k] de dimension N⇥N est transformée
en une image Y [k] de dimension M ⇥M (figure 5.5).
L’équation (5.34) est donc réécrite de la façon suivante :
Y () = X(⇥) avec
⇧
= 2
Mm
⇥ = 2
Mm⇥
, (5.37)
M Jtm. L’équation peut donc être transformée pour exprimer le sur-échantillonnage
oùm⇥ = N
de signaux multi-dimensionnels dans le domaine de Fourier discret :
Y [m] = X
⇤#
N
Jtm
⇥
modN
⌅
. (5.38)
Y () = X(⇥) avec
⇧
= 2
Mm
⇥ = 2
Mm⇥
, (5.37)
oùm⇥ = N
de signaux multi-dimensionnels dans le domaine de Fourier discret :
Y [m] = X
⇤#
N
M
Jtm
⇥
modN
⌅
. (5.38)
L’opérateur modulo (mod) est nécessaire pour tenir compte de la périodicité de transfor-mées
de Fourier discrètes rapides (FFT) des signaux. Selon une méthode analogue, le sous-échantillonnage
est :
Y [m] =
1
|detJ|
|de⌃tJ|−1
l=0
X
⇤#
N
M
J−tm− NJ−tvl
⇥
modN
⌅
. (5.39)
Dans certains cas particuliers les équations se simplifient fortement, permettant fréquem-ment
d’exprimer les échantillonnages comme des duplications (sur-échantillonnage) ou des
sommes de sous-parties des images (sous-échantillonnage). Par exemple dans le cas séparable,
les sous-échantillonnages bi-dimensionnels s’expriment comme un enchaînement de deux sous-échantillonnages
mono-dimensionnels. Ou encore, dans le cas quinconce le sous-échantillonnage
peut s’écrire comme la somme de quatre sous-images.
50
I More details in [NIC02] 3.
3. F. Nicolier, O. Laligant, F. Truchetet. Discrete wavelet transform imple-mentation
in fourier domain. Journal of Electronic Imaging, 11(3) :338–346, jul
2002
117 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.4. FMI-SPOMF IMAGE MATCHING
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
118 / 139

FMI-SPOMF ?
I SPOMF = Symmetric Phase Only Matched Filters
I FMI = Fourier-Mellin Invariant
I FT for matching images
I Translation, rotation, scaling invariant registering
I Described in [Chen94] 4
4. Q. Chen, M. Defrise and F. Deconinck, Symmetric phase-only matched
filtering of Fourier-Mellin transforms for image registration and recognition,
IEEE pattern analysis and machine intelligence, vol. 16, 1994, p. 1156-1168.
119 / 139

IMAGE MATCHING
I Where is the small bear ?
120 / 139

IMAGE MATCHING
From the image and a pattern (reference) :
The idea is to
I slide the pattern on the image,
I compute sum of the product pixel-to-pixel.
I This is a cross-correlation.
121 / 139

CC AND SSD RELATION
I A classical template matching solution is the Sum of Squared
Differences (SSD) - here with continuous functions :
SSDW =
Z
W
(f (t) g(t))2 dt (4.7)
I SSD is related to CC :
Z
W
(f (t) g(t))2 dt =
Z
W

f (t)2 + g(t)2 2f (t)g(t)

dt (4.8)
=
Z
W
f (t)2dt +
Z
W
g(t)2dt 2
Z
W
f (t)g(t)dt.
(4.9)
I If f is the pattern, the first term is constant !
122 / 139

CC AND SSD RELATION
Z
W
(f (t) g(t))2 dt = Cte +
Z
W
g(t)2 2
Z
W
f (t)g(t)dt. (4.10)
I If the local energy of the image (g) is constant :
Z
W
(f (t) g(t))2 = Cte 2
Z
W
f (t)g(t)dt. (4.11)
I In this case, the SSD is the same as the CC
I But the local energy of the image must be constant !
123 / 139

IMAGE MATCHING - CROSS CORRELATION
I The Cross-Correlation (CC) is defined as (for real signals) :
(f ? g)[n] =
¥å k=¥
f [k]g[n + k] (4.12)
=
¥å
k=¥
f [n k]g[n] (4.13)
I CC is strongly related to convolution :
(f ? g)[n] = f [n] g[n] (4.14)
I CC can also be easily expressed in Fourier Domain (fast
computations) :
FT[(f ? g)][u] = F[u]G[u] (4.15)
so (f ? g) = FT1[FT[f ]FT[g]]. (4.16)
124 / 139

CROSS-CORRELATION IN FOURIER DOMAIN
I Remember that the DFT implies the function is periodic
I The product F[u, v]G[u, v] implies F and G have the same
size
The pattern P is thus modified :
I its size must be the same as I,
I the origin of the image must corresponds to the center of
the pattern.
125 / 139

CROSS-CORRELATION - RESULT
I Local max at the center (good) but not the absolute one
I The second bear is not detected (rotation)
I CC is very sensitive to luminance
126 / 139

NORMALIZED CROSS-CORRELATION
I One solution (to luminance sensitivity) is to normalize the
images before the comparison :
NCCI,P =
1
N,y
x
å(I[x, y] I)(P[x, y] P)
sIsP
(4.17)
I is the average of I, sI is the standard deviation of I
I Very classic solution to template matching
I same as Pearson Correlation Coefficient
127 / 139

SPOMF
I Another solution is to only compare the Phase information
(structures !)
I The dectector is thus modified :
DI,P = FT1[
FT[I]
kFT[I]k
FT[P]
kFT[P]k
] (4.18)
I SPOMF : Symmetric Phase Only Matched Filters
128 / 139

SPOMF - RESULT
CC SPOMF
129 / 139

FMI-SPOMF IMAGE REGISTRATION
I CC based comparison (NCC and SPOMF) are rotation and
scaling sensitive
I Fourier-Mellin Transform is a solution
I The key point is to
I reduce rotation and scaling to translations
I and reduce the dimension of the parameter size.
I Polar coordinates : Rotation ! Translation
I Logarithmic scale : Scaling ! Translation
(log(ax) = log(a) + log(x))
130 / 139

FMI-SPOMF - PRINCIPLE
I 4 parameters : Translation (x and y), Rotation, Scaling
I Facts :
I Phase contains localisation
I Magnitude is insensitive to translation
I A rotation of the image rotates the spectral magnitude by
the same angle
I A scaling by s scales the spectral magnitude by s1
I Keeping magnitude only allows to isolate rotation and
scaling
I Rotation and scaling are transformed into translations ...
detected by SPOMF.
I Fourier-Mellin Transform = FT of polar-log magnitude
spectral image.
131 / 139

FMI - SPOMF - ALGORITHM
Given I and P :
I Compute the log-polar magnitude spectral images of I and
P
I Detect the max of the SPOMF image between I and P
I Identify s and q
I Re-scale and re-rotate P by (s1, q)
I Compute the SPOMF between I and the rectified P
I Locate the max, and identify (x, y) the translation vector.
132 / 139

LOG-POLAR REPRESENTATION
I Assuming I is a N N image, its log-polar representation
Ilp is :
Ip(r, q) = I(r cos q +
N
2
, r sin q +
N
2
(4.19)
Ilp(m, k) = Ip(
1
2
Nm
N ,
2pk
N p). (4.20)
I m 2 [1,N] is the discrete radial coordinate
I k 2 [1,N] is the discrete angular coordinate
I (an interpolation is needed : nearest-neighbor for example)
133 / 139

More general equations are given in [Chen94] 5 :
I A N N image I can be resampled onto a M K polar-log
grid in one step :
(
um,k = N/21
M1 (M 1)
m
M1 cos( pk
K ) + N2
vm,k = N/21
M1 (M 1)
m
M1 sin( pk
K ) + N2
, (4.21)
m 2 [0,M 1], k 2 [0,K 1].
5. Q. Chen, M. Defrise and F. Deconinck, Symmetric phase-only matched
filtering of Fourier-Mellin transforms for image registration and recognition,
IEEE pattern analysis and machine intelligence, vol. 16, 1994, p. 1156-1168.
134 / 139

I A detection of the max provides two integers m and k.
I Rotation and scaling :
q =
m N/2
M 1 (4.22)
s = (M 1)
k
M1 for 0 k M/2 (enlargement) (4.23)
s¯1 = (M 1)
Mk
M1 for M/2 k M (shrinkage) (4.24)
135 / 139

SOME EXAMPLES
original distorted
(s = 0.7, q = 30 deg)
136 / 139

SOME EXAMPLES
137 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.5. SOME FINAL WORDS
CONTENTS
1. INTRODUCTION
2. FOURIER AND ITS
REPRESENTATIONS
FOURIER ANALYSIS
4. FOURIER ANALYSIS
APPLICATIONS
138 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.5. SOME FINAL WORDS
SOME FINAL WORDS
I Fourier Transform still very popular in science
I Phase congruency - Local Phase. See Peter Kovesi
webpage 6
I Practice with :
I Playing with phase and magntiude
I Image Reconstruction
I Sampling in Fourier domain
I Template matching with phase spectrum
I Image registration with FMI-SPOMF
6. http://www.csse.uwa.edu.au/~pk/research
139 / 139

Practising Fourier Analysis with Digital Images

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Practising Fourier Analysis with Digital Images

Similaire à Practising Fourier Analysis with Digital Images (20)

Dernier

Dernier (20)

Practising Fourier Analysis with Digital Images