3F3 – Digital Signal Processing (DSP), January 2009, lecture slides 1, Dr Elena Punskaya, Cambridge University Engineering Department

1. Introduction to Digital Signal
Processing (DSP)
Elena Punskaya
www-sigproc.eng.cam.ac.uk/~op205
Some material adapted from courses by
Prof. Simon Godsill, Dr. Arnaud Doucet,
Dr. Malcolm Macleod and Prof. Peter Rayner
1

2. Books
Books:
– J.G. Proakis and D.G. Manolakis, Digital Signal
Processing 3rd edition, Prentice-Hall.
– R.G Lyons, Understanding Digital Signal
processing, 2nd edition, , Prentice-Hall.
2

3. What is Digital Signal Processing?
Digital: operating by the use of discrete signals to
represent data in the form of numbers
Signal: a parameter (electrical quantity or effect) that can
be varied in such a way as to convey information
Processing: a series operations performed according to
programmed instructions
changing or analysing information
which is measured as discrete
sequences of numbers
3

4. Applications of DSP - Biomedical
Biomedical: analysis of biomedical signals,
diagnosis, patient monitoring,
preventive health care, artificial
organs
Examples:
1) electrocardiogram (ECG) signal – provides
doctor with information about the condition of
the patient’s heart
2) electroencephalogram (EEG) signal – provides
Information about the activity of the brain
4

5. Applications of DSP - Speech
Speech applications:
Examples
1) noise reduction – reducing background noise
in the sequence produced by a sensing device (microphone)
2) speech recognition – differentiating
between various speech sounds
3) synthesis of artificial speech – text to speech
systems for blind
5

6. Applications of DSP - Communications
Communications:
Examples
1) telephony – transmission of information in digital form via
telephone lines, modem technology, mobile phones
2) encoding and decoding of the information
sent over a physical channel (to optimise
transmission or to detect or correct errors in
transmission)
6

7. Applications of DSP - Radar
Radar and Sonar:
Examples
1) target detection – position and
velocity estimation
2) tracking
7

8. Applications of DSP – Image Processing
Image Processing:
Examples
1) content based image retrieval – browsing,
searching and retrieving images from database
2) image enhancement
2) compression - reducing the redundancy in
the image data to optimise transmission /
storage
8

9. Applications of DSP – Music
Music Applications:
Examples:
1) Recording
2) Playback
3) Manipulation (mixing, special effects)
9

10. Applications of DSP - Multimedia
Multimedia:
generation storage and
transmission of sound, still
images, motion pictures
Examples:
1) digital TV
2) video conferencing
10

11. DSP Implementation - Operations
To implement DSP we must be able to:
Input Digital Digital
Signal Signal
DSP
Output
1) perform numerical operations including, for
example, additions, multiplications, data transfers
and logical operations
either using computer or special-purpose hardware
11

12. DSP chips
• Introduction of the microprocessor in the late 1970's and
early 1980's meant DSP techniques could be used in a
much wider range of applications.
DSP chip – a programmable
device, with its own native
instruction code
designed specifically to meet
numerically-intensive
requirements of DSP
capable of carrying out
millions of floating point
Bluetooth Household Home theatre
headset appliances system operations per second 12

13. DSP Implementation – Digital/Analog Conversion
To implement DSP we must be able to:
Digital Digital Analog
Signal Signal Signal
DSP Reconstruction
2) convert the digital information, after being processed
back to an analog signal
- involves digital-to-analog conversion & reconstruction
(recall from 1B Signal and Data Analysis)
e.g. text-to-speech signal (characters are used to generate artificial
sound)
13

14. DSP Implementation –Analog/Digital Conversion
To implement DSP we must be able to:
Analog Digital Digital
Signal Signal Signal
Sampling DSP
3) convert analog signals into the digital information
- sampling & involves analog-to-digital conversion
(recall from 1B Signal and Data Analysis)
e.g. Touch-Tone system of telephone dialling (when button is
pushed two sinusoid signals are generated (tones) and
transmitted, a digital system determines the frequences and
uniquely identifies the button – digital (1 to 12) output
14

15. DSP Implementation
To implement DSP we must be able to:
Analog Digital Digital Analog
Signal Signal Signal Signal
Sampling DSP Reconstruction
perform both A/D and D/A conversions
e.g. digital recording and playback of music (signal is sensed by
microphones, amplified, converted to digital, processed, and
converted back to analog to be played
15

16. Limitations of DSP - Aliasing
Most signals are analog in nature, and have to be sampled
loss of information
• we only take samples of the signals at intervals and
don’t know what happens in between
aliasing
cannot distinguish between
higher and lower frequencies
(recall from 1B Signal
and Data Analysis)
Sampling theorem: to avoid
aliasing, sampling rate must be
at least twice the maximum
frequency component
(`bandwidth’) of the signal
16
Gjendemsjø, A. Aliasing Applet, Connexions, http://cnx.org/content/m11448/1.14

17. Limitations of DSP - Antialias Filter
• Sampling theorem says there is enough information
to reconstruct the signal, which does not mean
sampled signal looks like original one
correct reconstruction is not
just connecting samples with
straight lines
needs antialias filter (to filter out all
high frequency components before
sampling) and the same for
reconstruction – it does remove
information though
Each sample
is taken at a
slightly earlier (recall from 1B Signal
part of a cycle and Data Analysis)
17

18. Limitations of DSP – Frequency Resolution
Most signals are analog in nature, and have to be sampled
loss of information
• we only take samples for a limited period of time
limited frequency
resolution
does not pick up “relatively”
slow changes
(recall from 1B Signal
and Data Analysis)
18

19. Limitations of DSP – Quantisation Error
Most signals are analog in nature, and have to be sampled
loss of information
• limited (by the number of bits available) precision in data
storage and arithmetic
quantisation error
smoothly varying signal
represented by “stepped”
waveform
(recall from 1B Signal
and Data Analysis)
19

20. Advantages of Digital over Analog Signal Processing
Why still do it?
• Digital system can be simply reprogrammed for other
applications / ported to different hardware / duplicated
(Reconfiguring analog system means hadware redesign, testing, verification)
• DSP provides better control of accuracy requirements
(Analog system depends on strict components tolerance, response may drift with
temperature)
• Digital signals can be easily stored without deterioration
(Analog signals are not easily transportable and often can’t be processed off-line)
• More sophisticated signal processing algorithms can be
implemented
(Difficult to perform precise mathematical operations in analog form)
20

21. Brief Review of Fourier Analysis
Elena Punskaya
www-sigproc.eng.cam.ac.uk/~op205
Some material adapted from courses by
Prof. Simon Godsill, Dr. Arnaud Doucet,
Dr. Malcolm Macleod and Prof. Peter Rayner
21

22. Time domain
Example: speech recognition
difficult to differentiate
between different sounds
in time domain
tiny segment
sound /i/
as in see
sound /a/
as in father
22

23. How do we hear?
Cochlea – spiral of tissue with liquid
and thousands of tiny hairs that
Inner Ear gradually get smaller
Each hair is connected to the nerve
The longer hair resonate with lower
frequencies, the shorter hair
resonate with higher frequencies
Thus the time-domain air pressure
signal is transformed into frequency
spectrum, which is then processed
www.uptodate.com by the brain
Our ear is a Natural Fourier Transform Analyser! 23

24. Fourier’s Discovery
Jean Baptiste Fourier showed that any
signal could be made up by adding together
a series of pure tones (sine wave) of
appropriate amplitude and phase
(Recall from 1A Maths)
Fourier Series
for periodic
square wave
infinitely large number
of sine waves is
required
24

25. xn }, n = 0, ±1, ±2, . . . , ±∞ is given by
Fourier Transform
∞
he Discrete Time Fourier Transform (DTFT) n e−jnωT .
X (ω) = x of a sampled sequen
n }, n = 0, ±1, ±2, . . . , ±∞ is given byn=−∞
The Fourier transform is −jnωT
∞
an equation to
X (ω) = xn e .
calculate the frequency, amplitude and
n=−∞
phase of each sine wave needed to
∞
make up any X (ω)signal x(t)e
given = : −jωt
dt.
∞
−∞
X (ω) = x(t)e−jωt dt.
here T is the sampling interval.
−∞
(recall from 1B Signal
here T is the sampling interval.
and Data Analysis)
a) Explain how the problems with computing the DT
a) Explain how the problems with computing the DTFT on a digit
computer can be overcome by choosing a uniform 25
computer can be overcome by choosing a uniformly spaced gr

26. Prism Analogy
Analogy:
a prism which splits
white light into a
spectrum of colors
White Spectrum
light of colours
white light consists of all
frequencies mixed
together
Signal Fourier Spectrum
the prism breaks them
apart so we can see the
Transform
separate frequencies
26

27. Signal Spectrum
Every signal has a frequency spectrum.
• the signal defines the spectrum
• the spectrum defines the signal
We can move back and forth between
the time domain and the frequency
domain without losing information
27

28. Time domain / Frequency domain
• Some signals are easier to visualise in the
frequency domain
• Some signals are easier to visualise in the
time domain
• Some signals are easier to define in the time
domain (amount of information needed)
• Some signals are easier to define in the
frequency domain (amount of information
needed)
Fourier Transform is most useful
tool for DSP
28

29. Fourier Transforms Examples
signal spectrum
cosine t ω
added higher
frequency t ω
component
Back to our sound recognition problem:
peaks correspond to
t the resonances of
sound /a/ ω
the vocal tract shape
as in father in logarithmis units of dB
they can be used to
sound /i/ t differentiate between
as in see ω
sounds
in logarithmis units of dB
29

30. n = 0, ±1, ±2, . . . , ±∞ is given=
X (ω) by xn e−jnωT .∞ X (ω) =Timex(t)e Transform (DTFT) of a sampled sequen
1. The Discrete Fourier dt.
{xn }, n = 0, ±1, ±2, . . . , ±∞ is given by
∞ n=−∞X (ω) = xn e−jnωT .−∞
X (ω) = xn e−jnωT . n=−∞ ∞
.
Discrete Time Fourier Transform (DTFT)
3.
n=−∞ X (ω) = xn e−jnωT .
n=−∞
2. ∞ ∞
X (ω) = x(t)e −jωt 2.
dt. xs (t) = x(t) δ(t − nT )
What = x(t)esampled signal? dt.
about X dt. = x(t)e−jωt
∞ −∞
∞ n=−∞ ∞
X (ω) (ω) −jωt X (ω) = x(t)e−jωt dt.
. 4. −∞
−∞ −∞
∞
∞ 3.
The DTFT is= x(t)
xs (t) defined as the nT ) Xs transform sof the sampled
δ(t − Fourier (ω) = x (t)e−jωt dt ∞
3. signal. Define the sampled signal in the usual way:
n=−∞ −∞ xs (t) = x(t) δ(t − nT )
∞
n=−∞
xs (t) = x(t) δ(t − nT ) ∞
. 5.n=−∞ x (t) = x(t)
s 4. δ(t − nT )
∞
n=−∞ ∞ ∞ ∞
Take Fourier transform directly =
Xs (ω) = xs (t)e Xdt(ω)
s
−jωt
x(t) δ(t (ω)nT )e−jωt dt dt
Xs − = xs (t)e−jωt
4. ∞ −∞ −∞ n=−∞ −∞
Xs (ω) = xs (t)e
−jωt
dt 1. The Discrete Time Fourier Transform (DTFT) of a sam
. 5. {x }, n = 0, ±1, ±2, . . . , ±∞ is given by
∞
6.
−∞ n
∞ Xs (ω) =
∞ xs (t)e−jωt dt (ω) =
∞
X
∞
x(t)
∞
δ(t∞ nT )e−jωt dt
−
s
Xs (ω) = x(t) δ(t − nT )e−jωt dt
−∞ Xs (ω) = x(nT )eX (ω) =
−∞
−jωnT
n=−∞ xn e
−jnωT
.
∞ −∞ n=−∞ n=−∞ n=−∞
∞
5. Xs (ω) = x(t) )e−jωt6.
δ(t − nT the sampling interval.
where T is dt
. n=−∞
2.
using the “sifting property of the δ-function to reach the last line
∞
−∞
∞
−jωnT
Xs (ω) = x(nT )e
∞
(a) Explain how
∞
the
problems with computing the DTFT on a digital
∞ n=−∞ 30
Xs (ω) (ω) = x(nT )e−jωnT δ(t − nT )e−jωt dtX (ω) =
Xs = x(t) be overcome by choosing a x(t)e−jωt dt.
computer can where T is the sampling interval. uniformly spaced grip
n=−∞
∞ n=−∞ −∞
of −∞ frequencies covering the range of ω from 0 to 2π/T and
N

31. 4.
∞
Discrete Time Fourier(ω) = xs (t)e−jωt dt
Xs Transform – Signal Samples
−∞
5.
1. The Discrete Time Fourier ∞ Transform (DTFT) of a sampled sequence
Note that this expression known as DTFT is a periodic function of
∞
the frequency usuallyXswritten as is given by − nT )e−jωt dt
{xn }, n = 0, ±1, ±2,(ω) ,=
. . . ±∞ x(t) δ(t
−∞ n=−∞
∞
X (ω) = xn e−jnωT .
6. n=−∞
∞
The signal sample valuesXs (ω) =expressed in terms of DTFT by
may be x(nT )e−jωnT
2.
noting that the equation above has n=−∞ the form of Fourier series (as a
function of ω) and hence the sampled signal can be obtained directly
7.
∞
as X (ω) = x(t)e−jωt dt.
2π
1 −∞
xn = X(ω)e+jnωT dωT
2π
3. 0
where T is the sampling interval. ∞
[You can show this for yourself by first noting that (*) is a complex Fourier
xs (t) = x(t) δ(t − nT )
series with coefficients however it is also covered in one of Part IB
(a) Explain how the problems with computing the DTFT on a digital
Examples Papers] n=−∞
computer can be overcome by choosing a uniformly spaced grip 31
4. of N frequencies covering the range of ω from 0 to 2π/T and
truncating the summation ∞ the expression above to just N data
in

32. Computing DTFT on Digital Computer
1. The Discrete Time Fourier Transform (DTFT) of a sampled sequence
{xn }, n = 0, ±1, ±2, . . . , ±∞ is given by
The DTFT
∞
X (ω) = xn e−jnωT .
n=−∞
2. expresses the spectrum of a sampled signal in terms of
the signal samples but is not computable on a digital
∞
X (ω) =
computer for two reasons: x(t)e−jωt dt.
−∞
3. 1. The frequency variable ω is continuous.
2. The summation involves an infinite number of
∞
samples. xs (t) = x(t) δ(t − nT )
n=−∞
4. 32
∞

33. Overcoming problems with computing DTFT
The problems with computing DTFT on a digital
computer can be overcome by:
Step 1. Evaluating the DTFT at a finite
collection of discrete frequencies.
no undesirable consequences, any
frequency of interest can always be
included in the collection
Step 2. Performing the summation over a
finite number of data points
does have consequences since
signals are generally not of finite duration
33

34. of discrete frequencies evaluating theundesirable a finite
discrete frequencies without any undesirable consequences; sec-
without any DTFT at consequences; sec-
ofdiscrete frequencies the frequency variable ω is consequences; sec-
of can be overcome by without any undesirable continuouscollection
ital computer: first, which
ond, thebe overcome involves infinite undesirablesamples which sec-
the summationby evaluatingany numberatof consequences; can
summation involves infinite DTFT ofof samples which can
involves infinite number a finite collection can
ond,discrete frequencies without the
ond, can summation
of the number samples which
be overcome frequencies without summation over a finite sec-
be overcome by performing the summation over finite number
overcome on a dig-
of the summation involves infinite number consequences; number
problems with calculating DTFTbyperforming the any undesirableof samples which can
beond, discreteby performing thesummation over aafinite number
frequency variable ωThepoints,which does have does number consequencessince sig-
ofbeis continuous whichdoes have Transform (DFT) since sig-
data points, which Fourier does have consequences number
Discrete
of datapoints, by performing the doeshave consequences since sig-
of data ond, the summation involves infinite have ofover a finite can
overcome which does have summation samples which
aluating the DTFTof beaovercome bynot of finitethe does haveIf overchooseaauniformly
nalsare finite collectiondoes have summation consequencesnumber
are generally performing duration. we a finite uniformly
nalsataregenerallywhich of finiteduration. If we choose a since sig-
nalsdata generally not of finite duration. If we choose uniformly
points, not
spaced grippoints, which does haveduration.the we chooseωa uniformly
spaced grip of N frequencies covering the range of since sig- 0 to
of data of N sec-
grip of N not of finite does have consequences ω from to
without any undesirable consequences; frequencies covering the range ofof ωfrom 00to
nals are generally frequencies covering If range
spaced are generally not of finite duration. If we choose a uniformly from
nals
The 2π/T andgrip whichthe summation in theexpression ω fromto just
discreteand truncate frequencies covering However, since the to to
2π/T andtruncate can summation in the expression from 0 tojust
2π/T set of of N the chosen is arbitrary. expression above
spaced truncate the
olves infinite number of samplesfrequencies summationin thethe range ofabove to 0 just above
spaced grip of N frequencies covering the range of ω
ming the summation2π/T points generallytheDFT: uniformly spaced grid of N to just
N 2π/T finite number
is over apoints we getthe DFT:
periodic truncate the DFT:
data points we get choose a
DTFTNdata andwe we getthe summation in the expression above
N data and truncate the summation in the expression above to just
oes have does frequencies data pointswerangethe DFT:0 to 2π. If the summation is then
N N covering the sig- ωT from
have consequences since get the DFT:
data points we get
f finite duration. If we to just Nadata 2π
truncated choose uniformly weN −1 the DFT np
points NN −1
2π get
2π p) = −1 x −j 2π2πnp , p = 0, . . . , N − 1
2π
X = X( p) = N −1 e e−j Nnp
Xp ω = X(
X(
Xpp =from 2πp)= N −1xn e−jN2π , ,pp= 0, . . . , , N− 11
xn n N
= 0, . N −
encies covering the range of N to n=0
02π
N p) n=0 x e−j
N p) = 2π np
summation in the expression above to N XXp = X( just = n=0 xn e−j NNnp , ,p p = 0,. . . N N 1 1
p = X( n = 0, . , . , − −
N n=0
n=0
he DFT: (c)
(c)
(c) inverse DFT can be used to obtain the sampled signal values from the
The
N −1
DFT: multiply each side byinterested in over p=0 to the presence of single
Assume you are N −1 N −1 sum detecting the presenceNof a a single
(c)(c) −1 and
Assume you areinterested in detecting N-1
N−1
N continuous-wave−1 N −1
N −1 N −1 N −1
N −1 N −1 j2π (q−n)p
−1
−j 2π np N −1 N sinusoidal tone in a j2π pq N −1 data sequence.
sinusoidal tone np atime-domain Ndata 2π2π(q−n)p
2π
= xn e N , p =continuous-wave 2π
0, . .X,pejNNpq 1 =
ej =
2π pq −j2π in 2π2π
2πN np
e time-domainn epesequence.
xep −j of ee N pq=
ne
−j j N = x
xn
−1
Xp N −pqthe 64-point xn epp NNnp{xjN}pqcan=be computedeppN the
. j 2πN = jj N(q−n)p
Show Xphow
e
Show howthe 64-pointxn DFT of {x } can bexcomputed at the
n
N
at
p=0 n=0 DFT
n=0 n
p=0
p=0 p=0ofn=0 n n=0 p=0
n=0 {xp=0was f =
p=0 discrete frequency n=0
p=0 30kHz if the sampling rate of p=0
n=0 n} s
discrete frequency of 30kHz if the sampling rate of {xn } was fs =
128kHz. Determine the total number of complex multiplications
(d) Assume you are complex in detecting complex multiplications
(d) Assume Determine the total detecting the presence
128kHz.you[20%].interested in number of the presence of a single
(d) Assume you are ofinterested in detecting the presence ofofa asingle
required. are
Orthogonality property interested exponentials single
required. [20%]. sinusoidal tone in time-domain data sequence.
continuous-wavesinusoidal tone in =a15 since 15 12864data sequence.
continuous-wave sinusoidal tone p aatime-domainkHz = 30kHz
−1 2π
−j N np j pq2π
N −1 N −1
continuous-wave (q−n)p
Answer. 302π jNkHz corresponds to in time-domain kHz sequence. data
xn ep Show uniformly 64-point DFT of {x 0n} can be 12864 the= at the
= Answer. the 64-point NDFT ofandfrequenciesbe computed 30kHz
how the 64-point of n=q
e N Show xhow 30 pkHz corresponds =ofp{xn15 can be 15computed at the
e
Show how the spaced grip DFTto64 {x}otherwise computed at the
(a n is if N since
= } can covering range
n
discretefrequency 2π/T30kHzNthe 64sampling rateofof= nn }the range=
discrete from 0spaced30kHz ifif thesampling srate1/T {x 128kHz ffss =
(a of ω frequency of grip considered, frequencies covering } was) f
uniformly to of 30kHz if = sampling = of {x n was
of
0 n=0 p=0
discrete frequency of is the where f rate {x} was =
34
s
of and one obtains the total number of complex multiplications
ω from 0 to 2π/Tthe considered, wherecomplex multiplications
128kHz. Determine
128kHz. Determine is total number of fs = 1/T = 128kHz )
sted in detecting 128kHz. Determine the total number 2π complex multiplications
the presence of a single
and one obtains 63 of
required. [20%].
required. [20%].
required. [20%]. X = x e−j 64 n15 ,

35. The Discrete Fourier Transform Pair
35

36. uncate the summation inN expression above to just
N −1 the
n=0 2π N data points we get the DFT:n=0
−j N np
ts wexget the DFT: 0, . . . , N − 1 N −1
ne ,p=
n=0 2π 2π N −1 −j 2π np
) 2π Properties = Nthe= = xnxne−j 2π np,pp= 0, .. .. ..,,N − 1 1 (DFT)
N −1Xp = X( X(
X2πof p) p)
p
e N N , = 0,
Discrete Fourier Transform
N−
= X( p) = N −1 pN ...
xn e−j N np , N −1 0,n=0 , N − 1
= n=0
−jN np j 2π pq
2π
NN−1 n=0 N −1 N −1 2π (q−n)p
jN N −1 N −1
1n ep(c)2π e N j 2π pq −1 xN −1
x −j np (c)2π =N e(q−n)p N
2π p −j 2π np j 2π pq j 2π (q−n)p
xn ep N Xp e N
e j N pq
= =xn n p=0 NN −1n ep
N −1 n=0
j
e−1
Np
x e N N −1 N −1 xn
= ep N
p=0 N −1 n=0 2πN −1 N −1
p=0 n=0
p=0
j 2π p j N pq
−j 2π
−j p npnpje2πN pq =
2π N j 2π
N −1 N −1 j 2π (q−n)p
n=0 ep N2π (q−n)p
jN p=0
Xe Xe=
N −1 N −1 N
pq
Xp+N • Xis periodic, for each
= p p−j 2π np 2π
= xnN −1 e N pq =
x ep
N −1 n
eN xn
x
j 2π (q−n)p n=0 n p=0
ep
pq X
= p+N =p=0xn e
Xp p=0 N
e = p x
j N pq p=0 n=0
ep N
p p=0 n=0 n X
p+N =X
Xp+N = Xp p
n=0 p=0
p=0 n=0 n=0 p=0
Xp+N = Xp
) (d) periodic, for
• = x is Xp+N = Xp each n
xn+N
xn+N
(d) =x n
xn+N = xn
(e) x data
xxn+N = xn
n+N = xn
• for real n+N = xn
Xp =(e)−p = XN −p
) X ∗ ∗
= X−p = XN −p
∗ ∗
Xp = X−p = XN −p
∗ ∗
ted in detecting the presence interestedX−pdetecting the presence of a single
(f) should check ∗thatofXppsingle ∗= XNX results from first principles]
a
Assume you are you = single these ∗
∗ ∗
[You = the presence X can X of a show
d in detecting X−p = Xdata sequence. −p = −pN −p
= in
Xpcontinuous-wave sinusoidal tone in a time-domain data sequence.
∗
dal tone in a time-domain N −p
lDFT of {xntime-domain 64-point DFT of {xn } can be computed at single
tone in a } can how computed sequence.
Show be the data at the
(f) Assume you are interested in detecting the presence of a the
DFT interested in detecting n }of 30kHz= thea a singlerate of {xn } was fs = a single
kHz if ofcontinuous-waveinterestedtone the sampling the presence of
)are the {xn } canrate ofsinusoidal fsatif in time-domain data sequence.
Assume you are computed in detecting
sampling be {x the presence of
discrete frequency was
wave sinusoidalof complex time-domain data sequence.
totalthe sampling the asinusoidaltotalof = n } of complex multiplications sequence.
number 128kHz.inDeterminen }DFT number time-domain at the
tone
continuous-wave 64-pointthewas fs {x a can be computeddata36
Hz if Show how rate of {x tone in
multiplications
he 64-point DFT of {xn } can be computed at the
otal number required. sampling rate of {x of was n }=rate of kHzcomputed at the
uency of of the [20%].multiplications
Show30kHz ifthe 64-point DFT n } {x fs can 128 {xn } was fs =
discrete frequency of 30kHz if the sampling
how complexkHz be
128

37. DFT Interpolation
37

38. Zero padding
38

39. Padded sequence
39

40. Zero-padding
40

41. Zero-padding
just visualisation, not additional information! 41

42. Circular Convolution
circular convolution
42

43. Example of Circular Convolution
Circular convolution of x1={1,2,0} and x2={3,5,4}
clock-wise anticlock-wise
1
3
5 4
0 2 folded sequence
y(0)=1×3+2×4+0×5 y(1)=1×5+2×3+0×4 y(2)=1×4+2×5+0×3
1 1 1
3 5 4
0 spins 1 spin 2 spins
…
5 4 4 3 3 5
0 2 0 2 0 2
x1(n)x2(0-n)|mod3 x1(n)x2(1-n)|mod3 x1(n)x2(2-n)|mod3 43

44. Example of Circular Convolution
clock-wise anticlock-wise
44

45. Example of Circular Convolution
45

46. Standard Convolution using Circular Convolution
46

47. Example of Circular Convolution
clock-wise anticlock-wise
1
3
0 2 5 0
4
folded sequence
y(0)=1×3+2×0+0×4+0×5 y(0)=1×5+2×3+0×0+0×4 0 y(0)=1×4+2×5+0×3+0×0
1 1 1
3 5 4
0 spins 1 spin 2 spins
0 5 0 2 0 4 3 2 0 0 5 2
4 0 3
…
x1(n)x2(0-n)|mod3 x1(n)x2(1-n)|mod3 x1(n)x2(2-n)|mod3 47
0 0 0

48. Standard Convolution using Circular Convolution
48

49. Proof of Validity
Circular convolution of the padded sequence corresponds to the standard
convolution
49

50. Linear Filtering using the DFT
FIR filter:
Frequency domain equivalent:
DFT and then IDFT can be used to compute standard convolution
product and thus to perform linear filtering.
50

51. Summary So Far
• Fourier analysis for periodic functions focuses on the
study of Fourier series
• The Fourier Transform (FT) is a way of transforming
a continuous signal into the frequency domain
• The Discrete Time Fourier Transform (DTFT) is a
Fourier Transform of a sampled signal
• The Discrete Fourier Transform (DFT) is a discrete
numerical equivalent using sums instead of integrals
that can be computed on a digital computer
• As one of the applications DFT and then Inverse DFT
(IDFT) can be used to compute standard convolution
product and thus to perform linear filtering 51

52. Fast Fourier Transform (FFT)
Elena Punskaya
www-sigproc.eng.cam.ac.uk/~op205
Some material adapted from courses by
Prof. Simon Godsill, Dr. Arnaud Doucet,
Dr. Malcolm Macleod and Prof. Peter Rayner
52

53. Computations for Evaluating the DFT
• Discrete Fourier Transform and its Inverse can be
computed on a digital computer
• What are computational requirements?
53

54. Operation Count
How do we evaluate computational complexity?
count the number of real multiplications and
additions
by ¨real¨ we mean either fixed-point or floating point
operations depending on the specific hardware
subtraction is considered equivalent to additions
divisions are counted separately
Other operations such as loading from memory, storing in
memory, loop counting, indexing, etc are not counted
(depends on implementation/architecture) – present
overhead 54

55. Operation Count – Modern DSP
Modern DSP:
a real multiplication and addition is a single machine
cycle y=ax+b called MAC (multiply/accumulate)
maximum number of multiplications and additions must
be counted, not their sum
traditional claim ¨multiplication is much more time-
consuming than addition¨ becomes obsolete
55

56. Discrete Fourier Transform as a Vector Operation
Again, we have a sequence of sampled values x n and
transformed values Xp , given by
Let us denote:
Then:
x
56

57. Matrix Interpretation of the DFT
This Equation can be rewritten in matrix form:
x
complex
as follows: numbers
can be
X0 0 0 0
… 0
x0 computed
X1 0 1 2
…
N-1
x1 off-line
= and
… … …
…
…
…
…
stored
N-1 2(N-1) (N-1) 2
XN-1 0
… x N-1
N x N matrix 57

58. DFT Evaluation – Operation Count
Multiplication of a complex N x N matrix by a complex
N-dimensional vector.
No attempt to save operations
N2 complex multiplications (¨x¨s)
N(N-1) complex additions (¨+¨s)
First row and first column are 1 however
2
save 2N-1 ¨x¨s, (N-1) complex ¨x¨s & N(N-1) ¨+¨s
Complex ¨x¨: 4 real ¨x¨s and 2 real ¨+¨s;
complex ¨+¨ : 2 real ¨+¨s
2
Total: 4(N-1) real ¨x¨s & 4(N-0.5)(N-1) real ¨+¨s 58

59. DFT Computational Complexity
Thus, for the input sequence of length N the number of
arithmetic operations in direct computation of DFT is
proportional to N 2
.
For N=1000, about a million operations are needed!
In 1960s such a number was considered prohibitive
in most applications.
59

60. Discovery of the Fast Fourier Transform (FFT)
When in 1965 Cooley and Tukey ¨first¨ announced
discovery of Fast Fourier Transform (FFT) in 1965 it
revolutionised Digital Signal Processing.
They were actually 150 years late – the principle of the
FFT was later discovered in obscure section of one of
Gauss’ (as in Gaussian) own notebooks in 1806.
Their paper is the most cited mathematical paper ever
written.
60

61. The Fast Fourier Transform (FFT)
The FFT is a highly elegant and efficient algorithm,
which is still one of the most used algorithms in speech
processing, communications, frequency estimation, etc
– one of the most highly developed area of DSP.
There are many different types and variations.
They are just computational schemes for computing the
DFT – not new transforms!
Here we consider the most basic radix-2 algorithm
which requires N to be a power of 2.
61

62. FFT Derivation 1
Lets take the basic DFT equation:
Now, split the summation into two parts: one for even n and one for odd n:
Take
outside
62

63. FFT Derivation 2
Thus, we have:
63

64. FFT Derivation 3
Now, take the same equation again where we split the summation
into two parts: one for even n and one for odd n:
And evaluate at frequencies p+N/2
64

65. FFT Derivation 4
simplified
65

66. FFT Derivation 5
=
=
66

67. FFT Derivation - Butterfly
Or, in more compact form:
(‘Butterfly’)
67

68. FFT Derivation - Redundancy
68

69. FFT Derivation - Computational Load
69

70. Flow Diagram for a N=8 DFT
Input: Output:
70

71. Flow Diagram for a N=8 DFT - Decomposition
71

72. Flow Diagram for a N=8 DFT – Decomposition 2
72

73. Bit-reversal
73

74. FFT Derivation Summary
• The FFT derivation relies on redundancy in the calculation
of the basic DFT
• A recursive algorithm is derived that repeatedly rearranges
the problem into two simpler problems of half the size
• Hence the basic algorithm operates on signals of length a
power of 2, i.e.
(for some integer M)
• At the bottom of the tree
we have the classic FFT
`butterfly’ structure
74

75. Computational Load of full FFT algorithm
The type of FFT we have considered, where N = 2M, is
called a radix-2 FFT.
It has M = log2 N stages, each using N / 2 butterflies
Complex multiplication requires 4 real multiplications
and 2 real additions
Complex addition/subtraction requires 2 real additions
Thus, butterfly requires 10 real operations.
Hence the radix-2 N-point FFT requires 10( N / 2 )log2 N
real operations compared to about 8N2 real operations
75
for the DFT.

76. Computational Load of full FFT algorithm
The radix-2 N-point FFT requires 10( N / 2 )log2 N real
operations compared to about 8N2 real operations for the
DFT.
This is a huge speed-up in typical applications, where N is
128 – 4096:
Direct DFT
FFT
76

77. Further advantage of the FFT algorithm
Input Output
77

78. The Inverse FFT (IFFT)
The IDFT is different from the DFT:
• it uses positive power of instead of negative ones
• There is an additional division of each output value by N
Any FFT algorithm can be modified to perform the IDFT by
• using positive powers instead of negatives
• Multiplying each component of the output by 1 / N
Hence the algorithm is the same but computational load
increases due to N extra multiplications 78

79. Many types of FFT
The form of FFT we have described is called “decimation in time”; there
is a form called “decimation in frequency” (but it has no advantages).
The "radix 2" FFT must have length N a power of 2. Slightly more efficient
is the "radix 4" FFT, in which 2-input 2-output butterflies are replaced by 4-
input 4-output units. The transform length must then be a power of 4
(more restrictive).
A completely different type of algorithm, the Winograd Fourier Transform
Algorithm (WFTA), can be used for FFT lengths equal to the product of a
number of mutually prime factors (e.g. 9*7*5 = 315 or 5*16 = 80). The
WFTA uses fewer multipliers, but more adders, than a similar-length FFT.
Efficient algorithms exist for FFTing real (not complex) data at about 60%
the effort of the same-sized complex-data FFT.
The Discrete Cosine and Sine Transforms (DCT and DST) are
79
similar real-signal algorithms used in image coding.

80. Applications of the FFT
There FFT is surely the most widely used signal processing
algorithm of all. It is the basic building block for a large
percentage of algorithms in current usage
Specific examples include:
• Spectrum analysis – used for analysing and detecting
signals
• Coding – audio and speech signals are often coded in the
frequency domain using FFT variants (MP3, …)
• Another recent application is in a modulation scheme called
OFDM, which is used for digital TV broadcasting (DVB) and
digital radio (audio) broadcasting (DAB).
• Background noise reduction for mobile telephony, speech
and audio signals is often implemented in the frequency
domain using FFTs 80

81. Practical Spectral Analysis
• Say, we have a symphony recording over 40 minutes
long – about 2500 seconds
• Compact-disk recordings are sampled at 44.1 kHz and
are in stereo, but say we combine both channels into a
single one by summing them at each time point
• Are we to compute the DFT of a sequence one hundred
million samples long?
a) unrealistic – speed and storage
requirements are too high
b) useless – we will get a very much noiselike
wide-band spectrum covering the range from 20
Hz to 20 kHz at high resolution including all
notes of all instruments with their harmonics 81

82. Short-Time Spectral Analysis
• We actually want a sequence of short DFTs, each
showing the spectrum of a signal of relatively short
interval
• If violin plays note E during this interval – spectrum
would exhibit energies at not the frequency of note E
(329 Hz) and characteristic harmonics of the violin
• If the interval is short enough – we may be able to
track not-to-note changes
• If the frequency resolution is high enough – we will be
able to identify musical instruments playing
Remember: our – ear – excellent spectrum analyser?
82

83. Short-Time Spectral Analysis – Computational Load
• If we take a time interval of about 93 milliseconds
(signal of length 4096) the frequency resolution will
be 11 Hz (adequate)
• The number of distinct intervals in the symphony is
40 x 60/0.093 - about 26,000
• We may choose to be conservative and overlap the
intervals (say 50%)
• In total, need to compute 52,000 FFTs of length 4096
the entire computation will be done in
considerably less time than the music
itself
Short-time spectral analysis also use windowing (more on
this later) 83

84. Case Study: Spectral analysis of a Musical Signal
Sample rate is
10.025 kHz
(T=1/10,025 s)
Extract a short segment
Load this into Matlab as a vector x
Take an FFT, N=512
X=fft(x(1:512));
Looks almost
periodic over short time 84
interval

85. Case Study: Spectral analysis of a Musical Signal, FFT
Note Conjugate symmetry
Symmetric as data are real:
Symmetric Anti-Symmetric
85

86. Case Study: Spectral analysis of a Musical Signal - Frequencies
86

87. The Effect of data length N
Low
N=32 resolution
FFT
N=128 FFT
N=1024 FFT High
resolution
87

88. The DFT approximation to the DTFT
DTFT at frequency : DFT:
• Ideally the DFT should be a `good’ approximation to the DTFT
• Intuitively the approximation gets better as the number of data
points N increases
• This is illustrated in the previous slide – resolution gets better as
N increases (more, narrower, peaks in spectrum).
• How to evaluate this analytically?
View the truncation in the summation as a multiplication by a
rectangle window function
88

89. Analysis
89

90. Pre-multiplying by rectangular window
90

91. Spectrum of the windowed signal
91

92. Fourier Transform of the Rectangular Window
92

93. Rectangular Window Spectrum
N=32 Central `Lobe’
Sidelobes
93

94. Rectangular Window Spectrum – Lobe Width
N=32
N=16
N=8
N=4
Lobe width inversely
Proportional to N
94

95. Now, imagine what happens when the sum of two frequency components is DFT-ed:
The DTFT is given by a train of delta functions:
Hence the windowed spectrum is just the convolution of the window spectrum with
the delta functions:
95

96. DFT for the data
Both components
separately
Both components
Together
96
ωΤ

97. Brief summary
• The rectangular window introduces broadening of any frequency
components (`smearing’) and sidelobes that may overlap with other
frequency components (`leakage’).
• The effect improves as N increases
• However, the rectangle window has poor properties and better choices
of wn can lead to better spectral properties (less leakage, in particular)
– i.e. instead of just truncating the summation, we can pre-multiply by a
suitable window function wn that has better frequency domain
properties.
• More on window design in the filter design section of the course – see
later
97

98. Summary so far …
• The Fourier Transform (FT) is a way of transforming
a continuous signal into the frequency domain
• The Discrete Time Fourier Transform (DTFT) is a
Fourier Transform of a sampled signal
• The Discrete Fourier Transform (DFT) is a discrete
numerical equivalent using sums instead of integrals
that can be computed on a digital computer
• The FFT is a highly elegant and efficient algorithm,
which is still one of the most used algorithms in
speech processing, communications, frequency
estimation, etc – one of the most highly developed
area of DSP
98

99. Thank you!
99