1. Compressed Sensing:
Introduction and Apps
Achuta Kadambi
Camera Culture, MIT

2. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.

3. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.
• Shannon-Nyquist  Bandlimited signals can be
sampled/reconstructed

4. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.
• Shannon-Nyquist  Bandlimited signals can be
sampled/reconstructed
• Rank-constrained Optimization  Low Rank signals can be
interpolated (Netflix Problem)

5. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.
• Shannon-Nyquist  Bandlimited signals can be
sampled/reconstructed
• Rank-constrained Optimization  Low Rank signals can be
interpolated. (Netflix Problem)
• Compressed Sensing  Sparse signals can be undersampled and
recovered.

6. Outline of this talk.
• Compressed Sensing overview.
• Very brief explanation on the why and how of Compressed Sensing.
• ‘Apps’ that use compressed sensing.
• Practical strategies for implementation (e.g. pseudocode, libraries).

7. Motivation: JPEG Compression
Our visual system is less sensitive to high
(spatial) frequency detail. Can we throw away
these frequencies and retain a similar image?
This is the intuition behind JPEG.
Spatial Frequency E.g.:
High  Hair, Blades of Grass, etc.
Low  Sky, Skin, etc.
Compressed Sensing: If we are going to throw
away stuff … why spend time acquiring it?

8. E.g.

9. Wired Magazine: “Fill in the Blanks…”

10. 1D Implementation in L1Magic
Step 1: The original signal and its Fourier Transform.
Original Signal (N = 256)
Spectrum

11. Implementation in L1Magic
Step 2: The subsampled signal
Red Entries (80 samples) are observed.
Blue Entries (176 samples) must be
recovered.
That means we observe only 30% of the
original signal.

12. Implementation in L1Magic
Step 3: Exact Recovery of the Signal.
Original Signal (N = 256)
Reconstruction (N = 256)

13. L1Magic for Images
Original Image:
1 million pixels
Reconstruction:
from 100,000
random measurements.

14. Goes back to Fourier

15. Fourier Transform
Intuition: Projection, or Inner Product, of Signal with Trigonometric Functions.

16. Sparsity goes back to Fourier (circa
1800)
Superposition of Sinusoids
Original Time Domain Function
Frequency Domain Representation

17. Discrete Fourier Transform
Example DFT:
Time Signal is a Delta. Spectrum is Broadband.

18. DFT in Matrix Form

19. Nyquist-Shannon Sampling Theorem
In Shannon’s words:
How to Reconstruct? (Interpolation)
Compressive Sensing: Can we do better?

20. Inverse Problem
Example 1: Sinc Interpolation. Given the Data (a sufficiently sampled
signal), how can we obtain the original signal?
Example 2: Blurry Photos. Given a Blurry Photo, from a Camera, how can
we go back to the original, sharp image?
Example 3: Given a discrete time signal, how can we obtain its discrete
spectrum? **DFT problem is a Linear Inverse Problem

21. Solving the DFT Problem
Done?

22. Solving the DFT Problem via
Optimization
Done?
Loss Function

23. Solving the DFT Problem via
Optimization
Done?
PseudoInverse:
Minimize MSE

24. Constraining our Solution via
Regularization
We can go beyond loss function,
e.g., Tikhonov Regularization
Additional Term allows for some prior on original signal. For
instance if Tikhonov Matrix is a first order difference, then you
are biasing x toward smooth solutions.
Linked to the Lagrange problem, as well as Maximum A
Posteriori from probability, and Weiner filter from Sig proc.

25. Compressed Sensing

26. Compressed Sensing Structure
• Underdetermined system. y=Ax.
•
•
•
•
Y is m-dimensional sampled vector
A is mxn matrix
X is n-dimensional original vector.
And m << n
y
A
x

27. Simply Solving y=Ax not good enough
• This gives you an affine space with many solutions to y=Ax.
• So we must constrain our problem to look for the sparse solution to
y=Ax.

28. Occams Razor
Occam's Razor: among otherwise equal explanations,
the simplest is best

29. Occams Razor
Occam's Razor: among otherwise equal explanations,
the simplest is best
CS Occam's Razor: among otherwise equal solutions, the sparsest is best
Unfortunately, this optimization is not tractable

30. Geometric Property of Norms

32. The l1 Optimization Problem

33. RIP/Spark/Coherence
• The sensing matrix A must be carefully chosen.
• For compressed sensing to work, the matrix A must satisfy the
Restricted Isometry Property (RIP):
• Calculating RIP is NP-hard. We can work with easier quantities than
the RIP, such as spark and mutual incoherence.

34. App1: Single-Pixel Camera

35. App1: Single-Pixel Camera
Design
Advantage: A
MP camera
with just a
single-pixel.

36. App2: Single-Pixel THz imaging.
Design Advantage: CS allows for
single-pixel THz sensors, which are
much easier to fabricate than pixel
array.
In general, you can buy amazing
things at single-pixel level, e.g.,
picosecond detectors, thermal IR
sensor, etc.
Chan et al. Applied Physics
2008

37. App2: Monitoring Breathing via
Smartphone
Very similar to
OMP, DFT
formulation.
Basically finding
sparse spectral
components that
characterize the
audio signal of
breathing.
Design Advantage: Compressed sensing
allows for low power acquisition and
reduced streaming.
Oletic, Skrapec, and Bilas MobiHealth 2012

38. App4: Biometrics … Face Recognition
Design Advantage: Using
compressed sensing to handle
the small sample size problem.
Before, the number of samples in
the database Is less than the
degrees of freedom of each
sample.

39. App5: Fast MRI
Design Advantage:
Less samples means
less time for an MRI
Scan, which means
less time a sick or
disabled patient lies
in the scanner.

40. App5: Fast MRI

41. App6: Compressive Sensing of High
Speed Periodic Videos
Design Advantage: Exploit
sparsity of Periodic Videos to
obtain a high speed video
without using a high speed
camera.
Veeraraghavan A, Reddy
D, Raskar R. IEEE PAMI

42. App7: Compressive Light Field
Photography
Marwah, Wetzstein, Bando, Ra
skar. ACM SIGGRAPH 2013.
Design Advantage:
Obtain High-Resolution
Light Field photos by
placing a coded mask in
front of the sensor.

43. App8: Sparsity-Induced Time of Flight
Cameras
Kadambi et al. ACM SIGGRAPH Asia
E.g., Light Sweep Movies from Refael’s talk last week. Goal is to
obtain a well-conditioned deconvolution problem.
Design Advantage: Deconvolve to obtain
bounces of light and construct a light
sweep video.

44. App9  YOUR App!
Design Advantage: <insert here>

45. Practical Strategies
Many libraries are available for C++/Matlab/etc.
• Recommended: L1magic (http://users.ece.gatech.edu/~justin/l1magic/)
• SPGL1 (http://www.cs.ubc.ca/~mpf/spgl1/)
• CVX (http://cvxr.com/cvx/)
• On phone, nothing exists yet, but you can use Efficient Java Matrix Library
(EJML) to implement solely in linear algebra.

46. L1Magic Pseudocode
For 1D signal.
x = original_signal;
R = randn(m,n);
A = orth(R’)’;
// read in your original signal.
// create a random matrix of dimension mxn
// sensing matrix with orthogonal columns.
y = A*x;
// create subsampled signal y of only m entries.
X0 = A’*y;
// Initial guess by taking matrix inverse.
x_hat = l1eq_pd(x0, A, [], y, 1e-3); // run l1 solver.
norm(x_hat – x);
// error; should be zero in ideal case

47. Take-home Messages
• Opportunity to integrate cutting-edge mathematical techniques into
your camera apps.
• Compressed Sensing is lightweight in terms of coding. The key is
correctly identifying the sparsity in your engineering problem.
• Not all signals are equal… find a weakness, e.g., sparsity/rank, and
exploit it.
• Exploit in Hardware, Exploit in Software…