Seminar

1. Alternating Direction Method
for Image Restoration
Jivnesh Dongre
16EC65R12
1

2. Introduction
 Image restoration: Operation of taking
corrupt image and estimating clean, original
image.
 Image restoration is performed by reversing
the process that blurred the image
 Objective of image restoration: Reduce
noise and recover resolution loss
2

3. Introduction
 Image restoration applications:
Science and engineering such as medical and astronomical
imaging, film restoration, image and video coding
 Original image corrupted by:
Invariant blur, build in nonlinearities and additive Gaussian white
noise
Objective function
Nonlinear least square
(NLS)data fitting term
Total variation(TV)
regularization term
3

4.  Nonlinear image degraded model:
𝑔 = 𝑠 𝐻𝑓𝑡𝑟𝑢𝑒 + 𝑛
where 𝑔=observed image, 𝑓𝑡𝑟𝑢𝑒=true image,
𝐻=blurring matrix, 𝑛=noise vector
 Nonlinear least square problem:
arg min
𝑓
1
2
𝑠 𝐻𝑓 − 𝑔 2
2
Zervakis and venetsanopoulos used steepend
descent method.
Zervakis and venetsanopoulos further considered
Gauss-Newton(GN) algorithm for NLS problem.
4
(1)
(2)

5.  TV based nonlinear least square problem :
arg min
𝑓
𝐸(𝑓) ≔
1
2
𝑠 𝐻𝑓 − 𝑔 2
2
+𝜇 𝑖=1
𝑚2
𝐷𝑖 𝑓 2
where 𝜇 = regularization parameter
𝑖=1
𝑚2
𝐷𝑖 𝑓 2 =discrete total variation of 𝑓
𝑎1 ≤ 𝑓 ≤ 𝑎2, 𝐷𝑖 𝑓=discrete gradient of 𝑓 at 𝑖 𝑡ℎ pixel
 Main idea:
Original optimization problem
Easier subproblems under ADM
split
5
(3)

6.  Alternating Direction Method Of Multipliers(ADM):
Subject to ℎ𝑓 = 𝑧 , 𝑓 = 𝑢, 𝑓 = 𝑣, 𝐷𝑖 𝑓 = 𝑝𝑖 , ,
and and are indicator functions given by,
=
0, 𝑖𝑓 𝑢 − 𝑎1 ≥ 0,
∞, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
=
0, 𝑖𝑓 𝑣 − 𝑎2 ≥ 0
∞, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
2
2
1 22 2
1
1
argmin ( ) ( ) ( )
2
m
i
i
s z g p k u k v  

   
1( )k u 2 ( )k v
1( )k u
2 ( )k v
6
(4)

7. Algorithm
7

8. Experimental Results
 Peak signal to noise ratio(PSNR):
PSNR=10 log10
max
𝑖
𝑓𝑡𝑟𝑢𝑒 𝑖
2
1
𝑚2 𝑖=1
𝑚2
𝑓 𝑖− 𝑓𝑡𝑟𝑢𝑒 𝑖
2
 Structural similarity (SSIM) index
PSNR should be high
8
(5)

9. Nonlinear Image Restoration
PSNR:b)26.95db,
c)28.68db
SSIM:b)0.8036,c)0.8848
a)True image,
b)observed image,
c)NLS model,
d)TVNLS model
Natural logarithm nonlinearity
Fig. 1[1]
Fig. 2[1]
9

10. Power Nonlinearity
PSNR:b)27.80db, c)30.27db
SSIM:b)0.8401, c)0.9124
a)True image
b)Observed image
c)NLS model
d)TVNLS model
Fig. 3[1]
Fig. 4[1]
10

11. Restoration results with different
blurs and noise level
NLS TVNLS
NONLINE
ARTITY
BLUR STANDAR
D
DEVIATIO
N
PSNR SSIM PSN
R
SSIM
Logarithm Gaussia
n
0.001 26.95 0.8036 28.68 0.8848
0.01 23.75 0.6012 25.76 0.8120
Moffat 0.001 28.52 0.7987 30.81 0.9039
0.01 24.26 0.6201 26.44 0.8225
Power Gaussia
n
0.001 27.80 0.8401 30.27 0.9124
0.01 25.07 0.7451 27.30 0.8619
Moffat 0.001 29.82 0.8599 33.49 0.9304
0.01 26.12 0.7609 28.88 0.8793
Table [1]
11

12. High Dynamic Range Imaging
 The nonlinear response is formulated as:
𝑔 = 𝑠 𝑟
where 𝑟 is true HDR radiance,
𝑔 is observed LDR image,
𝑠 is the camera response
Idea of majorize-minimize(MM) method: use
reweighted least squares technique to tackle the non
smooth TV term and linearized technique to tackle
the non linear least square data fitting term.
12
(6)

13. a)Tone mapped LDR image from true HDR
b)Noisy observed LDR image
c)Tone mapped LDR image from recovered HDR image by MM
method
d) Tone mapped LDR image from recovered HDR image by ADM
method
Fig. 5[1]
Fig. 6[1]
13

14. a)Tone mapped LDR image from true HDR
b)Noisy observed LDR image
c)Tone mapped LDR image from recoverd HDR image by MM method
d) Tone mapped LDR image from recoverd HDR image by ADM method
Fig. 7[1]
Fig. 8[1]
14

15. Conclusion
 TV based variation model to tackle
nonlinear image restoration problem.
 An efficient alternating direction method of
multipliers to solve the model.
 Numerical examples including nonlinear
image restoration and HDR imaging are
shown by author to illustrate the
effectiveness and efficiency of numerical
scheme.
15

16. REFERENCES
[1] C. Chen, M. K. Ng and X. L. Zhao, "Alternating Direction
Method of Multipliers for Nonlinear Image Restoration
Problems," in IEEE Transactions on Image Processing, vol.
24, no. 1, pp. 33-43, Jan. 2015.
[2]B. K. Gunturk and X. Li, Image Restoration: Fundamentals
and Advances. Boca Raton, FL, USA: CRC Press, 2012.
[3] Zhou Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli,
"Image quality assessment: from error visibility to structural
similarity," in IEEE Transactions on Image Processing, vol.
13, no. 4, pp. 600-612, April 2004.
[4] S. Kim, Y. W. Tai, S. J. Kim, M. S. Brown and Y. Matsushita,
"Nonlinear camera response functions and image
deblurring," Computer Vision and Pattern Recognition
(CVPR), IEEE Conference on, Providence, RI, 2012, pp. 25-
32.
16

17. Questions?
17