Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Lec17 sparse signal processing & applications

795 vues

Publié le

Lec-17: Sparse Signal Processing & Applications [notes]
Sparse signal processing, recovery of sparse signal via L1 minimization. Applications including face recognition, coupled dictionary learning for image super-resolution.

Publié dans : Formation
  • Stop getting scammed by online, programs that don't even work! ●●● https://tinyurl.com/y4urott2
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Soyez le premier à aimer ceci

Lec17 sparse signal processing & applications

  1. 1. Image Analysis & Retrieval CS/EE 5590 Special Topics (Class Ids: 44873, 44874) Fall 2016, M/W 4-5:15pm@Bloch 0012 Lec 17 Sparse Signal Processing & Applications Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph: x 2346. http://l.web.umkc.edu/lizhu p.1Z. Li, Image Analysis & Retrv, 2016 Fall
  2. 2. Outline  Recap:  Piece-wise Linear Models via Query Driven Solution  Subspace Indexing on Grassmann Manifold  Optimization of Subspace on Grassmann Manifold  Sparse Signal Processing  Sparse Representation and Robust PCA  Sparse Signal Processing  L1 norm and L1 Magic Solution  Application in occluded face recognition  Summary p.2Z. Li, Image Analysis & Retrv, 2016 Fall
  3. 3. Piece-wise Linear : Query Driven • Query-Driven Piece-wise Linear Model – No pre-determined structure on the training data – Local neighborhood data patch identified from query point q, – Local model built with local data, A(X, q) p.3Z. Li, Image Analysis & Retrv, 2016 Fall
  4. 4. DPC – Discriminant Power Coefficient  The tradeoffs in local data support size p.4Z. Li, Image Analysis & Retrv, 2016 Fall
  5. 5. Face Recognition  On ATT Data set: 40 subjects, 400 images: Extra credit: 10pts   Develop a query driven local Laplacianface model for HW-3 p.5Z. Li, Image Analysis & Retrv, 2016 Fall
  6. 6. Subspace Indexing on Grassmann Manifold • Subspace Clustering by Grassmann Metric: – It is a VQ like process. – Start with a data partition kd-tree, their leaf nodes and associated subspaces {Ak}, k=1..2h – Repeat » Find Ai and Aj, if darc(Ai, Aj) is the smallest among all, and the associated data patch are adjacent in the data space. » Delete Ai and Aj, replace with merged new subspace, and update associated data patch leaf nodes set. » Compute the empirical identification accuracy for the merged subspace » Add parent pointer to the merged new subspace for Ai and Aj . » Stop if only 1 subspace left. p.6Z. Li, Image Analysis & Retrv, 2016 Fall
  7. 7. Simulation • Face data set – Mixed data set of 242 individuals, and 4840 face images – Performance compared with PCA, LDA and LPP modeling p.7Z. Li, Image Analysis & Retrv, 2016 Fall
  8. 8. Newtonian Method in Optimization  Recall that in optimizing a functional over vector variables f(X), X in Rn, p.8 Credit: Kerstin Johnsson, Lund Univ Z. Li, Image Analysis & Retrv, 2016 Fall
  9. 9. Gradient & Hessian on Grassmann Manifold  Gradient on Grassmann manifold: p.9Z. Li, Image Analysis & Retrv, 2016 Fall
  10. 10. Hessian on Grassmann Manifold  Hessian:  FY = nxp 1st order differentiation  FYY= 2nd order differentiation along Y p.10Z. Li, Image Analysis & Retrv, 2016 Fall
  11. 11. Newton’s Method on Grassmann Manifold  Overall framework Prof. A. Edelman’s matlab package:  https://umkc.box.com/s/g2oyqvsb2lx2v9wzf0ju60wnspts4t9g p.11Z. Li, Image Analysis & Retrv, 2016 Fall
  12. 12. Outline  Recap:  Piece-wise Linear Models via Query Driven Solution  Subspace Indexing on Grassmann Manifold  Optimization of Subspace on Grassmann Manifold  Sparse Signal Processing  Sparse Representation and Robust PCA  Sparse Signal Processing  L1 norm and L1 Magic Solution  Application in occluded face recognition  Summary p.12Z. Li, Image Analysis & Retrv, 2016 Fall
  13. 13. Sparse representation • Signals/Images are sparse if it can have very few non-zero coefficients representation in certain subspace: – E.g. cameraman image X represented as 2-D DCT in Y: • How is this related to classification problem ? – Intuitively, sparse is good for classification, because it is to separate samples from different classes – Only when data points are dense and intertwined , classification is hard – How to characterize this mathematically ?  x y=dct2(x) Eigen face p.13Z. Li, Image Analysis & Retrv, 2016 Fall
  14. 14. Sparsity in Human Visual System p.14Z. Li, Image Analysis & Retrv, 2016 Fall
  15. 15. Sparse Signal Recovery  If x is sparse, i.e |x|0 is small, we can recovery x by a random projection measurement, y=Ax  Basis pursuit de-noising:  LASSO: p.15Z. Li, Image Analysis & Retrv, 2016 Fall
  16. 16. Sparse Face Model Consider a face recognition system  We have k=1,2,…,K subjects, each subject has nk training samples {[v1,1, .., v1,n1], [v2,1, .., v2,n2], …, [vK,1, .., vK,nK]}, each is a thumbnail image with d=wxh pixels.  Let us stack all training samples as a collection of column vectors, A, of d N, N=n1 + n2 + … + nK.  The problem is, for a given thumbnail image, y, with unknown class label, how to solve for its label ?K p.16Z. Li, Image Analysis & Retrv, 2016 Fall
  17. 17. Assume y is belonging to class i, then, Or, Where only a small number of coefficients in x has non-zero entry, thus sparse. Sparsity p.17Z. Li, Image Analysis & Retrv, 2016 Fall
  18. 18. Assume y is belonging to class 1, then, Most co-efficients related to other classes are zero, only a small number of non-zero coefficients in alpha 1 Illustration of Sparsity p.18Z. Li, Image Analysis & Retrv, 2016 Fall
  19. 19. • So the problem is rather straight forward – Give y = Ax, where • y is the unknown face image in Rd, • A is the d x N training data matrix, or dictionary, with N large • x is the coefficients of y as linear combination of training samples that is sparse, out of total N coefficients, only a small number of them are non-zero – Mathematically, we are looking for : • Where |x|0 is L0 norm, which counts number of non-zero coefficients in x. Mathematical formulation 𝑥0 = arg min 𝑥 𝑥 0, 𝑠. 𝑡. , 𝐴𝑥 = 𝑦 p.19Z. Li, Image Analysis & Retrv, 2016 Fall
  20. 20. • The L0 minimization problem is basically a combinatorial optimization problem • Not much structure to exploit fast algorithm • Dumbest solution: – Assuming that x has at most 3 non-zero coefficients, then search total – Possible coefficients combinations and find the one gives the best match – It is an kNN search in effect ! L0 minimization is NP hard 𝑁 1 + 𝑁 2 + 𝑁 3 p.20Z. Li, Image Analysis & Retrv, 2016 Fall
  21. 21. L0 and L1 norm  Lk norm (recall minkowski distance) p.21Z. Li, Image Analysis & Retrv, 2016 Fall
  22. 22. L1 solution L-2 ball p.22Z. Li, Image Analysis & Retrv, 2016 Fall
  23. 23. L1 based recognition p.23Z. Li, Image Analysis & Retrv, 2016 Fall
  24. 24. L1 solution for invalid input images • For non-face images: – Non sparse coefficients in x • Can threshold on residual to return not found result p.24Z. Li, Image Analysis & Retrv, 2016 Fall
  25. 25. Occlusion and Disguise • A big problem in biometrics is disguise and occlusion • The magic of sparsity and L1 minimization can deal with that effectively ! • Consider a face image with a small fraction p of its pixels corrupted: p.25Z. Li, Image Analysis & Retrv, 2016 Fall
  26. 26. • Let the occluded face images be y = Ax + e • Then re-state the constraint as, • then solve for P1 with y=Bw. Notice that sparsity in w is achieved thru sparsity in both x and e. Sparsity criteria takes care of occlusion p.26Z. Li, Image Analysis & Retrv, 2016 Fall
  27. 27. • Occlusion example – Large L2 errors, not recoverable by Eigenface/Fisherface: • Accuracy for sunglasses and scarves effects: Occluded face recognition p.27Z. Li, Image Analysis & Retrv, 2016 Fall
  28. 28. L1 vs L2 minimization • A natural question is why not solve y=Ax with L2 minimization ? – Typically, number of training samples is smaller than number of pixels in the training images, so why not do a pseudo-inverse like: – Which looks for a Maximum Likelihood estimation of true x, if noises are Gaussian with covariance sI. – However, the noises are non-gaussian and can be unbounded. The resulting L2 solution pretty bad p.28Z. Li, Image Analysis & Retrv, 2016 Fall
  29. 29. L2 solution for Occlusion • Example with occlusion: – (a): Occluded face – (b): x solved from L2 minimization, not sparse at all – (c ): error – (d ): reconstruction from x p.29Z. Li, Image Analysis & Retrv, 2016 Fall
  30. 30. L1 vs L2 minimization L1 vs L2 in 2D space: y=Ax p.30Z. Li, Image Analysis & Retrv, 2016 Fall
  31. 31. Sparsity is bad news for L2 • Given training set A, the unknown image y is under- determined in A: – R(A): a set of y that satisfies y=Ax: p.31Z. Li, Image Analysis & Retrv, 2016 Fall
  32. 32. Numeric solution for L1 minimization Candes (of CalTech)’s group has this L1 magic matlab toolbox  Check out manual on course webpage  Stephen Boyd:  Boyd’s nice book on Optimization can be downloaded from his webpage at Stanford.  Excellent book, with slides, homework and solutions.  https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf p.32Z. Li, Image Analysis & Retrv, 2016 Fall
  33. 33. Numerical Tool from L1 Magic  L1 Magic Toolbox: p.33 % signal length N = 512; % number of spikes in the signal, must be sparse w.r.t N T = 20; % number of observations to make K = 120; % random +/- 1 signal x = zeros(N,1); q = randperm(N); x(q(1:T)) = sign(randn(T,1)); subplot(3,1,1); plot(x); title('x(t)'); axis([1 500 -1.2 1.2]); % measurement matrix: random measuring fprintf('n Creating random measurement matrix...'); A = randn(K,N); % othorgonalize A = orth(A')'; % observations y = A*x; % initial guess = min energy x0 = A'*y; subplot(3,1,2); plot(x0); title('x_0(t)'); axis([1 500 -1.2 1.2]); % solve with primal-dual method xp = l1eq_pd(x0, A, [], y, 1e-3); subplot(3,1,3); plot(xp); title('x(t) recovered by L1 magic'); axis([1 500 -1.2 1.2]); % test l1magic end Z. Li, Image Analysis & Retrv, 2016 Fall
  34. 34. L1 Magic demo 50 100 150 200 250 300 350 400 450 500 -1 0 1 x(t) 50 100 150 200 250 300 350 400 450 500 -1 0 1 x0(t) 50 100 150 200 250 300 350 400 450 500 -1 0 1 x(t) recovered by l1 magic Original sparse signal L1 magic recovered sparse signal Pseudo-inverse: L2 recovery p.34Z. Li, Image Analysis & Retrv, 2016 Fall
  35. 35. Sparse face  Recover face as sparse signal p.35 % create our measure matrix A: face + nonface icons A=zeros(1600, w*h); A(1:400, :) = faces; A(401:1600, :) = nfaces; [N, dim]=size(A); % in col vec form A = A'; % pick a face: offs in 1-400 figure(3); colormap('gray'); offs = 20; y = faces(offs, :)'; subplot(2,2,1); axis off; imagesc(reshape(y, h,w)); title('fontsize{11}original'); % solve for xp = min |x|, s.t. y=Ax % initial guess = min energy x0 = A'*y; % solve with primal-dual method xp = l1eq_pd(x0, A, [], y, 1e-3); % normalize x0 = x0./norm(x0); xp = xp./norm(xp); % reconstructed face yp = A*xp; subplot(2,2,2); axis off; imagesc(reshape(yp, h,w)); title('fontsize{11}sparse reconstruction'); Z. Li, Image Analysis & Retrv, 2016 Fall
  36. 36. L1Magic for Face Recognition p.36Z. Li, Image Analysis & Retrv, 2016 Fall
  37. 37. Super Resolution Super-Resolution  Super-resolves a lower resolution patch, say k x k, to 3k x 3k.  Mathematically, learn a function: p.37 𝑓 𝑥 → 𝑌, 𝑥 ∈ 𝑅 𝑑, 𝑌 ∈ 𝑅 𝐷 Z. Li, Image Analysis & Retrv, 2016 Fall
  38. 38. Basic Framework  Super-resolve is the inverse of down scaling:  Low res patch y is the blurred and scaled high res patch x:  Assume the high res image is sparse on some dictionary (true, say DCT): p.38 Output OriginalInput Training patches ≈ 𝑦 = 𝑆𝐻𝑥 Z. Li, Image Analysis & Retrv, 2016 Fall
  39. 39. Coupled Dictionary Learning  Pre-train a common set of coupled low and high resolution dictionary  Super-resolve by solving L1 minimization on lower resolution patch, and use the same coeffiients to superresolve the higher resolution patch p.39Z. Li, Image Analysis & Retrv, 2016 Fall
  40. 40. Coupled Dictionary Learning  Learn two sets of Dictionaries, Dh, Dl, that have common sparse coefficients for low and high resolution image patches, y and x:  Reconstruction of low res patch with sparse coefficients:  Furthermore, introduce a linear projection, F, to enforce perceptual metrics  Then the high res patch x, can be constructed as p.40 min 𝛼 0 , 𝑠. 𝑡. , 𝐷𝑙 𝛼 − 𝑦 2 ≤ 𝜖 min 𝛼 0 , 𝑠. 𝑡. , 𝐹𝐷𝑙 𝛼 − 𝐹𝑦 2 ≤ 𝜖 𝑥 = 𝐷ℎ 𝛼 Yang, J Wright, TS Huang, Y Ma, Image super-resolution via sparse representation, IEEE Trans. Image Processing, vol.19 (11), 2861-2873 Z. Li, Image Analysis & Retrv, 2016 Fall
  41. 41. Coupled Dictionary Learning  Put together, super resolve is to solve:  Sparse reconstruction of lower resolution y  Enforce local consistence with high res patches, extract adjacent overlapping stripes, via P, to be in agreement, w is the previously reconstructed patch pixels:  Solution via Lagrangian relaxation: p.41Z. Li, Image Analysis & Retrv, 2016 Fall
  42. 42. Overall Algorithm  Patch level super-resolution, complete with global image gradient search p.42Z. Li, Image Analysis & Retrv, 2016 Fall
  43. 43. Dictionary Training Training data: low and high resolution image patches Yl={yk}, Xh={xk}:  Enforce the common sparse coefficients p.43Z. Li, Image Analysis & Retrv, 2016 Fall
  44. 44. Results  Dictionary Training  From flowers and animals data set, covering a variety of texture  Training dictionary from more than 100,000 samples p.44 𝐷ℎ 𝐷𝑙 Z. Li, Image Analysis & Retrv, 2016 Fall
  45. 45. Results  3x super-resolution p.45 Bicubic Neighbor embedding [Chang CVPR ‘04] Low-resolution input OriginalCoupled Dictionary Z. Li, Image Analysis & Retrv, 2016 Fall
  46. 46. Related Work  Potential paper review project p.46Z. Li, Image Analysis & Retrv, 2016 Fall
  47. 47. Summary  Sparse Signal Processing  If signal is sparse in some (unknown) domain, then from a random measurement, we can reliably recover the signal via L1 minimization  Applications: Robust PCA and Face Recognition with Occlusion  Face images are sparse linear combination from a face dictionary  Recovery from solving L1 problem ~ caveat: only additive noises can be delt.  Applications: Coupled Dictionary for Image Super Resolution  Coupled dictionary: high and low res image patches sharing the same coefficients. p.47 min 𝑥 𝑥 1, 𝑠. 𝑡. 𝑦 = 𝐴𝑥 Z. Li, Image Analysis & Retrv, 2016 Fall

×