Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Prochain SlideShare
Chargement dans…5
×

Matrix factorization

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Identifiez-vous pour voir les commentaires

Matrix factorization

1. 1. Matrix Factorization Lecture #7 EEE 574 Dr. Dan Tylavsky
2. 2. © Copyright 1999 Daniel Tylavsky Matrix FactorizationLet’s look at solving Ax=b A, b are dense.Two possible (general approaches):– Indirect Methods - methods which asymptotically approach (but never reach) the true solution as the number of steps increases. Ex: Point Jacobi, Gauss-Siedel, Successive Over relaxation, Multigrid– •
3. 3. © Copyright 1999 Daniel Tylavsky Matrix Factorization Direct Methods - methods that yield the theoretically exact result in a fixed number of steps. (Assuming an infinitely precise computing engine.) Ex: Elimination Methods Gauss Elimination Gauss-Jordan Elimination– Cholesky Factorization (Numerically Symmetric) • – LU or LDU factorization (a.k.a., Product form of the Inverse),(Variations Include Crout, Doolittle, Banachiewicz) – Orthogonalization Methods – QR factorization – Given Rotations • – –
4. 4. © Copyright 1999 Daniel Tylavsky Matrix Factorization Conjugate Gradient Methods - Exact solution in # of steps equal to the number of different eigenvalues. Various VariationsAdvantages of Elimination Methods • – No convergence criteria (however, pivoting in general non-positive definite case.) Factors can be used repeatedly. Factors can be ‘easily’ modified to accommodate matrix/network changes.– Partial factorized matrices give linear network equivalents.–––
5. 5. © Copyright 1999 Daniel TylavskyMatrix Factorization Definitions– Symmetric: A=AT • Positive Definite: xTAx>0, (A>0 is short hand notation) • Diagonally Dominant: • aii ≥ ∑a j ≠i ,with inequality for at least one i. ij • Properly Diagonally Dominant: aii > ∑a j ≠i ij for all i. • Rank: Order of the largest sub-matrix with nonzero determinant.
6. 6. © Copyright 1999 Daniel Tylavsky Matrix Factorization Properties– Symmetric pos. def. (A>0) matrix has all positive real eigenvalues. • If A>0, then Det(A)=|A|>0. • A rank 1 matrix must have the outer-product form, xyT. • 1  2 4 6  x =  2  , y T = [ 2 4 6]   xy T = 4 8 12    3   6 12 18   • Symmetric A>0 has a unique Cholesky factorization. A=U U, A=U’ DU’ T T • If A is properly diagonally dominant and all of its diagonal elements are positive, then A is positive definite.
7. 7. © Copyright 1999 Daniel Tylavsky Matrix Factorization – Factorizing a non-symmetric matrix.  a11 a12 a13  a1n  a a 22 a 23  a2n   21  α bT A =  a31 a32 a33  a3 n  =  1 ~  1   c A1          1   a n1 an 2 a n3  ann    – I claim A can be expanded in the following form: α1 0 1 0 1  b1 T  A=  α1  = L1 A 2U 1  c1 I  0  A2  0  I  
8. 8. © Copyright 1999 Daniel TylavskyMatrix Factorization– To find the value of A2, multiply. α1 0 1 0 1  b1 T  A=  α1  = L1 A 2U 1  c1 I  0  A2  0  I    b1 T  α1 T b1   α1 0 1 α1 b1  T A=  α1  =  =   c1 A1  T  c1b1 ~  c1 A2  0 I   c1 A2 +    α1    Outer product term of– Therefore: first row & first column T ~ T ~ c1b1 (absent. diag. element) c1b1 ∴ A2 + = A1 ⇒ A2 = A1 − α1 α 1
9. 9. © Copyright 1999 Daniel Tylavsky Matrix Factorization– Do this repeatedly until we have the product form of A. 1 0T  1 0   T T U 1 = L1 0 α 2 b2 U 1 2 A = L1 A U 1 = L1  0 A2    ~   c2 A2  1 0 0  1 0 T 0 T  1 0 0T  1 0 0T   T=   T  T T A 2 = 0 α 2 b2 b2  0 α 2 0  0 1 0  0 1 α2  ~    0 c2  A2   0 c2 I  0 0 A3  0 0   I   = L2 A3 U2
10. 10. © Copyright 1999 Daniel Tylavsky Matrix Factorization– To find A3, multiply the result. 1 0 0T  1 0 0T  1 0 0T     T b2 A 2 = 0 α 2 T T 0  0 1 0  0 1 = L2 A3U 2 α2   0 c2  I  0 0 A3  0 0   I   1 0 0T  1 0 0T  1 0 0T       T T b2  = 0 α 2 T = 0 α 2 0  0 1 b2  α2   0 c2 A3  0 0    I   0 c2 A3 + c2 b2 T   α2   T ~ ~ c b T c b ∴ A3 + 2 2 = A2 ⇒ A3 = A2 − 2 2 α2 α2 – We can write: A = L1 L2 A3U 2U1
11. 11. © Copyright 1999 Daniel TylavskyMatrix Factorization– Continuing on yields: A = L1 L2  Ln −1 A nU n −1 U 2U 1– where: 1    0  An =  =L n  0 1   n   a nn – We can write: A = L1 L2  Ln −1 LnU n −1 U 2U1
12. 12. Elementary-Matrix-Multiplication-Superposition Principal.– © Copyright 1999 Daniel Tylavsky Matrix Factorization i j 1 i j  1          1 ←i  αi   Li L j =   αj  ← j ,i < j  1  1   ci      cj     1    1  1        αi ←i   Li L j =  αj  ← j ,i < j  1     ci c j    1  
13. 13. © Copyright 1999 Daniel Tylavsky Matrix Factorization This implies I can write:– α1   α2     α3    L1 L2  Ln −1 Ln =  c1  =L  c2      c3    αn   
14. 14. © Copyright 1999 Daniel Tylavsky Matrix Factorization Taking the transpose of the elementary-matrix- multiplication-superposition principal allows me to write:–  T b1  1 α1   T   1 b2   α2   T b3  U n −1U n − 2 U 2U 1 =  1 α3  = U          1       1  Hence we can write:– A = LU
15. 15. © Copyright 1999 Daniel Tylavsky Matrix Factorization Let’s work though an example:– 10.000 -1.000 -2.000 -3.000 -1.000 12.100 -2.200 -2.700 A= -2.000 -2.200 15.880 -6.300 -3.000 -2.700 -6.300 23.400 Divide Top Row by a11. – T ~ c1b1 Divide 3rd Row by α3. – Compute A2 = A1 − – α1 Divide 2nd Row by α2. – T ~ c3b3 T ~ c2 b2 – Compute A4 = A3 − α3 – Compute A3 = A2 − α2 10.000 0 0 0 0 0.000 0.000 0.000 1.000 1.000 -0.100 -0.200 -0.300 -0.100 -0.200 -0.300 -0.100 -0.200 -0.300 -1.000 12.000 -2.400 -3.400 0 0 12.100 -2.200 -2.700 -3.000 00 1 11 0 0 -0.200 -0.250 -0.200 -0.250 0 0 A= -2.000 -2.400 15.480 -6.900 * -2.200 15.880 -6.300 * 15.000 -7.500 0 00 0 00 11 1 0 -0.500 -0.500 0 -3.000 -3.000 -6.900 22.500 -2.700 -6.300 23.400 -7.500 21.750 18.000 00 0 00 00 0 11 1
16. 16. © Copyright 1999 Daniel TylavskyMatrix FactorizationIndividual Problem: Verify that previous answer is correct. 10.000 0 0 0 1.000 -0.100 -0.200 -0.300 -1.000 12.000 0 0 0 1 -0.200 -0.250A= -2.000 -2.400 15.000 0 * 0 0 1 -0.500 -3.000 -3.000 -7.500 18.000 0 0 0 1Individual Problem: Factor A=LU for: 20.000 -20.000 -8.000 -8.000 -20.000 40.000 3.000 3.000 A= -8.000 3.000 14.450 -5.550 -8.000 3.000 -5.550 24.450Note that result isn’t symmetrical.
17. 17. © Copyright 1999 Daniel Tylavsky Matrix Factorization To make this symmetric use the L’DL’T form. – Let l11  – Write:  l 22    A = LD −1 DU = L DLT D=  l33     l 44  10.000 0 0 0 1.000 -0.100 -0.200 -0.300 -1.000 12.000 0 0 0 1 -0.200 -0.250 A= -2.000 -2.400 15.000 0 * 0 0 1 -0.500 -3.000 -3.000 -7.500 18.000 0 0 0 1 1 0 0 0 10 0 0 0 1.000 -0.100 -0.200 -0.300 -0.100 1 0 0 0 12 0 0 0 1 -0.200 -0.250 A= -0.200 -0.200 1 0 * 0 0 15 0 * 0 0 1 -0.500 -0.300 -0.250 -0.500 1 0 0 0 18 0 0 0 1– Division by diag. of row & col. Performed @ same time.
18. 18. © Copyright 1999 Daniel TylavskyMatrix FactorizationIndividual Problem: Modify LU factors of A: 20.000 -20.000 -8.000 -8.000 -20.000 40.000 3.000 3.000 A= -8.000 3.000 14.450 -5.550 -8.000 3.000 -5.550 24.450To get L’DL’T:
19. 19. © Copyright 1999 Daniel TylavskyMatrix FactorizationData structure for storing LU factors? 10.000 0 0 0 1.000 -0.100 -0.200 -0.300 -1.000 12.000 0 0 0 1 -0.200 -0.250A= -2.000 -2.400 15.000 0 * 0 0 1 -0.500 -3.000 -3.000 -7.500 18.000 0 0 0 1 For LU factors could use LC-D-UR/X: Pos: 0 1 2 3 4 5 6 7 8 9 10 11 12DiagL(k) 10.0 12 15 18 U(k): -0.1 -0.2 -0.3 -0.2 -0.3 -0.5 L(k): -1.0 -2.0 -3.0 -2.4 -3.0 -7.5 Indx(k): 2 3 4 3 4 4 ERP(k): 0 3 5 6
20. 20. © Copyright 1999 Daniel Tylavsky Matrix Factorization 1 0 0 0 10 0 0 0 1.000 -0.100 -0.200 -0.300 -0.100 1 0 0 0 12 0 0 0 1 -0.200 -0.250A= -0.200 -0.200 1 0 * 0 0 15 0 * 0 0 1 -0.500 -0.300 -0.250 -0.500 1 0 0 0 18 0 0 0 1For L’DL’T factors could use CR(L)-D/X: Pos: 0 1 2 3 4 5 6 7 8 9 10 11 12 Diag(k) 10.0 12 15 18 L(k): -0.1 -0.2 -0.3 -0.2 -0.3 -0.5RIndx(k): 2 3 4 3 4 4 ERP(k): 0 3 5 6
21. 21. © Copyright 1999 Daniel Tylavsky Matrix Factorization If we store & work through A by rows, factorization is done in a slightly different way:– 10.000 -1.000 -2.000 -3.000 -1.000 12.100 -2.200 -2.700 A= -2.000 -2.200 15.880 -6.300 – Divide Top Row by a11. -3.000 -2.700 -6.300 23.400 T ~ c1b1 – Compute A2 = A1 − α1 for 2nd row. Divide 2nd Row by α2. – T ~ c1b1 T ~ c2 b2 – Compute A2 = A1 − α1and A3 = A2 − for α2 3rd row. 10.000 0 0 0 1 -0.100 -0.200 -0.300 -1.000 12.000 12.100 0 -2.400 -2.200 0 -3.000 -2.700 0 1 -0.200 0 -0.250 0 A= -2.000 -2.400 -2.200 15.000 15.880 -7.500 -6.300 * 0 0 1 0 -3.000 -2.700 -6.300 23.400 0 0 0 1
22. 22. © Copyright 1999 Daniel Tylavsky Matrix FactorizationIndividual Problem: Finish the factorization. 10.000 0 0 0 1 -0.100 -0.200 -0.300 -1.000 12.000 0 0 0 1 -0.200 -0.250A= -2.000 -2.400 15.000 -7.500 * 0 0 1 0 -3.000 -2.700 -6.300 23.400 0 0 0 1
23. 23. © Copyright 1999 Daniel Tylavsky Matrix Factorization – Cholesky Factorization (Symmetric A=LLT.)  a11 a12 a13  a1n  a a 22 a 23  a2n   12  α bT A =  a13 a 23 a33  a 3n  =  1 ~  1    b A1          1  a1n a2n a3 n  a nn    – A can be expanded in the following form:  α1 0 1  T b1  0   α1 A =  b1  α1  = L1 A L1 2 T Etc.  I  0 A2      α1    0  I   – Not used. Calculating square-roots is expensive.
24. 24. The End