SlideShare une entreprise Scribd logo
1  sur  116
Télécharger pour lire hors ligne
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Regularized Principal Component Analysis for
Spatial Data
Wen-Ting Wang
Institute of Statistics, National Chiao Tung University
January 25, 2017
Joint work with Hsin-Cheng Huang, Academia Sinica
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Outline
1 Background
2 Regularized Principal Component Analysis
3 Computation algorithm
4 Numerical Example
5 Summary
2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Outline
1 Background
2 Regularized Principal Component Analysis
3 Computation algorithm
4 Numerical Example
5 Summary
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Background
• Spatial processes of interest:
{ηi(s); s ∈ D} ; i = 1, . . . , n
– D ⊂ Rd
– mean zero
– common covariance function: Cη(s∗
, s) = Cov(ηi(s∗
), ηi(s))
– η1(·), . . . , ηn(·): uncorrelated
4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Background
• Spatial processes of interest:
{ηi(s); s ∈ D} ; i = 1, . . . , n
– D ⊂ Rd
– mean zero
– common covariance function: Cη(s∗
, s) = Cov(ηi(s∗
), ηi(s))
– η1(·), . . . , ηn(·): uncorrelated
• Observed data at locations s1, . . . , sp ∈ D,
Yi(sj) = ηi(sj) + ϵij; i = 1, . . . , n, j = 1, . . . , p
– ϵij ∼ (0, σ2
): white noise
– ϵij and ηi(sj) are uncorrelated for any i, j
4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Targets
1 Detect the dominant patterns (modes) of η1(·), . . . , ηn(·)
– interpret the variability of spatial data physically
2 Estimate spatial covariance function Cη(·, ·)
– no specific assumption (e.g., parametric form or stationarity)
– spatial prediction (kriging) of {ηi(s); s ∈ D}
5
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Example: dominant patterns
• Two dominant patterns (Deser, 2009)
Basin-wide mode East-west dipole mode
−0.04 −0.02 0.00 0.02 0.04
– Data: Indian Ocean sea surface temperature anomalies (Monthly average)
– related to El Ninõ–Southern Oscillation (ENSO)
6
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Rank-K spatial model
• Data:
Yi(sj) = ηi(sj) + ϵij; i = 1, . . . , n, j = 1 . . . , p
7
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Rank-K spatial model
• Data:
Yi(sj) =
K∑
k=1
ξikφk(sj) + ϵij; i = 1, . . . , n, j = 1 . . . , p
– ξi1, . . . , ξiK ∼ (0, Λ); ΛK×K is positive-definite
– φ1(·) . . . , φK (·): K unknown orthonormal functions
– ξik uncorrelated with ϵij
8
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Rank-K spatial model
• Data:
Yi(sj) =
K∑
k=1
ξikφk(sj) + ϵij; i = 1, . . . , n, j = 1 . . . , p
– ξi1, . . . , ξiK ∼ (0, Λ); ΛK×K is positive-definite
– φ1(·) . . . , φK (·): K unknown orthonormal functions
– ξik uncorrelated with ϵij
• Spatial covariance function:
Cη(s∗
, s) =
K∑
k=1
K∑
k′=1
λkk′ φk(s∗
)φk′ (s)
– λkk′ : (k, k′
) entry of Λ
8
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Goal
• Find φ1(·), . . . , φK(·) to represent the dominant patterns.
9
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Goal
• Find φ1(·), . . . , φK(·) to represent the dominant patterns.
• Standard approach: Principal component analysis (PCA)
9
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Principal Component Analysis
• p-dimensional data vector:
Yi = (Yi(s1), . . . , Yi(sp))′
∼ (0, Σ)
• Idea: Find ϕ ∈ Rp with ϕ′ϕ = 1,which maximizes Var(ϕ′Yi)
10
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Principal Component Analysis
• p-dimensional data vector:
Yi = (Yi(s1), . . . , Yi(sp))′
∼ (0, Σ)
• Idea: Find ϕ ∈ Rp with ϕ′ϕ = 1,which maximizes Var(ϕ′Yi)
• Spectral decomposition: Σ
– eigenvalues: λ1 ≥ · · · ≥ λp
– eigenvectors: ϕ1, . . . , ϕp
10
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Principal Component Analysis
• p-dimensional data vector:
Yi = (Yi(s1), . . . , Yi(sp))′
∼ (0, Σ)
• Idea: Find ϕ ∈ Rp with ϕ′ϕ = 1,which maximizes Var(ϕ′Yi)
• Spectral decomposition: Σ
– eigenvalues: λ1 ≥ · · · ≥ λp
– eigenvectors: ϕ1, . . . , ϕp
• Dominant patterns: ϕ1, . . . , ϕK (with λ1, . . . , λK large)
10
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Principal Component Analysis
• Data matrix: Yn×p = (Y1, . . . , Yn)′
• Sample covariance matrix: S = Y ′Y /n
• Spectral decomposition: S
– sample eigenvalues: ˜λ1 ≥ · · · ≥ ˜λp
– sample eigenvectors: ˜ϕ1, . . . , ˜ϕp
• ˜ϕ1, . . . , ˜ϕK are estimates of ϕ1, . . . , ϕK
11
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Principal Component Analysis
• Data matrix: Yn×p = (Y1, . . . , Yn)′
• Sample covariance matrix: S = Y ′Y /n
• Spectral decomposition: S
– sample eigenvalues: ˜λ1 ≥ · · · ≥ ˜λp
– sample eigenvectors: ˜ϕ1, . . . , ˜ϕp
• ˜ϕ1, . . . , ˜ϕK are estimates of ϕ1, . . . , ϕK
• Problems:
– high estimation variability: n is small or p is large
→ unstable and noisy patterns
→ weak physical interpretation
– without spatial structure of ϕ
11
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Example:
PCA : φ
~
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
12
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Example:
True : φ PCA : φ
~
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
12
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Motivation
To enhance the interpretablity, we implement PCA by incorporating
13
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Motivation
To enhance the interpretablity, we implement PCA by incorporating
1 spatial structure of eigenvectors
13
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Motivation
To enhance the interpretablity, we implement PCA by incorporating
1 spatial structure of eigenvectors
2 sparsity of eigenvectors
13
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Motivation
To enhance the interpretablity, we implement PCA by incorporating
1 spatial structure of eigenvectors
2 sparsity of eigenvectors
3 orthogonality of eigenvectors
13
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Outline
1 Background
2 Regularized Principal Component Analysis
3 Computation algorithm
4 Numerical Example
5 Summary
14
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Quick review
• Data Yi = (Yi(s1), . . . , Yi(sp))′; i = 1, . . . , n
– Yi(sj) =
K∑
k=1
ξikφk(sj) + ϵij; j = 1 . . . , p
• φ1(·) . . . , φK (·): K unknown orthonormal functions
• ξi1, . . . , ξiK ∼ (0, Λ); ΛK×K ≻ 0
• ϵij ∼ (0, σ2
); ϵij: uncorrelated with ξik
15
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Quick review
• Data Yi = (Yi(s1), . . . , Yi(sp))′; i = 1, . . . , n
– Yi(sj) =
K∑
k=1
ξikφk(sj) + ϵij; j = 1 . . . , p
• φ1(·) . . . , φK (·): K unknown orthonormal functions
• ξi1, . . . , ξiK ∼ (0, Λ); ΛK×K ≻ 0
• ϵij ∼ (0, σ2
); ϵij: uncorrelated with ξik
• Spatial covariance function:
Cη(s∗
, s) =
K∑
k=1
K∑
k′=1
λkk′ φk(s∗
)φk′ (s)
– λkk′ : (k, k′
) entry of Λ
15
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Quick review
• Data Yi = (Yi(s1), . . . , Yi(sp))′; i = 1, . . . , n
– Yi(sj) =
K∑
k=1
ξikφk(sj) + ϵij; j = 1 . . . , p
• φ1(·) . . . , φK (·): K unknown orthonormal functions
• ξi1, . . . , ξiK ∼ (0, Λ); ΛK×K ≻ 0
• ϵij ∼ (0, σ2
); ϵij: uncorrelated with ξik
• Spatial covariance function:
Cη(s∗
, s) =
K∑
k=1
K∑
k′=1
λkk′ φk(s∗
)φk′ (s)
– λkk′ : (k, k′
) entry of Λ
• Unknown parameters: φ1(·), . . . , φK(·), Λ, σ2
15
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
PCA (alternative version)
• Data matrix: Yn×p = (Y1, . . . , Yn)′
• PCA :
˜Φ = arg min
Φ:Φ′
Φ=IK
∥Y − Y ΦΦ′
∥2
F
– Φp×K = (ϕ1, . . . , ϕK ) with ϕjk = φj(sk)
– ∥M∥2
F =
n∑
i=1
p
∑
j=1
m2
ij
16
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Regularized PCA
• Data matrix: Yn×p = (Y1, . . . , Yn)′
• Φp×K = (ϕ1, . . . , ϕK) with ϕjk = φj(sk)
• Objective function
∥Y − Y ΦΦ′
∥2
F
subject to Φ′
Φ = IK
17
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Regularized PCA
• Data matrix: Yn×p = (Y1, . . . , Yn)′
• Φp×K = (ϕ1, . . . , ϕK)
• Objective function
∥Y − Y ΦΦ′
∥2
F +τ1
K∑
k=1
J(φk) + τ2
K∑
k=1
p∑
j=1
|φk(sj)|
subject to Φ′
Φ = IK
– J(φk) =
∑
z1+···+zd=2
∫
Rd
(
∂2
φk(s)
∂xz1
1 . . . ∂x
zd
d
)2
ds
• s = (x1, . . . , xd)′
17
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Regularized PCA
• Data matrix: Yn×p = (Y1, . . . , Yn)′
• Φp×K = (ϕ1, . . . , ϕK)
• Objective function
∥Y − Y ΦΦ′
∥2
F +τ1
K∑
k=1
J(φk) + τ2
K∑
k=1
p∑
j=1
|φk(sj)|
subject to Φ′
Φ = IK
– J(φk) =
∑
z1+···+zd=2
∫
Rd
(
∂2
φk(s)
∂xz1
1 . . . ∂x
zd
d
)2
ds
• s = (x1, . . . , xd)′
– τ1: smoothness parameter
– τ2: sparseness parameter
17
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Spatial PCA (SpatPCA)
• J(φk) = ϕ′
kΩϕk
– Ωp×p: determined only by s1, . . . , sp
– Ref: Green and Silverman (1994)
18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Spatial PCA (SpatPCA)
• J(φk) = ϕ′
kΩϕk
– Ωp×p: determined only by s1, . . . , sp
– Ref: Green and Silverman (1994)
• Proposal: SpatPCA
ˆΦ = arg min
Φ:Φ′
Φ=IK



∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
kΩϕk + τ2
K∑
k=1
p∑
j=1
|ϕjk|



18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Spatial PCA (SpatPCA)
• J(φk) = ϕ′
kΩϕk
– Ωp×p: determined only by s1, . . . , sp
– Ref: Green and Silverman (1994)
• Proposal: SpatPCA
ˆΦ = arg min
Φ:Φ′
Φ=IK



∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
kΩϕk + τ2
K∑
k=1
p∑
j=1
|ϕjk|



• As τ1 = τ2 = 0, ˆϕk is the k-th eigenvector of S.
18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
SpatPCA: ˆφ1(·), . . . , ˆφK(·)
• ( ˆφ1(·), . . . , ˆφK(·)) minimizes
∥Y − Y ΦΦ′
∥2
F +τ1
K∑
k=1
J(φk) + τ2
K∑
k=1
p∑
j=1
|φk(sj)|,
subject to Φ′
Φ = IK
• ˆφk(·): smoothing spline based on ˆϕk
ˆφk(s) =
p∑
i=1
aig(∥s − si∥) + b0 +
d∑
j=1
bjxj
– s = (x1, . . . , xd)′
– g(r) =



1
16π
r2
log r; if d = 2,
Γ(d/2 − 2)
16πd/2
r4−d
; if d = 1, 3,
– a = (a1, . . . , ap)′
and b = (b0, b1, . . . , bd)′
depend on ˆϕk
19
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Why considering two penalties?
20
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
1D Example
τ1 = 0
True PCA SpatPCA
21
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 0
True PCA SpatPCA
22
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 0.03
True PCA SpatPCA
23
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 0.09
True PCA SpatPCA
24
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 0.32
True PCA SpatPCA
25
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 3.81
True PCA SpatPCA
26
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 156.17
True PCA SpatPCA
27
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 6405.22
True PCA SpatPCA
28
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
τ1 = 25000
True PCA SpatPCA
29
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 0
True PCA SpatPCA
30
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 39
True PCA SpatPCA
31
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 82
True PCA SpatPCA
32
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 126
True PCA SpatPCA
33
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 212
True PCA SpatPCA
34
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 342
True PCA SpatPCA
35
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 472
True PCA SpatPCA
36
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ1 = 0 (only sparseness)
τ2 = 520
True PCA SpatPCA
37
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 0
True PCA SpatPCA
38
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 0.02
True PCA SpatPCA
39
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 0.04
True PCA SpatPCA
40
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 0.09
True PCA SpatPCA
41
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 0.41
True PCA SpatPCA
42
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 4.19
True PCA SpatPCA
43
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 42.68
True PCA SpatPCA
44
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
τ1 = τ2 = 100
True PCA SpatPCA
45
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
2D Example
True : φ PCA : φ
~
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
46
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 0
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
47
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 0
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
48
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 93
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
49
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 201
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
50
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 437
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
51
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 2053
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
52
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 20932
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
53
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 213414
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
54
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 1: ˆϕ as τ2 = 0 (only smoothness)
True : φ τ1 = 5e+05
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
55
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 0
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
56
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 35
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
57
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 42
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
58
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 50
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
59
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 73
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
60
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 127
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
61
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 220
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
62
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 2: ˆϕ as τ2 = 0 (only sparseness)
True : φ τ2 = 270
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
63
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 0
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
64
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 33
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
65
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 38
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
66
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 43
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
67
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 55
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
68
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 81
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
69
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 118
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
70
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Case 3: ˆϕ as τ1 = τ2
True : φ τ1 = τ2 = 136
−0.2 0.0 0.2 0.4 0.6 0.8 1.0
71
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
(τ1, τ2) selection
• M-fold cross-validation:
CV(τ1, τ2) =
1
M
M∑
m=1
∥Y (m)
− Y (m) ˆΦ
(−m)
τ1,τ2
( ˆΦ
(−m)
τ1,τ2
)′
∥2
F
72
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
(τ1, τ2) selection
• M-fold cross-validation:
CV(τ1, τ2) =
1
M
M∑
m=1
∥Y (m)
− Y (m) ˆΦ
(−m)
τ1,τ2
( ˆΦ
(−m)
τ1,τ2
)′
∥2
F
– Partition {Y1, . . . , Yn} into M parts with equal (or roughly) size
– Y (m)
: the sub-matrix of Y corresponding to the m-th part
– ˆΦ
(−m)
τ1,τ2
: the estimate of Φ for (τ1, τ2) based on Y (−m)
• Y (−m)
: remaining data, i.e., Y excluding Y (m)
72
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
(τ1, τ2) selection
• M-fold cross-validation:
CV(τ1, τ2) =
1
M
M∑
m=1
∥Y (m)
− Y (m) ˆΦ
(−m)
τ1,τ2
( ˆΦ
(−m)
τ1,τ2
)′
∥2
F
– Partition {Y1, . . . , Yn} into M parts with equal (or roughly) size
– Y (m)
: the sub-matrix of Y corresponding to the m-th part
– ˆΦ
(−m)
τ1,τ2
: the estimate of Φ for (τ1, τ2) based on Y (−m)
• Y (−m)
: remaining data, i.e., Y excluding Y (m)
• Find τ1 and τ2 which minimize CV(τ1, τ2)
72
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Estimation of spatial covariance function
• Spatial covariance function: Cη(s∗
, s) =
K∑
k=1
K∑
k′=1
λkk′ φk(s∗
)φk′ (s)
• Till now, σ2 and Λ are unknown
• Λ has K(K + 1)/2 unknown elements
73
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Estimation of spatial covariance function
• Spatial covariance function: Cη(s∗
, s) =
K∑
k=1
K∑
k′=1
λkk′ φk(s∗
)φk′ (s)
• Λ has K(K + 1)/2 unknown elements
• Apply the regularized method (Tzeng and Huang (2015)):
(
ˆσ2
, ˆΛ
)
= arg min
(σ2,Λ):σ2≥0, Λ⪰0
{
1
2
S − ( ˆΦΛ ˆΦ
′
+ σ2
I)
2
F
+ γ∥ ˆΦΛ ˆΦ
′
∥∗
}
– 1st term : goodness of fit based on var(Yi) = ΦΛΦ′
+ σ2
I
– ˆΦ: given SpatPCA estimate
– γ ≥ 0
– ∥M∥∗ = tr((M′
M)1/2
)
73
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Estimation of spatial covariance function
• Spatial covariance function: Cη(s∗
, s) =
K∑
k=1
K∑
k′=1
λkk′ φk(s∗
)φk′ (s)
• Λ has K(K + 1)/2 unknown elements
• Apply the regularized method (Tzeng and Huang (2015)):
(
ˆσ2
, ˆΛ
)
= arg min
(σ2,Λ):σ2≥0, Λ⪰0
{
1
2
S − ( ˆΦΛ ˆΦ
′
+ σ2
I)
2
F
+ γ∥ ˆΦΛ ˆΦ
′
∥∗
}
– 1st term : goodness of fit based on var(Yi) = ΦΛΦ′
+ σ2
I
– ˆΦ: given SpatPCA estimate
– γ ≥ 0
– ∥M∥∗ = tr((M′
M)1/2
)
• Proposed estimate: ˆCη(s∗
, s) =
K∑
k=1
K∑
k′=1
ˆλkk′ ˆφk(s∗
) ˆφk′ (s)
73
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Solution of (σ2
, Λ)
Closed-form solutions :
• ˆΛ = ˆV diag
(
ˆλ∗
1, . . . , ˆλ∗
K
)
ˆV ′
• ˆσ2 =



1
p − ˆL
(
tr(S) −
ˆL∑
k=1
(
ˆdk − γ
)
)
; if ˆd1 > γ,
1
p
(tr(S)) ; if ˆd1 ≤ γ ,
– ˆV diag( ˆd1, . . . , ˆdK ) ˆV ′
is the eigen-decomposition of ˆΦ
′
S ˆΦ with
ˆd1 ≥ · · · ≥ ˆdK
– ˆL = max
{
L : ˆdL − γ > 1
p−L
(
tr(S) −
∑L
k=1( ˆdk − γ)
)
, L = 1, . . . , K
}
– ˆλ∗
k = max( ˆdk − ˆσ2
− γ, 0); k = 1, . . . , K.
74
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
γ selection
Selection of γ by minimizing the CV criterion:
CV2(γ) =
1
M
M∑
m=1
S(m)
− ˆΦ ˆΛ
(−m)
γ
ˆΦ
′
− (ˆσ2
γ)(−m)
I
2
F
• Partition Y into M parts, Y (1), . . . , Y (M)
• S(m): sample covariance matrix associated with Y (m)
• Y (−m): remaining data
•
((
ˆσ2
γ
)(−m)
, ˆΛ
(−m)
γ
)
: estimate of (σ2, Λ) based on Y (−m)
75
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Outline
1 Background
2 Regularized Principal Component Analysis
3 Computation algorithm
4 Numerical Example
5 Summary
76
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Computation
• Original optimization problem
min
Φ
∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
iΩϕk + τ2
K∑
k=1
p∑
j=1
|ϕjk|,
subject to Φ′
Φ = IK
• Difficulties: orthogonal constraint and ℓ1 norm penalty
77
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Computation
• Original optimization problem
min
Φ
∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
iΩϕk + τ2
K∑
k=1
p∑
j=1
|ϕjk|,
subject to Φ′
Φ = IK
• Difficulties: orthogonal constraint and ℓ1 norm penalty
• Alternating direction method of multipliers (ADMM)
– Gabay and Mercier (1976), Boyd, et. al. (2010).
– Idea: original optimization problem → several easy subproblems
77
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Alternating direction method of multipliers
• Equivalent problem (ADMM form):
min
Φ,Q,R
∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
iΩϕk + τ2
K∑
k=1
p∑
j=1
|rjk| ,
subject to Q′Q = IK and Φ = Q = R
78
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Alternating direction method of multipliers
• Equivalent problem (ADMM form):
min
Φ,Q,R
∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
iΩϕk + τ2
K∑
k=1
p∑
j=1
|rjk| ,
subject to Q′Q = IK and Φ = Q = R
• Augmented Lagrangian function:
L(Φ, Q, R, Γ1, Γ2)
=∥Y − Y ΦΦ′
∥2
F + τ1
K∑
k=1
ϕ′
iΩϕk + τ2
K∑
k=1
p∑
j=1
|rjk|
+ tr(Γ′
2(Φ − R)) +
ρ
2
∥Φ − R∥2
F
+ tr(Γ′
1(Φ − Q)) +
ρ
2
∥Φ − Q∥2
F subject to Q′Q = IK
– Γ1, Γ2:Lagrange multipliers; some ρ > 0
78
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Algorithm:at the ℓ-th iteration
Φ(ℓ+1)
= arg min
Φ
L
(
Φ, Q(ℓ)
, R(ℓ)
, Γ
(ℓ)
1 , Γ
(ℓ)
2
)
Q(ℓ+1)
= arg min
Q:Q′Q=IK
L
(
Φ(ℓ+1)
, Q, R(ℓ)
Γ
(ℓ)
1 , Γ
(ℓ)
2
)
R(ℓ+1)
= arg min
R
L
(
Φ(ℓ+1)
, Q(ℓ+1)
, R, Γ
(ℓ)
1 , Γ
(ℓ)
2
)
Γ
(ℓ+1)
1 = Γ
(ℓ)
1 + ρ
(
Φ(ℓ+1)
− Q(ℓ+1)
)
Γ
(ℓ+1)
2 = Γ
(ℓ)
2 + ρ
(
Φ(ℓ+1)
− R(ℓ+1)
)
79
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Algorithm:at the ℓ-th iteration
Φ(ℓ+1)
=
1
2
(τ1Ω + ρIp − Y ′
Y )−1
{
ρ
(
Q(ℓ)
+ R(ℓ)
)
− Γ1 − Γ2
}
Q(ℓ+1)
= U(ℓ)
(
V (ℓ)
)′
R(ℓ+1)
=
1
ρ
Sτ2
(
ρΦ(ℓ+1)
+ Γ
(ℓ)
1
)
Γ
(ℓ+1)
1 = Γ
(ℓ)
1 + ρ
(
Φ(ℓ+1)
− Q(ℓ+1)
)
Γ
(ℓ+1)
2 = Γ
(ℓ)
2 + ρ
(
Φ(ℓ+1)
− R(ℓ+1)
)
• U(ℓ)D(ℓ)
(
V (ℓ)
)′
= Φ(ℓ+1)
+
1
ρ
Γ
(ℓ)
2 (SVD)
• Sτ (S) = {sign(sik) max(|sik| − τ, 0)}
80
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Algorithm:at the ℓ-th iteration
Φ(ℓ+1)
=
1
2
(τ1Ω + ρIp − Y ′
Y )−1
{
ρ
(
Q(ℓ)
+ R(ℓ)
)
− Γ1 − Γ2
}
Q(ℓ+1)
= U(ℓ)
(
V (ℓ)
)′
R(ℓ+1)
=
1
ρ
Sτ2
(
ρΦ(ℓ+1)
+ Γ
(ℓ)
1
)
Γ
(ℓ+1)
1 = Γ
(ℓ)
1 + ρ
(
Φ(ℓ+1)
− Q(ℓ+1)
)
Γ
(ℓ+1)
2 = Γ
(ℓ)
2 + ρ
(
Φ(ℓ+1)
− R(ℓ+1)
)
• U(ℓ)D(ℓ)
(
V (ℓ)
)′
= Φ(ℓ+1)
+
1
ρ
Γ
(ℓ)
2 (SVD)
• Sτ (S) = {sign(sik) max(|sik| − τ, 0)}
• All subproblems have closed-form solutions.
80
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Outline
1 Background
2 Regularized Principal Component Analysis
3 Computation algorithm
4 Numerical Example
5 Summary
81
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Example: Artificial Sea Surface Temperature Data
• Data settings:
Yi(sj) = ξi1φ1(sj) + ξi2φ2(sj) + ϵij;
– j = 1, . . . , p = 2780, i = 1, . . . , n = 60
– s1, . . . , s2780: located in the Indian Ocean
– ξi1 ∼ N(0, 101.7), ξi2 ∼ N(0, 17.1), cov(ξi1, ξi2) = 0
– ϵij ∼ N(0, 1)
– (τ1, τ2): selected by 5-fold CV
φ1(s) φ2(s)
−0.04 −0.02 0.00 0.02 0.04
82
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Result I: φ1(s)
True
PCA SpatPCA
−0.04 −0.02 0.00 0.02 0.04
83
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Result I: φ2(s)
True
PCA SpatPCA
−0.04 −0.02 0.00 0.02 0.04
84
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Result II: Performance
• Loss function: Loss( ˆCη) =
p∑
i=1
p∑
j=1
(
ˆCη(si, sj) − Cη(si, sj)
)2
• 50 replications
– γ: selected by 5-fold CV
85
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Result II: Performance
• Loss function: Loss( ˆCη) =
p∑
i=1
p∑
j=1
(
ˆCη(si, sj) − Cη(si, sj)
)2
• 50 replications
– γ: selected by 5-fold CV
• Boxplot:
q
q
q
0
5000
10000
15000
PCA SpatPCA
85
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Example: 8-hour average ozone data
• Region: midwestern US
• Number of effective sites: 67 (irregular locations)
• Number of time points: 89 days (June 3 through August 31, 1987)
• At each sites, time-series is linearly detrended
−93 −83
3744
longitude
latitude
illinois indiana
iowa
kentucky
michigan:south
missouri
ohio
wisconsin
86
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Result I: φ1(s)
PCA + interpolation SpatPCA
0.00 0.05 0.10 0.15 0.20
87
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Outline
1 Background
2 Regularized Principal Component Analysis
3 Computation algorithm
4 Numerical Example
5 Summary
88
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Summary
SpatPCA:
• higher-dimensional analysis → (low-dimensional) structure
• with the smoothness and sparseness penalties
• enhance physical interpretation
89
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Summary
SpatPCA:
• higher-dimensional analysis → (low-dimensional) structure
• with the smoothness and sparseness penalties
• enhance physical interpretation
• non-stationary spatial covariance function
• can cope with irregular locations
89
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Summary
SpatPCA:
• higher-dimensional analysis → (low-dimensional) structure
• with the smoothness and sparseness penalties
• enhance physical interpretation
• non-stationary spatial covariance function
• can cope with irregular locations
• simple and efficient algorithm
• R package: SpatPCA
– CRAN: https://cran.r-project.org/web/packages/SpatPCA/index.html
– GitHub: https://github.com/egpivo/SpatPCA
89
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Background Regularized Principal Component Analysis Computation algorithm Numerical Example Summary
Thanks for your attention!
90

Contenu connexe

Tendances

Lect 3 background mathematics for Data Mining
Lect 3 background mathematics for Data MiningLect 3 background mathematics for Data Mining
Lect 3 background mathematics for Data Mininghktripathy
 
Application of Principal Components Analysis in Quality Control Problem
Application of Principal Components Analysisin Quality Control ProblemApplication of Principal Components Analysisin Quality Control Problem
Application of Principal Components Analysis in Quality Control ProblemMaxwellWiesler
 
Principal Component Analysis and Clustering
Principal Component Analysis and ClusteringPrincipal Component Analysis and Clustering
Principal Component Analysis and ClusteringUsha Vijay
 
An intro to applied multi stat with r by everitt et al
An intro to applied multi stat with r by everitt et alAn intro to applied multi stat with r by everitt et al
An intro to applied multi stat with r by everitt et alRazzaqe
 
Pca(principal components analysis)
Pca(principal components analysis)Pca(principal components analysis)
Pca(principal components analysis)kalung0313
 
Dimension Reduction: What? Why? and How?
Dimension Reduction: What? Why? and How?Dimension Reduction: What? Why? and How?
Dimension Reduction: What? Why? and How?Kazi Toufiq Wadud
 
Principal component analysis, Code and Time Complexity
Principal component analysis, Code and Time ComplexityPrincipal component analysis, Code and Time Complexity
Principal component analysis, Code and Time ComplexityYounesCharfaoui
 
presentation 2019 04_09_rev1
presentation 2019 04_09_rev1presentation 2019 04_09_rev1
presentation 2019 04_09_rev1Hyun Wong Choi
 
Ke yi small summaries for big data
Ke yi small summaries for big dataKe yi small summaries for big data
Ke yi small summaries for big datajins0618
 
How Does Math Matter in Data Science
How Does Math Matter in Data ScienceHow Does Math Matter in Data Science
How Does Math Matter in Data ScienceMutia Ulfi
 
Matrix algebra in_r
Matrix algebra in_rMatrix algebra in_r
Matrix algebra in_rRazzaqe
 
Statistical Methods
Statistical MethodsStatistical Methods
Statistical Methodsguest2137aa
 
Simulation run statistics
Simulation run statisticsSimulation run statistics
Simulation run statisticschandan sharma
 
ENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICS
ENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICSENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICS
ENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICSVLSICS Design
 

Tendances (20)

PCA
PCAPCA
PCA
 
Pca
PcaPca
Pca
 
Principal Component Analysis
Principal Component AnalysisPrincipal Component Analysis
Principal Component Analysis
 
Lect 3 background mathematics for Data Mining
Lect 3 background mathematics for Data MiningLect 3 background mathematics for Data Mining
Lect 3 background mathematics for Data Mining
 
Application of Principal Components Analysis in Quality Control Problem
Application of Principal Components Analysisin Quality Control ProblemApplication of Principal Components Analysisin Quality Control Problem
Application of Principal Components Analysis in Quality Control Problem
 
Principal Component Analysis and Clustering
Principal Component Analysis and ClusteringPrincipal Component Analysis and Clustering
Principal Component Analysis and Clustering
 
Pca analysis
Pca analysisPca analysis
Pca analysis
 
An intro to applied multi stat with r by everitt et al
An intro to applied multi stat with r by everitt et alAn intro to applied multi stat with r by everitt et al
An intro to applied multi stat with r by everitt et al
 
Pca(principal components analysis)
Pca(principal components analysis)Pca(principal components analysis)
Pca(principal components analysis)
 
Dimension Reduction: What? Why? and How?
Dimension Reduction: What? Why? and How?Dimension Reduction: What? Why? and How?
Dimension Reduction: What? Why? and How?
 
Principal component analysis, Code and Time Complexity
Principal component analysis, Code and Time ComplexityPrincipal component analysis, Code and Time Complexity
Principal component analysis, Code and Time Complexity
 
presentation 2019 04_09_rev1
presentation 2019 04_09_rev1presentation 2019 04_09_rev1
presentation 2019 04_09_rev1
 
Ke yi small summaries for big data
Ke yi small summaries for big dataKe yi small summaries for big data
Ke yi small summaries for big data
 
How Does Math Matter in Data Science
How Does Math Matter in Data ScienceHow Does Math Matter in Data Science
How Does Math Matter in Data Science
 
Scaling and Normalization
Scaling and NormalizationScaling and Normalization
Scaling and Normalization
 
Matrix algebra in_r
Matrix algebra in_rMatrix algebra in_r
Matrix algebra in_r
 
Statistical Methods
Statistical MethodsStatistical Methods
Statistical Methods
 
Simulation run statistics
Simulation run statisticsSimulation run statistics
Simulation run statistics
 
ENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICS
ENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICSENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICS
ENHANCING MULTIPLIER SPEED IN FAST FOURIER TRANSFORM BASED ON VEDIC MATHEMATICS
 
Data structures
Data structuresData structures
Data structures
 

En vedette

Steps for Principal Component Analysis (pca) using ERDAS software
Steps for Principal Component Analysis (pca) using ERDAS softwareSteps for Principal Component Analysis (pca) using ERDAS software
Steps for Principal Component Analysis (pca) using ERDAS softwareSwetha A
 
MVFI Meeting (January 14th, 2011)
MVFI Meeting (January 14th, 2011)MVFI Meeting (January 14th, 2011)
MVFI Meeting (January 14th, 2011)ivangomezconde
 
Principal Component Analysis For Novelty Detection
Principal Component Analysis For Novelty DetectionPrincipal Component Analysis For Novelty Detection
Principal Component Analysis For Novelty DetectionJordan McBain
 
Principal Component Analysis(PCA) understanding document
Principal Component Analysis(PCA) understanding documentPrincipal Component Analysis(PCA) understanding document
Principal Component Analysis(PCA) understanding documentNaveen Kumar
 
"Principal Component Analysis - the original paper" presentation @ Papers We ...
"Principal Component Analysis - the original paper" presentation @ Papers We ..."Principal Component Analysis - the original paper" presentation @ Papers We ...
"Principal Component Analysis - the original paper" presentation @ Papers We ...Adrian Florea
 
Principal component analysis and matrix factorizations for learning (part 2) ...
Principal component analysis and matrix factorizations for learning (part 2) ...Principal component analysis and matrix factorizations for learning (part 2) ...
Principal component analysis and matrix factorizations for learning (part 2) ...zukun
 
fauvel_igarss.pdf
fauvel_igarss.pdffauvel_igarss.pdf
fauvel_igarss.pdfgrssieee
 
Nonlinear component analysis as a kernel eigenvalue problem
Nonlinear component analysis as a kernel eigenvalue problemNonlinear component analysis as a kernel eigenvalue problem
Nonlinear component analysis as a kernel eigenvalue problemMichele Filannino
 
Kernel Entropy Component Analysis in Remote Sensing Data Clustering.pdf
Kernel Entropy Component Analysis in Remote Sensing Data Clustering.pdfKernel Entropy Component Analysis in Remote Sensing Data Clustering.pdf
Kernel Entropy Component Analysis in Remote Sensing Data Clustering.pdfgrssieee
 
Different kind of distance and Statistical Distance
Different kind of distance and Statistical DistanceDifferent kind of distance and Statistical Distance
Different kind of distance and Statistical DistanceKhulna University
 
KPCA_Survey_Report
KPCA_Survey_ReportKPCA_Survey_Report
KPCA_Survey_ReportRandy Salm
 
Analyzing Kernel Security and Approaches for Improving it
Analyzing Kernel Security and Approaches for Improving itAnalyzing Kernel Security and Approaches for Improving it
Analyzing Kernel Security and Approaches for Improving itMilan Rajpara
 
Adaptive anomaly detection with kernel eigenspace splitting and merging
Adaptive anomaly detection with kernel eigenspace splitting and mergingAdaptive anomaly detection with kernel eigenspace splitting and merging
Adaptive anomaly detection with kernel eigenspace splitting and mergingieeepondy
 
Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...
Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...
Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...hanshang
 
Explicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdf
Explicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdfExplicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdf
Explicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdfgrssieee
 
A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...
A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...
A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...Sahidul Islam
 
Pca and kpca of ecg signal
Pca and kpca of ecg signalPca and kpca of ecg signal
Pca and kpca of ecg signales712
 
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at GoogleDataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at GoogleHakka Labs
 
Probabilistic PCA, EM, and more
Probabilistic PCA, EM, and moreProbabilistic PCA, EM, and more
Probabilistic PCA, EM, and morehsharmasshare
 

En vedette (20)

Principal component analysis
Principal component analysisPrincipal component analysis
Principal component analysis
 
Steps for Principal Component Analysis (pca) using ERDAS software
Steps for Principal Component Analysis (pca) using ERDAS softwareSteps for Principal Component Analysis (pca) using ERDAS software
Steps for Principal Component Analysis (pca) using ERDAS software
 
MVFI Meeting (January 14th, 2011)
MVFI Meeting (January 14th, 2011)MVFI Meeting (January 14th, 2011)
MVFI Meeting (January 14th, 2011)
 
Principal Component Analysis For Novelty Detection
Principal Component Analysis For Novelty DetectionPrincipal Component Analysis For Novelty Detection
Principal Component Analysis For Novelty Detection
 
Principal Component Analysis(PCA) understanding document
Principal Component Analysis(PCA) understanding documentPrincipal Component Analysis(PCA) understanding document
Principal Component Analysis(PCA) understanding document
 
"Principal Component Analysis - the original paper" presentation @ Papers We ...
"Principal Component Analysis - the original paper" presentation @ Papers We ..."Principal Component Analysis - the original paper" presentation @ Papers We ...
"Principal Component Analysis - the original paper" presentation @ Papers We ...
 
Principal component analysis and matrix factorizations for learning (part 2) ...
Principal component analysis and matrix factorizations for learning (part 2) ...Principal component analysis and matrix factorizations for learning (part 2) ...
Principal component analysis and matrix factorizations for learning (part 2) ...
 
fauvel_igarss.pdf
fauvel_igarss.pdffauvel_igarss.pdf
fauvel_igarss.pdf
 
Nonlinear component analysis as a kernel eigenvalue problem
Nonlinear component analysis as a kernel eigenvalue problemNonlinear component analysis as a kernel eigenvalue problem
Nonlinear component analysis as a kernel eigenvalue problem
 
Kernel Entropy Component Analysis in Remote Sensing Data Clustering.pdf
Kernel Entropy Component Analysis in Remote Sensing Data Clustering.pdfKernel Entropy Component Analysis in Remote Sensing Data Clustering.pdf
Kernel Entropy Component Analysis in Remote Sensing Data Clustering.pdf
 
Different kind of distance and Statistical Distance
Different kind of distance and Statistical DistanceDifferent kind of distance and Statistical Distance
Different kind of distance and Statistical Distance
 
KPCA_Survey_Report
KPCA_Survey_ReportKPCA_Survey_Report
KPCA_Survey_Report
 
Analyzing Kernel Security and Approaches for Improving it
Analyzing Kernel Security and Approaches for Improving itAnalyzing Kernel Security and Approaches for Improving it
Analyzing Kernel Security and Approaches for Improving it
 
Adaptive anomaly detection with kernel eigenspace splitting and merging
Adaptive anomaly detection with kernel eigenspace splitting and mergingAdaptive anomaly detection with kernel eigenspace splitting and merging
Adaptive anomaly detection with kernel eigenspace splitting and merging
 
Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...
Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...
Modeling and forecasting age-specific mortality: Lee-Carter method vs. Functi...
 
Explicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdf
Explicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdfExplicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdf
Explicit Signal to Noise Ratio in Reproducing Kernel Hilbert Spaces.pdf
 
A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...
A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...
A Comparative Study between ICA (Independent Component Analysis) and PCA (Pri...
 
Pca and kpca of ecg signal
Pca and kpca of ecg signalPca and kpca of ecg signal
Pca and kpca of ecg signal
 
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at GoogleDataEngConf: Feature Extraction: Modern Questions and Challenges at Google
DataEngConf: Feature Extraction: Modern Questions and Challenges at Google
 
Probabilistic PCA, EM, and more
Probabilistic PCA, EM, and moreProbabilistic PCA, EM, and more
Probabilistic PCA, EM, and more
 

Similaire à Regularized Principal Component Analysis for Spatial Data

5 DimensionalityReduction.pdf
5 DimensionalityReduction.pdf5 DimensionalityReduction.pdf
5 DimensionalityReduction.pdfRahul926331
 
A practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningA practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningBruno Gonçalves
 
13_Unsupervised Learning.pdf
13_Unsupervised Learning.pdf13_Unsupervised Learning.pdf
13_Unsupervised Learning.pdfEmanAsem4
 
Sampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying FrameworkSampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying FrameworkNesreen K. Ahmed
 
super-cheatsheet-artificial-intelligence.pdf
super-cheatsheet-artificial-intelligence.pdfsuper-cheatsheet-artificial-intelligence.pdf
super-cheatsheet-artificial-intelligence.pdfssuser089265
 
DimensionalityReduction.pptx
DimensionalityReduction.pptxDimensionalityReduction.pptx
DimensionalityReduction.pptx36rajneekant
 
Probability and Statistics Cookbook
Probability and Statistics CookbookProbability and Statistics Cookbook
Probability and Statistics CookbookChairat Nuchnuanrat
 
Mat189: Cluster Analysis with NBA Sports Data
Mat189: Cluster Analysis with NBA Sports DataMat189: Cluster Analysis with NBA Sports Data
Mat189: Cluster Analysis with NBA Sports DataKathleneNgo
 
Regularized Estimation of Spatial Patterns
Regularized Estimation of Spatial PatternsRegularized Estimation of Spatial Patterns
Regularized Estimation of Spatial PatternsWen-Ting Wang
 
Solutions to Statistical infeence by George Casella
Solutions to Statistical infeence by George CasellaSolutions to Statistical infeence by George Casella
Solutions to Statistical infeence by George CasellaJamesR0510
 
Accelerating Metropolis Hastings with Lightweight Inference Compilation
Accelerating Metropolis Hastings with Lightweight Inference CompilationAccelerating Metropolis Hastings with Lightweight Inference Compilation
Accelerating Metropolis Hastings with Lightweight Inference CompilationFeynman Liang
 
Data Profiling in Apache Calcite
Data Profiling in Apache CalciteData Profiling in Apache Calcite
Data Profiling in Apache CalciteJulian Hyde
 

Similaire à Regularized Principal Component Analysis for Spatial Data (20)

ClusteringDEC
ClusteringDECClusteringDEC
ClusteringDEC
 
Machine learning cheat sheet
Machine learning cheat sheetMachine learning cheat sheet
Machine learning cheat sheet
 
Discrete mathematics
Discrete mathematicsDiscrete mathematics
Discrete mathematics
 
5 DimensionalityReduction.pdf
5 DimensionalityReduction.pdf5 DimensionalityReduction.pdf
5 DimensionalityReduction.pdf
 
A practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) LearningA practical Introduction to Machine(s) Learning
A practical Introduction to Machine(s) Learning
 
13_Unsupervised Learning.pdf
13_Unsupervised Learning.pdf13_Unsupervised Learning.pdf
13_Unsupervised Learning.pdf
 
Ee3054 exercises
Ee3054 exercisesEe3054 exercises
Ee3054 exercises
 
Sampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying FrameworkSampling from Massive Graph Streams: A Unifying Framework
Sampling from Massive Graph Streams: A Unifying Framework
 
super-cheatsheet-artificial-intelligence.pdf
super-cheatsheet-artificial-intelligence.pdfsuper-cheatsheet-artificial-intelligence.pdf
super-cheatsheet-artificial-intelligence.pdf
 
DimensionalityReduction.pptx
DimensionalityReduction.pptxDimensionalityReduction.pptx
DimensionalityReduction.pptx
 
MAINPH
MAINPHMAINPH
MAINPH
 
Probability and Statistics Cookbook
Probability and Statistics CookbookProbability and Statistics Cookbook
Probability and Statistics Cookbook
 
pattern recognition
pattern recognition pattern recognition
pattern recognition
 
Mat189: Cluster Analysis with NBA Sports Data
Mat189: Cluster Analysis with NBA Sports DataMat189: Cluster Analysis with NBA Sports Data
Mat189: Cluster Analysis with NBA Sports Data
 
Regularized Estimation of Spatial Patterns
Regularized Estimation of Spatial PatternsRegularized Estimation of Spatial Patterns
Regularized Estimation of Spatial Patterns
 
Solutions to Statistical infeence by George Casella
Solutions to Statistical infeence by George CasellaSolutions to Statistical infeence by George Casella
Solutions to Statistical infeence by George Casella
 
XGBoostLSS - An extension of XGBoost to probabilistic forecasting, Alexander ...
XGBoostLSS - An extension of XGBoost to probabilistic forecasting, Alexander ...XGBoostLSS - An extension of XGBoost to probabilistic forecasting, Alexander ...
XGBoostLSS - An extension of XGBoost to probabilistic forecasting, Alexander ...
 
MLE.pdf
MLE.pdfMLE.pdf
MLE.pdf
 
Accelerating Metropolis Hastings with Lightweight Inference Compilation
Accelerating Metropolis Hastings with Lightweight Inference CompilationAccelerating Metropolis Hastings with Lightweight Inference Compilation
Accelerating Metropolis Hastings with Lightweight Inference Compilation
 
Data Profiling in Apache Calcite
Data Profiling in Apache CalciteData Profiling in Apache Calcite
Data Profiling in Apache Calcite
 

Dernier

chemical bonding Essentials of Physical Chemistry2.pdf
chemical bonding Essentials of Physical Chemistry2.pdfchemical bonding Essentials of Physical Chemistry2.pdf
chemical bonding Essentials of Physical Chemistry2.pdfTukamushabaBismark
 
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICESAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICEayushi9330
 
module for grade 9 for distance learning
module for grade 9 for distance learningmodule for grade 9 for distance learning
module for grade 9 for distance learninglevieagacer
 
Dubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai Young
Dubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai YoungDubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai Young
Dubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai Youngkajalvid75
 
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Monika Rani
 
pumpkin fruit fly, water melon fruit fly, cucumber fruit fly
pumpkin fruit fly, water melon fruit fly, cucumber fruit flypumpkin fruit fly, water melon fruit fly, cucumber fruit fly
pumpkin fruit fly, water melon fruit fly, cucumber fruit flyPRADYUMMAURYA1
 
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...Silpa
 
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts ServiceJustdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Servicemonikaservice1
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxMohamedFarag457087
 
Grade 7 - Lesson 1 - Microscope and Its Functions
Grade 7 - Lesson 1 - Microscope and Its FunctionsGrade 7 - Lesson 1 - Microscope and Its Functions
Grade 7 - Lesson 1 - Microscope and Its FunctionsOrtegaSyrineMay
 
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...chandars293
 
GBSN - Microbiology (Unit 3)
GBSN - Microbiology (Unit 3)GBSN - Microbiology (Unit 3)
GBSN - Microbiology (Unit 3)Areesha Ahmad
 
Factory Acceptance Test( FAT).pptx .
Factory Acceptance Test( FAT).pptx       .Factory Acceptance Test( FAT).pptx       .
Factory Acceptance Test( FAT).pptx .Poonam Aher Patil
 
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPirithiRaju
 
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxCOST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxFarihaAbdulRasheed
 
Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.Silpa
 
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Servicenishacall1
 
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 bAsymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 bSérgio Sacani
 

Dernier (20)

chemical bonding Essentials of Physical Chemistry2.pdf
chemical bonding Essentials of Physical Chemistry2.pdfchemical bonding Essentials of Physical Chemistry2.pdf
chemical bonding Essentials of Physical Chemistry2.pdf
 
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICESAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
 
module for grade 9 for distance learning
module for grade 9 for distance learningmodule for grade 9 for distance learning
module for grade 9 for distance learning
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Dubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai Young
Dubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai YoungDubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai Young
Dubai Call Girls Beauty Face Teen O525547819 Call Girls Dubai Young
 
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
 
pumpkin fruit fly, water melon fruit fly, cucumber fruit fly
pumpkin fruit fly, water melon fruit fly, cucumber fruit flypumpkin fruit fly, water melon fruit fly, cucumber fruit fly
pumpkin fruit fly, water melon fruit fly, cucumber fruit fly
 
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
Locating and isolating a gene, FISH, GISH, Chromosome walking and jumping, te...
 
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts ServiceJustdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptx
 
Grade 7 - Lesson 1 - Microscope and Its Functions
Grade 7 - Lesson 1 - Microscope and Its FunctionsGrade 7 - Lesson 1 - Microscope and Its Functions
Grade 7 - Lesson 1 - Microscope and Its Functions
 
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
 
GBSN - Microbiology (Unit 3)
GBSN - Microbiology (Unit 3)GBSN - Microbiology (Unit 3)
GBSN - Microbiology (Unit 3)
 
Factory Acceptance Test( FAT).pptx .
Factory Acceptance Test( FAT).pptx       .Factory Acceptance Test( FAT).pptx       .
Factory Acceptance Test( FAT).pptx .
 
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
 
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxCOST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
 
Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.
 
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
 
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 bAsymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
 
Site Acceptance Test .
Site Acceptance Test                    .Site Acceptance Test                    .
Site Acceptance Test .
 

Regularized Principal Component Analysis for Spatial Data