A simulation study examining the effects of `regularization' on estimates of genetic covariance matrices for small samples is presented. This is achieved by penalizing the likelihood, and three types of penalties are examined. It is shown that regularized estimation can substantially enhance the accuracy of estimates of genetic parameters. Penalties shrinking estimates of genetic covariances or correlations towards their phenotypic counterparts acted somewhat differently to those aimed reducing the spread of sample eigenvalues. While improvements of estimates were found to be comparable overall, shrinkage of genetic towards phenotypic correlations resulted in least bias.

1. Penalized maximum likelihood
estimates of genetic covariance
matrices with shrinkage towards
phenotypic dispersion
Karin Meyer1
, Mark Kirkpatrick2
, Daniel Gianola3
1
Animal Genetics and Breeding Unit, University of New England, Armidale
2
Section of Integrative Biology, University of Texas, Austin
3
University of Wisconsin-Madison, Madison
AAABG 2011

2. Penalized REML | Introduction
Motivation
Multivariate genetic analyses: more than 2-4 traits
→ desirable!
→ technically increasingly feasible
→ inherently problematic
SAMPLING VARIANCE ↑↑ with no. of traits
K. M. / M. K. / D. G. | | AAABG 2011 2 / 13

3. Penalized REML | Introduction
Motivation
Multivariate genetic analyses: more than 2-4 traits
→ desirable!
→ technically increasingly feasible
→ inherently problematic
SAMPLING VARIANCE ↑↑ with no. of traits
Measures to alleviate S.V.
large → gigantic data sets
parsimonious models → less parameters than covar.s
estimation → use additional information
Bayesian: Prior
REML: Impose penalty P on likelihood
→ P = f(parameters)
→ designed to reduce S.V.
K. M. / M. K. / D. G. | | AAABG 2011 2 / 13

4. Penalized REML | Introduction
Penalized REML
Maximize:
log LP = log L − 1
2
ψ P
Tuning factor
Penalty on parameters
Standard, unpenalized REML log likelihood
Objectives:
Introduce new type of P
→ prior: Σ ∼ IW
Compare efficacy of different P
K. M. / M. K. / D. G. | | AAABG 2011 3 / 13

5. Penalized REML | REML and penalties
Penalties to improve estimates of ΣG
ˆΣP estimated much more accurately than ˆΣG
Idea: ‘Borrow strength’ from ˆΣP
1 Shrink canonical eigenvalues towards their mean
→ ‘bending’ (Hayes & Hill 1981)
Pℓ
λ
∝ i(log λi − ¯λ)2
λi : eig.values of ˆΣ
−1
P
ˆΣG
K. M. / M. K. / D. G. | | AAABG 2011 4 / 13

6. Penalized REML | REML and penalties
Penalties to improve estimates of ΣG
ˆΣP estimated much more accurately than ˆΣG
Idea: ‘Borrow strength’ from ˆΣP
1 Shrink canonical eigenvalues towards their mean
→ ‘bending’ (Hayes & Hill 1981)
Pℓ
λ
∝ i(log λi − ¯λ)2
λi : eig.values of ˆΣ
−1
P
ˆΣG
2 a) Shrink ˆΣG towards ˆΣP
→ assume ΣG ∼ IW(Σ−1
P
, ψ)
→ obtain penalty as minus log density of IW
PΣ ∝ C log | ˆΣG| + tr ˆΣ−1
G
ˆΣ0
P
b) Shrink ˆRG towards ˆRP
PR ∝ C log |ˆRG| + tr ˆR−1
G
ˆR0
P
K. M. / M. K. / D. G. | | AAABG 2011 4 / 13

7. Penalized REML | Simulation study
Simulation
Paternal half-sib design: s = 100, n = 10
5 traits, h2
1
≥ . . . ≥ h2
5
, MVN
60 sets of population values
→ vary mean & spread of λi
1000 replicates per case
3 penalties
Pℓ
λ
regress log( ˆλi) towards their mean
PΣ shrink ˆΣG towards ˆΣP
PR shrink ˆRG towards ˆRP
Obtain ˆΣψ
G
and ˆΣψ
E
for values of ψ, range 0 − 1000
K. M. / M. K. / D. G. | | AAABG 2011 5 / 13

8. Penalized REML | Simulation study
Simulation - cont.
Estimate ψ using population values (V∞)
→ construct MSB & MSW → validation
→ ˆψ maximize log L in valid. data
Evaluate effect of penalty
→ Loss: deviation of ˆΣX from ΣX
L1(ΣX, ˆΣX) = tr(Σ−1
X
ˆΣX) − log |Σ−1
X
ˆΣX| − q
→ Percentage Reduction In Average Loss
PRIAL = 100






1 −
¯L1(ΣX, ˆΣ
ˆψ
X
)
¯L1(ΣX, ˆΣ0
X
)






K. M. / M. K. / D. G. | | AAABG 2011 6 / 13
penalized
unpenalized

9. Penalized REML | Results | PRIAL
PRIAL in estimated covariance matrices
Pλ
l
PΣ PR
40
60
80
100
q
q
q
q
q
q
q
Genetic
71.4 70.6 72.3
Pλ
l
PΣ PR
0
20
40
60
80
Residual
43.4 13.3 37.1
Pλ
l
PΣ PR
0
2
4
6
8
q
q
q
qq
q
qq
q
q
q
q
q
Phenotypic
1.2 1.2 2.2
K. M. / M. K. / D. G. | | AAABG 2011 7 / 13

10. Penalized REML | Results | PRIAL
PRIAL for ˆΣG – 60 individual cases
in order of PRIAL for Pℓ
λ
5K
5L
5J
5D
2K
5H
5F
2F
5E
2L
4F
4K
2H
4L
5I
2D
4H
3K
3F
2E
5C
1D
4D
3L
1C
3H
3D
2J
5G
4J
4E
5B
3E
5A
4I
1G
3C
3A
3B
1E
3G
3J
1F
4A
2I
4C
4B
1K
3I
4G
2C
1B
2G
1H
2B
2A
1L
1J
1A
1I
40
60
80
100
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q q q q q
q q
q q q
q
q q
q q
q
q
q q
q
q
q
q q
q
q
q
q
q
Penalty
Pλ
l
PΣ PR
K. M. / M. K. / D. G. | | AAABG 2011 8 / 13

11. Penalized REML | Results | PRIAL
Extension: PRIAL “double” penalties
P so far: improve ˆΣG
“Double”
→ PΣ on ˆΣG & ˆΣE
PR on ˆRG & ˆRE
Pℓ
λ
on log λi &
log(1−λi)
→ 1: joint ψ
2: separate ψ
Pℓ
λ
PΣ PR
ˆΣG G 71 71 72
G+E 1 73 70 73
2 74 73 74
ˆΣE G 43 13 37
G+E 1 62 54 60
2 65 65 63
Little impact on PRIAL for ˆΣG
Can ↑↑ PRIAL for ˆΣE w/out ↓ PRIAL for ˆΣG
Separate ˆψ: extra effort, limited add. improvement
K. M. / M. K. / D. G. | | AAABG 2011 9 / 13

12. Penalized REML | Results | PRIAL
Summary: PRIAL
Substantial improvements in ˆΣG & ˆΣE feasible
→ Results shown ‘optimistic’ → ˆψ using pop.values
PRIAL heavily influenced by spread of can. eigenvalues
Pℓ
λ
best if λi ≈ ¯λ; overshrink if not
PΣ & PR worst if λi ≈ ¯λ; more robust than Pℓ
λ
if λi ≈ ¯λ
similar canonical eigenvalues unlikely in practice?
New type of P useful
K. M. / M. K. / D. G. | | AAABG 2011 10 / 13

13. Penalized REML | Results | Bias
Mean estimates of can. eigenvalues
Penalties act differently – illustrated for single case
λi = 0.5, 0.2, 0.15, 0.1, 0.05; ¯λ = 0.2
0
0.2
0.4
λ1 λ3 λ5
q
q
q
q
q
None
q
q
q
q
q
Pλ
l
λ1 λ3 λ5
q
q
q
q
q
PΣ
q
q
q
q
q
q
q
q
q
q
PR
q Pop.value
K. M. / M. K. / D. G. | | AAABG 2011 11 / 13

14. Penalized REML | Results | Bias
Mean relative bias (in %)
1 2 3 4 5
Can. eigenvalues
None 9 27 17 -20 -79
Pℓ
λ
-4 16 29 58 101
PΣ 8 25 25 39 75
PR 1 16 21 37 57
Heritabilities
None -1 4 5 7 12
Pℓ
λ
-7 5 12 23 45
PΣ 1 10 15 26 44
PR -2 2 5 9 17
K. M. / M. K. / D. G. | | AAABG 2011 12 / 13

15. Penalized REML | Finale
Conclusions
Can obtain ‘better’ estimates of genetic cov. matrices
→ borrow strength from phenotypic covariance matrix
→ easy to implement by penalty on log L
K. M. / M. K. / D. G. | | AAABG 2011 13 / 13

16. Penalized REML | Finale
Conclusions
Can obtain ‘better’ estimates of genetic cov. matrices
→ borrow strength from phenotypic covariance matrix
→ easy to implement by penalty on log L
Comparable improvement (PRIAL) from different P
None best overall
PR: Shrinking ˆRG towards ˆRP
less variable
less bias in estimates of genetic parameters
easier to implement than Pℓ
λ
K. M. / M. K. / D. G. | | AAABG 2011 13 / 13

17. Penalized REML | Finale
Conclusions
Can obtain ‘better’ estimates of genetic cov. matrices
→ borrow strength from phenotypic covariance matrix
→ easy to implement by penalty on log L
Comparable improvement (PRIAL) from different P
None best overall
PR: Shrinking ˆRG towards ˆRP
less variable
less bias in estimates of genetic parameters
easier to implement than Pℓ
λ
Penalized REML recommended!
Small to moderate data sets, ≥ 4 traits
K. M. / M. K. / D. G. | | AAABG 2011 13 / 13

18. P-REML
part of
everyday
toolkit