Call girls in new Ashok NagarDelhi꧁ 8447779280꧂ Escort Service Women Seeking ...
Reading the Lindley-Smith 1973 paper on linear Bayes estimators
1. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
.
.
Bayes Estimates for the Linear Model
.
.. .
.
Reading Seminar in Statistical Classics
Director: C. P. Robert
Presenter: Kaniav Kamary
12 Novembre, 2012
2. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Outline
.
. . Introduction
1
The Model and the bayesian methods
.
. . Exchangeability
2
.
. . General bayesian linear model
3
.
. . Examples
4
.
. . Estimation with unknown Covariance
5
3. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
The Model and the bayesian methods
The linear model :
Structure of the linear model:
E(y ) = Aθ
y : a vector of the random variables
A: a known design Matrix
Θ: unknown parameters
For estimating Θ:
The usual estimate by the method of least squares.
Unsatisfactory or inadmissibility in demensions greater
than two.
Improved estimates with knowing prior information about
the parameters in the bayesian framework
4. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Outline
.
. . Introduction
1
.
. . Exchangeability
2
un example
.
. . General bayesian linear model
3
.
. . Examples
4
.
. . Estimation with unknown Covariance
5
5. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
un example
The concept of exchangeability
In general linear model suppose A = I :
E(yi ) = Θi for i = 1, 2, . . . , n and yi ∼ N(θi , σ 2 ) iid
The distribution of θi is exchangeable if:
The prior opinion of θi is the same of that of θj or any other θk
where i, j, k = 1, 2, . . . , n.
In the other hand:
A sequence θ1 , . . . , θn of random variables is said to be
exchangeable if for all k = 2, 3, . . .
θ1 , . . . , θn ∼ θπ(1) , θπ(2) , θπ(k)
=
for all π ∈ S(k ) where S(k ) is the group of permutation of
1, 2, . . . , k
6. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
un example
The concept of exchangeability. . .
One way for obtaining an exchangeable distribution p(Θ):
∏
n
p(Θ) = p(θi | µ)dQ(µ) (1)
i=1
p(Θ): exchangeable prior knowledge described by a mixture
Q(µ): arbitrary probability distribution for each µ
µ: the hyperparameters
A linear structure to the parameters:
E(θi ) = µ
7. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
un example
Estimate of Θ
If θi ∼ N(µ, τ 2 ):
a closer parallelism between the two stage for y and Θ
By assuming that µ have a uniform distribution over the real line
then: yi y.
2 + τ2
θi∗ = σ
1 1
(2)
σ 2 + τ2
∑n
i=1 yi
where y. = n and θi∗ = E(θi | y ).
8. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
un example
The features of θi∗
ˆ
A weighted averages of yi = θi , overall mean y. and
inversely proportional to the variances of yi and θi
A biased estimate of θi
Use the estimates of τ 2 and σ 2
An admissible estimate with known σ 2 , τ 2
A bayes estimates as substitution for the usual
least-squares estimates
9. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
un example
The features of θi∗ . . .
.
Judging the merit of θi with one of the other estimates .
..
The condition that the average M.S.E for θi ∗ to be less than that
ˆ
for θi is: ∑
(θi − θ. )2
< 2τ 2 + σ 2 (3)
n−1
∑
s2 = (θi −θ. ) is an usual estimate for τ 2 . Hence, the chance of
2
n−1
unequal (3) being satisfied is high for n as law as 4 and rapidly
tends to 1 as n increases.
.
.. .
.
10. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Outline
.
. . Introduction
1
.
. . Exchangeability
2
.
. . General bayesian linear model
3
The posterior distribution of the parameters
.
. . Examples
4
.
. . Estimation with unknown Covariance
5
11. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
The posterior distribution of the parameters
The structure of the model
Let:
Y : a column vector
.
Lemma .
..
Suppose Y ∼ N(A1 Θ1 , C1 ) and Θ1 ∼ N(A2 Θ2 , C2 ) that Θ1 is a
vector of P1 parameters, that Θ2 is a vector of P2
hyperparameters.
Then (a): Y ∼ N(A1 A2 Θ2 , C1 + A1 C2 AT ),
1
and (b): Θ1 | Y ∼ N(Bb, B) where:
−1 −1
B −1 = AT C1 A1 + C2
1
−1 −1
. b = AT C1 y + C2 A2 Θ2
1 (4)
.. .
.
12. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
The posterior distribution of the parameters
The posterior distribution with three stages
.
Theorem .
..
With the assumptions of the Lemma, suppose that given Θ3 ,
Θ2 ∼ N(A3 Θ3 , C3 )
then for i = 1, 2, 3:
Θ1 | {Ai }, {Ci }, Θ3 , Y ∼ N(Dd, D)
with
−1 −1
D −1 = AT C1 A1 + {C2 + A2 C3 AT }
1 2 (5)
and
T −1 T −1 −1
. d = A1 C1 y + A1 C1 A1 + {C2 + A2 C3 A2 }
T
A2 A3 Θ3 (6)
.. .
.
13. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
The posterior distribution of the parameters
The properties
.
Result of the Lemma .
..
For any matrices A1 , A2 , C1 and C2 of appropriate dimensions
and for witch the inverses stated, we have:
−1 −1 −1 −1 −1
C1 − C1 A1 (AT C1 A1 + C2 )−1 AT C1 = (C1 + A1 C2 AT )
1 1 1
. (7)
.. .
.
.
Properties of the bayesian estimation .
..
The E(Θ1 | {Ai }, {Ci }, Θ3 , Y ) is:
A weighed average of the least-squares estimates
−1 −1 −1
(AT C1 A1 ) AT C1 y .
1 1
A weithed average of the prior mean A2 A3 Θ3 .
It may be regarded as a point estimate of Θ1 to replace the
usual least-squares estimate.
.
.. .
.
14. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
The posterior distribution of the parameters
Results of the Theorem
.
Corollary1 .
..
An alternative expression for D −1 :
T −1 −1 −1 T −1 −1 T −1 −1
. A1 C1 A1 + C2 − C2 A2 {A2 C2 A2 + C3 } A2 C2 (8)
.. .
.
.
Corollary2 .
..
−1
If C3 = 0, the posterior distribution of Θ1 is N(D0 d0 , D0 ) with:
−1 −1 −1 −1 −1 −1 T −1
D0 = AT C1 A1 + C2 − C2 A2 {AT C2 A2 }
1 2 A2 C2 (9)
and
−1
. d0 = AT C1 y
1 (10)
.. .
.
15. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Outline
.
. . Introduction
1
.
. . Exchangeability
2
.
. . General bayesian linear model
3
.
. . Examples
4
Two-factor Experimental Designs
Exchangeability Between Multiple Regression Equation
Exchangeability within Multiple Regression Equation
.
. . Estimation with unknown Covariance
5
16. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Two-factor Experimental Designs
The structure of the Two-factor Experimental Designs
The usual model of n observations with the errors
independent N(0, σ 2 ):
E(yij ) = µ + αi + βj , 1 ≤ i ≤ t, 1 ≤ j ≤ b
ΘT
1 = (µ, α1 , . . . , αt , β1 , . . . , βb ) (11)
yij : an observation in the ith treatment and the jth block.
The exchangeable prior knowledge of {αi } and {βj } but
independent
αi ∼ N(0, σα ), βj ∼ N(0, σβ ), µ ∼ N(w, σµ )
2 2 2
The vague prior knowledge of µ and σµ → ∞
2
−1
C2 : the diagonal matrix that leading diagonal of C2 is
−2 −2 −2 −2
(0, σα , . . . , σα , σβ , . . . , σβ )
C1 : the unit matrix times σ 2
17. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Two-factor Experimental Designs
Bayesian estimate of the parameters
With substituting the assumptions stated and C3 = 0 in to (5)
and (6), then:
−1
D −1 = σ −2 AT A1 + C2
1
d = σ −2 AT y
1 (12)
Hence Θ∗ , the bayes estimate Dd, satisfies the equation as
1
following
−1
(AT A1 + σ 2 C2 )Θ∗ = AT y
1 1 1 (13)
by solving (13),
µ = y..
−1
αi∗ = (bσα + σ 2 )
2
bσα (yi. − y.. )
2
−1
βj∗ = (tσβ + σ 2 )
2
tσβ (y.j − y.. )
2
(14)
18. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exchangeability Between Multiple Regression Equation
The structure of the Multiple Regression Equation
The usual model for p regressor variables where
j = 1, 2, . . . , m:
yj ∼ N(Xj βj , Inj σj2 ) (15)
A1 : a diagonal matrix with xj as the jth diagonal submatrix
ΘT = (β1 , β2 , . . . , βm )
1
T T T
Suppose variables X and Y were related with the usual linear
regression structure and
βj ∼ N(ξ, Σ), Θ2 = ξ
A2 : a matrix of order mp × p, all of whose p × p submatrices
are unit matrices
19. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exchangeability Between Multiple Regression Equation
Bayesian estimation for the parameters of the Multiple Regression
Equation. . .
The equation for the bayes estimates βj∗ is
σ1 −2 X1 T X1 + Σ−1 ··· 0
. .
.
. σ2 −2 X2 T X2 + Σ−1 .
.
0 ··· σm −2 Xm T Xm + Σ−1
β1 ∗ β. ∗ σ1 −2 X1 T y
β2 ∗ β. ∗ σ2 −2 X1 T y
−1
× . −Σ . = . (16)
.
. .
. .
.
βm ∗ β. ∗ σm −2 X1 T y
∑ βi ∗
where β. ∗ = m .
20. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exchangeability Between Multiple Regression Equation
Bayesian estimation for the parameters of the Multiple Regression
Equation
By solving equation (16) for βj ∗ , the bayes estimate is
−1
βj ∗ = (σj −2 Xj T Xj + Σ−1 ) (σj −2 Xj T y + Σ−1 β. ∗ ) (17)
Noting that D0 −1 , given in Corollary 2 (9) and the matrix
Lemma 7, we obtain a weighted form of (17) with β. ∗ replaced
∑
by wj βj ∗ :
−1
∑m
−1 −1
wi = { (σj −2 Xj T Xj + Σ−1 ) σj −2 Xj T Xj } (σi −2 Xi T Xi + Σ−1 ) σi −2 Xi T Xi
j=1
(18)
21. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exchangeability within Multiple Regression Equation
the model and bayes estimates of the parameters
A single multiple regression:
y ∼ N(X β, In σ 2 ) (19)
The individual regression coefficients in β T = (β1 , β2 , . . . , βp )
are exchangeable and βj ∼ N(ξ, σβ 2 ).
.
bayes estimate with two possibilities .
..
σ2
to suppose vague prior knowledge for ξ with k = σ2
β
β ∗ = {Ip + k (X T X )−1 (Ip − p−1 )}−1 β
ˆ (20)
to put ξ = 0, reflecting a feeling that the βi are small
β ∗ = {Ip + k (X T X )−1 }−1 β
ˆ (21)
.
.. .
.
22. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Outline
.
. . Introduction
1
.
. . Exchangeability
2
.
. . General bayesian linear model
3
.
. . Examples
4
.
. . Estimation with unknown Covariance
5
Exposition and method
Two-factor Experimental Designs(unknown Covariance)
Exch between Multiple Regression(unknown Covariance)
Exch within Multiple Regression(unknown Covariance)
23. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exposition and method
Method
θ: the parameters of interest in the general model
ϕ: the nuisance parameters
Ci : the unknown dispersion matrices
.
The method and its defect .
..
assign a joint prior distribution to θ and ϕ
provide the joint posterior distribution p(θ, ϕ | y )
integrating the joint posterior with respect to ϕ and leaving
the posterior for θ
for using loss function, necessity another integration for
calculate the mean
require the constant of proportionality in bayes’s formula
for calculating the mean
the
. above argument is technically most complex to execute.
.. .
.
24. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exposition and method
Solution
For simplified the method:
considering an approximation
using the mode of the posterior distribution in place of the
mean
using the mode of the joint distribution rather than that of
the θ-margin
taking the estimates derived in section 2 and replace the
unknown values of the nuisances parameters by their
modal estimates
25. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exposition and method
Solution. . .
The modal value:
∂ ∂
p(θ, ϕ | y ) = 0, p(θ, ϕ | y ) = 0
∂θ ∂ϕ
assuming that p(ϕ | y ) ̸= 0 as
∂
p(θ | y , ϕ) = 0 (22)
∂θ
The approximation is good if:
the samples are large
the resulting posterior distributions approximately normal
26. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Two-factor Experimental Designs(unknown Covariance)
The prior distributions for σ 2 , σα 2 and σβ 2 are invers-χ2 .
νλ να λα νβ λ β
2
∼ χν 2 , 2
∼ χνα 2 , ∼ χνβ 2
σ σα σβ 2
With assuming the three variances independent.
The joint distribution of all quantities:
−1 −1
(σ 2 ) 2
(n+ν+2)
× exp {νλ + S 2 (µ, α, β)}
2σ 2
−1
−1 ∑2
×(σα )
2 2
(t+να +2)
exp 2σ2 {να λα + αi }
2 −1 −1
α
∑2
×(σβ ) 2 (b+νβ +2) exp 2σ2 {νβ λβ + βj }
β
27. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Two-factor Experimental Designs(unknown Covariance)
Estimates of the parameters of the model
To find the modal estimates:
reversing the roles of θ and ϕ with supposing µ, α and β
known
{νλ + S 2 (µ∗ , α∗ , β ∗ )}
s2 =
(n + ν + 2)
∑
{να λα + αi ∗2 }
sα 2 =
(t + να + 2)
∑
{νβ λβ + βj ∗2 }
sβ 2 = (23)
(b + νβ + 2)
solving (13) with trial value of σ 2 , σα and σβ
2 2
inserting the value in to (23)
28. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch between Multiple Regression(unknown Covariance)
Suppositions of the model. . .
In the model (15), suppose σj 2 = σ 2 with νλ ∼ χν 2 and Σ−1 has
σ2
a Wishart distribution with ρ degree of freedom and matrix R
independent of σ 2 .
The joint distribution of all the quantities:
−1 ∑
m
−1n
(σ 2 ) 2 × exp{ (yj − Xj βj )T (yj − Xj βj )}
2σ 2
j=1
−1 ∑
m
−1m
× (| Σ |) 2 exp{ (βj − ξ)T Σ−1 (βj − ξ)}
2
j=1
−1
−1(ρ−p−1)
× (| Σ |) 2 tr Σ−1 R}
exp{
2
−1(ν+2) −νλ
× (σ 2 ) 2 exp{ 2 } (24)
2σ
29. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch between Multiple Regression(unknown Covariance)
The joint posterior distribution
The joint posterior density for β, σ 2 and Σ−1 :
−1 ∑
m
−1(n+ν+2) T
(σ )2 2 × exp{ 2 {m−1 νλ + (yj − Xj βj ) (yj − Xj βj )}}
2σ
j=1
−1(m+ρ−p−2)
× (| Σ |) 2
−1 ∑ m
× exp{ tr Σ−1 {R + (βj − β. )(βj − β. )T }} (25)
2
j=1
∑m
where β. = m−1 j=1 βj .
30. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch between Multiple Regression(unknown Covariance)
The modal estimates
.
The estimates of the parameters .
.. ∑m −1 ∗ T ∗
2 j=1 {m νλ + (yj − Xj βj ) (yj − Xj βj )}
s = (26)
(n + ν + 2)
and ∑m
{R + ∗ − β.∗ )(βj∗ − β.∗ )T }
∗ j=1 (βj
Σ = (27)
. (m + ρ − p − 2)
.. .
.
31. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch between Multiple Regression(unknown Covariance)
The modal estimates . . .
The posterior distribution of the βj ’s, free of σ 2 and Σ:
∑
n
1
{ {m−1 νλ + (yj − Xj βj )T (yj − Xj βj )}}− 2 (n+ν)
j=1
− 1 (m+ρ−1)
∑
m 2
×| R + (βj − β. )(βj − β. )T | (28)
j=1
The mode of this distribution can be used in place of the modal
values for the wider distribution.
32. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch between Multiple Regression(unknown Covariance)
Application
.
an application in an educational context .
..
data from the American Collage Testing Program 1968, 1969
prediction of grade-point average at 22 collages
the results of 4 tests (English, Mathematics, Social Studies,
Natural Sciences),p = 5, m = 22, and nj varying from 105 to 739
Table: Comparison of predictive efficiency
reduction the error by under 2 per cent by using the bayesian
method in the first row but 9 per cent with the quarter sample
.
.
33. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch within Multiple Regression(unknown Covariance)
Assumptions of the model regression
In the model 19 and βj ∼ N(ξ, σβ 2 ),
suppose
νλ νβ λ β
2
∼ χν 2 , ∼ χνβ 2
σ σβ 2
The posterior distribution of β, σ 2 and σβ 2 :
−1(n+ν+2) −1
(σ 2 ) 2 × exp{ {νλ + (y − X β)T (y − X β)}}
2σ 2
−1(p+νβ +1)
× (σβ 2 ) 2
−1 ∑ p
× exp{ 2
{νβ λβ + (βj − β. )2 }} (29)
2σβ
j=1
∑p
that β. = p−1 j=1 βj .
34. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch within Multiple Regression(unknown Covariance)
The modal estimation. . .
The modal equations:
−1
β ∗ = {Ip + k ∗ X T X (Ip − p−1 Jp )}−1 β
ˆ
{νλ + (y − X β ∗ )T (y − X β ∗ )}
s2 =
(n + ν + 2)
∑p
{νβ λβ + j=1 (βj ∗ − β. ∗ )2 }
sβ 2 = (30)
(p + νβ + 1)
where k ∗ = s2
sβ ∗ .
35. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch within Multiple Regression(unknown Covariance)
Comparison between the methods of the estimates
The main difference lies in the choice of k
in absolute value, the least-squares procedure produce
regression estimates too large, of incorrect sign and
unstable with respect to small changes in the data
The ridge method avoid some of these undesirable
features
The bayesian method reaches the same conclusion but
has the added advantage of dispensing with the rather
arbitrary choice of k and allows the data to estimate it
Table: 10-factor multiple regression example
36. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch within Multiple Regression(unknown Covariance)
A brief explanation of a recent paper
On overview of the Bayesian Linear Model with unknown
Variance:
Yn×p = Xp×1 + ξ
The bayesian approache to fitting the linear model consists of
three steps (S.Kuns, 2009)[4]:
assign priors to all unknown parameters
write down the likelihood of the data given the parameters
determine the posterior distribution of the parameters
given the data using bayes’ theorem
If Y ∼ N(X β, k −1 ) then a conjugate prior distribution for the
parameters is: β, k ∼ NG(β0 , Σ0 , a, b). In other word:
p−2 −1
f (β, k ) = CK a+ 2 exp k {(β − β0 )T Σ−1 (β − β0 ) + 2b}
0
2
ba
where C = p 1
(2π) 2 |Σ0 | 2 Γ(a)
37. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Exch within Multiple Regression(unknown Covariance)
A brief explanation of a recent paper...
The posterior distribution is:
∗ + p −1 −1
f (β, k | Y ) ∝ k a 2 exp{ k ((β − β ∗ )T (Σ∗ )−1 (β − β ∗ ) + 2b∗ )}
2
β ∗ = (Σ0 −1 + X T X )−1 ((Σ0 −1 β0 + X T y )
Σ∗ = (Σ0 −1 + X T X )−1
n
a∗ = a +
2
1
b∗ = b + (β0 T Σ0 −1 β0 + y T y − (β ∗ )T (Σ∗ )−1 β ∗ )
2
And β | y follows a multivariate t-distribution:
−1
(ν+p)
1 2
f (β | y ) ∝ (1 + (β − β ∗ )T (Σ∗ )−1 (β − β ∗ ))
ν
38. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
References
L. D. Brown, On the Admissibility of Invariant Estimators of One
or More Location Parameters, The Annals of Mathematical
Statistics, Vol. 37, No. 5 (Oct., 1966), pp. 1087-1136.
A. E. Hoerl, R. W. kennard, Ridge Regression: Biased
Estimation for Nonorthogonal Problems, Technometrics, Vol.
12, No. 1. (Feb., 1970), pp. 55-67.
T. Bouche, Formation LaTex, (2007).
S. Kunz, The Bayesian Linear Model with unknown Variance,
Seminar for Statistics. ETH Zurich, (2009).
V. Roy, J. P. Hobert, On Monte Carlo methods for Bayesian
multivariate regression models with heavy-tailed errors, (2009).
39. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
References
Thank you
for your Attention
40. Introduction Exchangeability General bayesian linear model Examples Estimation with unknown Covariance References
. ..... .... ...... ...............
Result
.
Proof. .
..
To prove (a), suppose y = A1 Θ1 + u, Θ1 = A2 Θ2 + v that:
u ∼ N(0, C1 )
and v ∼ N(0, C2 ) Then y = A1 A2 Θ2 + A1 v + u that:
A1 v + u ∼ N(0, C1 + A1 C2 AT )
1
To prove (b), by using the Bayesian Theorem:
1
p(Θ1 | Y ) ∝ e− 2 Q
Q = (Θ1 − Bb)T B −1 (Θ1 − Bb)
−1 −1
+ y T C1 y + Θ2 T A2 T C2 A2 Θ2 − bT Bb (31)
.
.