1. Theorem 1: Invariance property
of maximum-likelihood
estimators
Benjie Lloyd V. Tabios
September 11, 2014
2. 1
Invariance property of maximum-likelihood
estimators
Let ^
= ^#(X1;X2; : : : ;Xn) be the maximum-
likelihood estimator of in the density f(x; ),
where is assumed unidimensional. If (:) is a
function with a single-valued inverse, then the
maximum-likelihood estimator of () is ( ^ ).
1
3. 2
For example, in the normal density with 110
known the maximum-likelihood estimator of 2
is
(1) 1=n
(Xi o)2
Xn
i=1
By the invariance property of maximum-likelihood
estimators, the maximum- likelihood estimator
of is
vuut
(2) 1=n
(Xi o)2
Xn
i=1
Similarly, the maximum-likelihood estimator of,
say, log 2 is
log
h
1=n
Xn
i=1
(Xi o)2
i
(3)
2
4. 3
The invariance property of maximum-likelihood
estimators that is exhibited in Theorem 1 above
can and should be extended. Following Zehna
[43], we extend in two directions: First.
will be taken as k-dimensional rather than uni-
dimensional, and, second, the assumption that
(:) has a single-valued inverse will be removed.
3
5. 4
It can be noted that such extension is neces-
sary by considering two simple examples. As a
6. rst example, suppose an estimate of the vari-
ance, namely (1 ), of a Bernoulli distribu-
tion is desired. Example 5 gives the maximum-
likelihood estimate of to be x, but since
(1) is not a one-to;'one function of , The-
orem 1 does not give the maximum-likelihood
estimator of (1 ).
4
7. 5
Theorem 2 will give such an estimate, and
it will be x(1 x). As a second example,
consider sampling from a normal distribution
where both and 2 are unknown, and sup-
pose an estimate of [X2] = 2+2 is desired.
Example 6 gives the maximum-likelihood esti-
mates of and 2, but 2+2 is not a one-to-
one function of and 2, and so the maximum-
likelihood estimate of 2 + 2 is not known.
Such an estimate will be obtainable from The-
orem 2. It will be x2 +(1=n)
P
(xi x)2.
5
8. 6
Let = (1; : : : ; k) be a k-dimensional parame-
ter, and, as before, let
denote the parameter
space. Suppose that the maximum-likelihood
estimate of () = (1(); : : : ; r()), where 1
r k, is desired. Let T denote the range space
of the transformation () = (1(); : : : ; r()).
T is an r-dimensional space. De
9. ne
M(; x1; : : : ; xn) = supf:()=g L(; x1; : : : ; xn).
M(; x1; : : : ; xn)is called the likelihood function
induced by ().*
6
10. 7
When estimating we maximized the likeli-
hood function L(; x1; : : : ; xn) as a function of
for
11. xed x1; : : : ; xn; when estimating = ()
we will maximize the likelihood function in-
duced by (), namely M(; x1; : : : ; xn) as a
function of for
12. xed x1; : : : ; xn. Thus, the
maximum-likelihood estimate of = (), de-
noted by ^, is any value that maximizes the
induced likelihood function for
13. xed x1; : : : ; xn;
that is, ^ is such that
M(^; x1; : : : ; xn) M(; x1; : : : ; xn)
for all 2 T. The invariance property of maximum-
likelihood estimation is given in the following
theorem.
7
14. 8
in addition......
The maximum likelihood estimate is invariant
under functional transformations. That is, if
T = t(X1; : : : ;Xn) is the mle of and if u() is
a function of , then u(T) is the mle of u().
For example, if ^ is the mle of , then ^2 is
the mle of 2. That is, c2 = ^2.
8
