1. 1 De

2. nition 3 Maximum-likelihood estimator
Let
L() = L(; x1; ; xn) (1)
be the likelihood function for the random variables X1;X2; ;Xn. If b [where
b = b#(x1; x2; ; xn) is a function of the observations x1; ; xn] is the value of
in which maximizes L(), then b
= b#(X1;X2; ;Xn) is the maximum-likelihood
estimator of . b = b#(x1; ; xn) is the maximum-likelihood estimate of for the
sample x1; ; xn.
The most important cases which we shall consider are those in which X1;X2; ;Xn
is a random sample from some density f(x; ), so that the likelihood function is
L() = f(x1; )f(x2; ) f(xn; ): (2)
Many likelihood functions satisfy regularily conditions;so the maximum-likelihood
estimator is the solution of the equation
dL()
d
= 0: (3)
Also L() and log L()have their maxima at the same value of , and it is some-
times easier to

3. nd the maxima of the logarithm of the likelihood.
If the likelihood function contains k parameters, that is, if
L(1; 2; ; k);=
Yn
i=1
f(xi; 1; 2; k); (4)
then the maximum-likelihood estimators of the parameters 1; 2; ; k are the
1 = b#1(X1; ;Xn);b
random variables b
2 = b#2(X1; ;Xn); ;b
k = b#k(X1; ;Xn)
where b1; b2; ; bk are the values in which maximize L(1; 2; ; k).
If certain regularity conditions are satis

4. ed, the point where the likelihood is a
maximum is a solution of the k equations
@L(1; ; k)
@1
= 0 (5)
@L(1; ; k)
@2
= 0 (6)
...
(7)
@L(1; ; k)
@k
= 0 (8)
(9)
.
1

5. example:
Suppose that a randon sample of size n is drawn from the bernoulli distribution,
f(x; p) = pxq(1x)I(0;1)(x); 0 p 1 and q = 1 p (10)
(11)
and the likelihood function is
L(p) =
Yn
i=1
pxiqnxi (12)
= p
P
xiqn
P
xi (13)
let, y =
P
xi we obtain,
L(p) = pyqny (14)
logL(p) = y log p + (n y) log q (15)
and to

6. nd the location of its maximum, we compute
(remembering q = 1 p)
d log L(p)
dp
=
y
p
(n y)
q
(16)
=
y
p
(n y)
(1 p)
(17)
=
y(1 p) p(n y)
p(1 p)
(18)
=
y py pn + py
p(1 p)
(19)
=
y pn
p(1 p)
(20)
(21)
from our def'n 3, dL()
d = 0
by equating this to zero, we have
y pn
p(1 p)
= 0 (22)
y pn = 0 (23)
pn
y
=
n
n
(24)
^p =
y
n
(25)
=
P
xi
n
(26)
=
1
n
X
xi (27)
= x (28)
Note that the likelihood function depends on the x0
is only through
P
xi;
Now, to sketch the likelihood function for n=3, we have
L0 = L(p;
X
xi = 0) = p0(1 p)30 = (1 p)3 (29)
L1 = L(p;
X
xi = 1) = p1(1 p)31 = p(1 p)2 (30)
L2 = L(p;
X
xi = 2) = p2(1 p)32 = p2(1 p) (31)
L3 = L(p;
X
xi = 3) = p3(1 p)33 = p3 (32)
(33)
2

7. Thus, the point where the maximum of each of the curves takes place for 0 p 1
is the same as the given when n=3.
3