6 1. The Fisher Efficiency
Example 1.5. Let X be a Binomial(n , θ
2)
observation, that is, a random
number of successes in n independent Bernoulli trials with the probability of
a success p = θ
2
, 0 θ 1. An unbiased estimator of the parameter θ does
not exist. In fact, if
ˆ
θ =
ˆ(X)
θ were such an estimator, then its expectation
would be an even polynomial of θ,

ˆ(X)
θ =
n
k = 0
ˆ(k)
θ
n
k
θ
2k
(1 θ
2)n−k,
which cannot be identically equal to θ.
1.2. The Fisher Information
Introduce the Fisher score function as the derivative of the log-likelihood
function with respect to θ,
l (Xi , θ) =
ln p(Xi , θ)
∂θ
=
∂p(Xi , θ)/∂θ
p(Xi , θ)
.
Note that the expected value of the Fisher score function is zero. Indeed,
l (Xi , θ) =
R
∂p(x , θ)
∂θ
dx =

R
p(x , θ) dx
∂θ
= 0.
The total Fisher score function for a sample X1 , . . . , Xn is defined as the
sum of the score functions for each individual observation,
Ln(θ) =
n
i = 1
l (Xi , θ).
The Fisher information of one observation Xi is the variance of the Fisher
score function l (Xi , θ),
I(θ) = Varθ l (Xi , θ) =
(
l (Xi , θ)
)2
=
ln p (X, θ)
θ
2
=
R
ln p(x , θ)
∂θ
2
p(x , θ) dx
=
R
(
∂p(x , θ)/∂θ
)2
p(x , θ)
dx.
Remark 1.6. In the above definition of the Fisher information, the density
appears in the denominator. Thus, it is problematic to calculate the Fisher
information for distributions with densities that may be equal to zero for
some values of x; even more so, if the density turns into zero as a function of
x on sets that vary depending on the value of θ. A more general approach to
the concept of information that overcomes this difficulty will be suggested
in Section 4.2.
Previous Page Next Page