6 1. The Fisher Efficiency Example 1.5. Let X be a Binomial(n , θ 2 ) observation, that is, a random number of successes in n independent Bernoulli trials with the probability of a success p = θ 2 , 0 θ 1. An unbiased estimator of the parameter θ does not exist. In fact, if ˆ = ˆ(X) were such an estimator, then its expectation would be an even polynomial of θ, ˆ(X) = n k = 0 ˆ(k) n k θ 2k (1 θ 2 )n−k, which cannot be identically equal to θ. 1.2. The Fisher Information Introduce the Fisher score function as the derivative of the log-likelihood function with respect to θ, l (Xi , θ) = ln p(Xi , θ) ∂θ = ∂p(Xi , θ)/∂θ p(Xi , θ) . Note that the expected value of the Fisher score function is zero. Indeed, l (Xi , θ) = R ∂p(x , θ) ∂θ dx = R p(x , θ) dx ∂θ = 0. The total Fisher score function for a sample X1 , . . . , Xn is defined as the sum of the score functions for each individual observation, Ln(θ) = n i = 1 l (Xi , θ). The Fisher information of one observation Xi is the variance of the Fisher score function l (Xi , θ), I(θ) = Varθ l (Xi , θ) = ( l (Xi , θ) ) 2 = ln p (X, θ) θ 2 = R ln p(x , θ) ∂θ 2 p(x , θ) dx = R ( ∂p(x , θ)/∂θ ) 2 p(x , θ) dx. Remark 1.6. In the above definition of the Fisher information, the density appears in the denominator. Thus, it is problematic to calculate the Fisher information for distributions with densities that may be equal to zero for some values of x even more so, if the density turns into zero as a function of x on sets that vary depending on the value of θ. A more general approach to the concept of information that overcomes this difficulty will be suggested in Section 4.2.
Previous Page Next Page