Normal Distribution

This is a short discussion of testing normally distributed variables. This will give an introduction to the use of maximum likelihood methods for a continuous distribution and provide results which will be use to provide approximate tests for binomial models. The normal distribution will be covered in more detail in Linear Normal Models .

Maximum Likelihood

~X is a normally distributed random variable, with mean &mu. and variance &sigma.&powtwo., _ ~X ~ N(&mu., &sigma.&powtwo.) . _ Suppose &sigma.&powtwo. is known, we want to estimate &mu.. For an observed value ~x, define the likelihood function:

L ( &mu. ) _ #:= _ L ( &mu. | ~x ) _ #:= _ f ( ~x | &mu. ) _ = _ fract{1, &sqrt.${2&pi.&sigma.&powtwo.}} exp rndb{fract{- (~x - &mu.)^2, 2&sigma.&powtwo.}}

Where f ( ~x ) is the density function of the normal distribution. L ( &mu. ) reaches its maximum when _ &mu. = ~x, _ so _

est{&mu.} _ = _ ~x

Likelihood Ratio

Consider the hypothesis _ H: &mu. = &mu._0 . _ The Likelihood Ratio Test is

LR (~x) _ = _ fract{L(&mu._0),L(est{&mu.})} _ = _ fract{exp \{ - (~x - &mu._0)^2 / 2&sigma.&powtwo.\},exp \{ - (~x - ~x)^2 / 2&sigma.&powtwo.\}}

_ _ _ _ _ _ _ _ _ = _ exp \{ - (~x - &mu._0)^2 / 2&sigma.&powtwo.\}

Significance Probability

Recall that for a given hypothesis H: &mu. = &mu._0, the significance probability for the result ~x is:

SP( ~x ) _ = _ P\{ ~y | LR (~y) =< LR (~x) \}

Now _ _ _ LR (~y) =< LR (~x) _ _ <=> _ _ (~y - &mu._0)^2 >= (~x - &mu._0)^2

case 1) _ ~x > &mu._0, _ then

LR (~y) =< LR (~x) _ _ <=> _ _ ~y >= ~x _ _ or _ _ ~y =< 2&mu._0 - ~x

SP _ = _ P(LR (~y) =< LR (~x)) _ = _ P(~y >= ~x) _ _ + _ _ P(~y =< 2&mu._0 - ~x)

_ _ _ = _ rndb{1 - &Phi.rndb{fract{~x - &mu._0, &sigma.}}} _ + _ &Phi.rndb{fract{&mu._0 - ~x, &sigma.}} _ _ = _ _ 2&Phi.rndb{fract{&mu._0 - ~x, &sigma.}}

case 2) _ ~x < &mu._0, _ then

LR (~y) =< LR (~x) _ _ <=> _ _ ~y - &mu._0 _ >= _ &mu._0 - ~x _ _ or _ _ &mu._0 - ~y _ >= _ ~x - &mu._0

_ _ _ _ _ _ _ _ _ _ <=> _ _ ~y >= 2&mu._0 - ~x _ _ or _ _ ~y =< ~x

SP _ = _ P(LR (~y) =< LR (~x)) _ = _ P(~y =< ~x) _ _ + _ _ P(~y >= 2&mu._0 - ~x)

_ _ _ = _ &Phi.rndb{fract{~x - &mu._0, &sigma.}} _ + _ rndb{1 - &Phi.rndb{fract{&mu._0 - ~x, &sigma.}}} _ _ = _ _ 2&Phi.rndb{fract{~x - &mu._0, &sigma.}}

In general

SP( ~x ) _ = _ 2&Phi.rndb{fract{ - | ~x - &mu._0 | , &sigma.}}

Confidence Interval

For which values of &mu._0 will we accept the hypothesis at level &alpha.? I.e. SP(~x) >= &alpha., given &mu._0?

Let &epsilon._{&alpha.} be the number such that &Phi.(&epsilon._{&alpha.}) = &alpha.. _ &epsilon._{&alpha.} is the &alpha. # 100 percentile of the N(0,1) distribution.

case 1) _ ~x > &mu._0, _ and _ 2&Phi.((&mu._0 - ~x) / &sigma.) >= &alpha.

&Phi.rndb{fract{&mu._0 - ~x, &sigma.}} _ _ >= _ _ &alpha./2

fract{&mu._0 - ~x, &sigma.} _ _ >= _ _ &epsilon._{&alpha./2}

&mu._0 _ _ >= _ _ ~x + &sigma.&epsilon._{&alpha./2}

case 2) _ ~x < &mu._0, _ and _ 2&Phi.((~x - &mu._0) / &sigma.) >= &alpha.

&Phi.rndb{fract{~x - &mu._0, &sigma.}} _ _ >= _ _ &alpha./2

fract{~x - &mu._0, &sigma.} _ _ >= _ _ &epsilon._{&alpha./2}

&mu._0 _ _ =< _ _ ~x - &sigma.&epsilon._{&alpha./2}

Combining the two results, we can define the confidence interval as:

~x + &sigma.&epsilon._{&alpha./2} _ _ =< _ _ &mu._0 _ _ =< _ _ ~x - &sigma.&epsilon._{&alpha./2}

[Remember that &epsilon._{&alpha./2} =< 0 _ if _ &alpha./2 =< 0.5 _ - _ which it is as &alpha. < 1 always.]

 

---