Linear Normal Model Examples

Page Contents

Uniform Mean

Consider the case where we have a number of independent observations, all of which are assumed to come from the same normal distribution.

I.e. #{~Y} &tilde. N( #{&mu.}, &sigma.^2#I ), _ &mu._1 = &mu._2 = ... = &mu._{~n} _ _ [ that is: ~{Y_i} &tilde. N( &mu., &sigma.^2 ), _ all ~i. ]

So #{&mu.} &in. L, dim L = 1, and L is spanned by #~e, where #~e = ( 1, ... , 1 )

( #{~y} - p( #{~y} ) ) &dot. #~e _ _ = _ _ #0

#{~y} &dot. #~e _ _ _ _ _ _ _ = _ _ p( #{~y} ) &dot. #~e

Now

_ #{~y} &dot. #~e _ = _ sum{~{y_i},1,~n} _ _ _ _ and _ _ _ _ p( #{~y} ) &dot. #~e _ = _ ( est{&mu.}, ... , est{&mu.} ) &dot. #~e _ = _ ~n est{&mu.}

est{&mu.} _ = _ fract{1,~n}rndb{sum{~{y_i},1,~n}} _ = _ ${~y}

p( #{~y} ) _ = _ ( ${~y}, ... , ${~y} )

RSS _ = _ || #{~y} - p( #{~y} ) || ^2 _ = _ sum{( ~{y_i} - ${~y} )^2,1,~n}

~s_1^2 _ = _ fract{1,~n - 1} sum{( ~{y_i} - ${~y} )^2,1,~n}

One-way ANOVA - Grouped observations

Suppose we have ~k groups of observations ( of the same variable ), where the ~{n_i} observations in each group [&sum._{~i = 1...~k} ~n_{~i} = ~n] are assumed to have the same distribution, i.e. all the observation in a group have the same mean, and the variance is the same for all the observations:

#{~Y} &tilde. N( #{&mu.}, &sigma.^2#I ), _ #{&mu.} = ( &mu._{1,1}, ... , &mu._{1,~n_1}, _ . . . _ &mu._{~k,1}, ... , &mu._{~k,~n_{~k}} ) _ and _ &mu._{~i,~j} = &theta._{~i}.

So #{&mu.} &in. L, dim( L ) = ~k, and L is spanned by #{~v}_1, ... , #{~v}_{~k}, _ where:

#{~v}_1 = ( 1, ... ,1, . . . 0, ... ,0, . . . 0, ... , 0 ) _ _ _ _ _ _ _ _ _ [first group - ~{n_1} 1's]

#{~v}_{~i} = ( 0, ... ,0, . . . 1, ... ,1, . . . 0, ... , 0 ) _ _ _ _ _ _ _ _ _ [~i^{th} group - ~{n_i} 1's]

#{~v}_{~k} = ( 0, ... ,0, . . . 0, ... ,0, . . . 1, ... ,1 ) _ _ _ _ _ _ _ _ _ [~k^{th} group - ~{n_k} 1's]

Now

( #{~y} - p( #{~y} ) ) &dot. #{~v}_{~i} _ = _ #0, _ _ _ _ ~i = 1, ... ~k

i.e.

_ #{~y} &dot. #{~v}_{~i} _ = _ p( #{~y} ) &dot. #{~v}_{~i}, _ _ _ _ &forall. ~i

but

p( #{~y} ) &dot. #{~v}_{~i} _ = _ ~n_{~i} est{&theta._~i}

_ #{~y} &dot. #{~v}_{~i} _ = _ sum{~{y_{i,j}}, ~j = 1,~n_{~i}}

so

est{&theta._~i} _ = _ fract{1,~n_{~i}}sum{~{y_{i,j}}, ~j = 1,~n_{~i}} _ = _ ${~y}_{~i} _ _ _ _ _ _ the group mean

p( #{~y} ) _ = _ ( ${~y}_1, ... ${~y}_1, _ . . . _ ${~y}_{~i}, ... ${~y}_{~i}, _ . . . _ ${~y}_{~k}, ... ${~y}_{~k} )

RSS _ = _ || #{~y} - p( #{~y} ) || ^2 _ = _ sum{,~i,}sum{( ~{y_{i, j}} - ${~y}_{~i} )^2,~j,}

~s_1^2 _ = _ fract{RSS,~n - ~k}