Expand/Contract All

Decomposition by Factor

Page Contents

Orthogonal Set of Factor

Let @A be a set of factors, on an observation set Y, that meet the following criteria:

  1. I &in. @A
  2. F, G &in. @A &imply. F &perp. G
  3. F, G &in. @A &imply. F &min. G &in. @A

Then @A is called an #~{orthogonal set} of factors

As we shall see, orthogonal sets of factor have some properties that make the statistical analysis possible. We'll start by looking at how such a set lends itself to a decomposition of &reals.^~n which is governed by the factors in the set.

Decomposition of the Identity Map

We will now demonstrate how we can associate certain quantities with the individual factors. For the time being this is rather abstract, but it will soon become clear how these quantities can be used in analysing the data.

Let @A be an orthogonal set of factors as defined above, and let ~p_F be the orthogonal projection onto the linear space associated with the factor F, with the associated matrix P_F. Obviously

prod{(P_F + (I &minus. P_F)),F &in. @A, _ } _ = _ I, _ _ _ _ _ _ where I is the identity map

But by expanding the product:

prod{(P_F + (I &minus. P_F)),F &in. @A, _ } _ = _ sum{ prod{P_F,F&in.M, _ }prod{(I &minus. P_F),F&nin.M, _ } ,M &subseteq. @A, _ } _ _ _ _ _ [i.e. sum taken over all subsets of @A.]

_ _ _ _ _ _ _ _ _ _ _ = _ sum{Q_M , M &subseteq. @A, _ }, _ _ _ _ _ _ where _ _ _ _ _ _ Q_M _ = _ prod{P_F,F&in.M, _ }prod{(I &minus. P_F),F&nin.M, _ }

Now if _ G >= F _ then _ P_FP_G = P_GP_F = P_F , _ and if _ F &in. M _ but _ G &nin. M , _ then _ P_F (I &minus. P_G) _ is a factor of Q_M , _ and

P_F (I &minus. P_G) _ = _ P_F - P_FP_G _ = _ 0 _ _ => _ _ Q_M _ = _ 0.

Also, if _ F &in. M , _ G &in. M , _ but _ F&min.G &nin. M , _ then _ P_FP_G (I &minus. P_{F&min.G}) _ is a factor of Q_M , _ and

P_FP_G (I &minus. P_{F&min.G}) _ = _ P_FP_G (I &minus. P_FP_G) _ = 0 _ _ => _ _ Q_M _ = _ 0.

In the sum, &sum. Q_M we need only consider non-zero Q_M, i.e. we need only sum over sets M, where

  • F &in. M, _ G >= F _ _ => _ _ G &in. M
  • F, G &in. M _ _ => _ _ F&min.G &in. M.

This means that the minimum element of M specifies Q_M. _ So for any factor G, write _ Q_G = Q_M , _ where _ M = \{ F | F &ge. G \} _ i.e:

Q_G _ = _ prod{P_F,F&ge.G, _ }prod{(I &minus. P_F),F!&ge.G, _ }

where !&ge. represents 'not greater than or equal to'. [Note this is not synonymous with < as this is only a partial ordering.]

Now _ P_FP_G = P_G _ for any _ F &ge. G . _ So

prod{P_F,F&ge.G, _ } _ = _ P_G

So we have:

Q_G _ = _ P_G prod{(I &minus. P_F),F!&ge.G, _ }
I _ = _ sum{ Q_G ,G &in. @A, _ }

Decomposition of R^~n

Now Q_G is the product of orthogonal projections [or strictly speaking, their associated matrices], and so Q_G is an orthogonal projection for any factor G &in. &Delta.. _ Let _ V_G = im Q_G , _ then V_G is a subspace of &reals.^{~n}, _ and _ V_G &perp. V_F _ any two factors G ≠ F &in. &Delta.. _ So

&reals.^{~n} _ = _ oplus{ V_G ,G &in. &Delta., }

V_G _ = _ im Q_G

Note that if F !&ge. G then P_F Q_G = 0, _ so

P_F _ = _ P_F &mult. I _ = _ P_F &mult. sum{ Q_G ,G &in. &Delta., _ } _ = _ sum{P_FQ_G ,G &le. F, _ } _ = _ sum{Q_G ,G &le. F, _ }

Since P_F Q_G = Q_G when G &le. F. We have:

P_F _ = _ sum{Q_G ,G &le. F, _ }
L_F _ = _ oplus{ V_G ,G &le. F, _ }

Uniqueness of the Decomposition

The above decomposition is unique, for suppose

I _ = _ sum{ R_G ,G &in. &Delta., _ }

&reals.^{~n} _ = _ oplus{ W_G ,G &in. &Delta., _ }, _ _ _ _ _ _ W_G _ = _ Im R_G

Let &Theta. &subset. &Delta. be the subset of all factors, G, in the design for which R_G ≠ Q_G. _ Let F &in. &Theta. where F is not the minimal element of &Delta..

Now

P_F _ = _ sum{ Q_G,G &le. F, _ } _ _ _ _ _ _ => _ _ _ _ _ _ sum{ Q_G,G < F, _ } _ = _ P_F - Q_F _ _ != _ _ P_F - R_F _ = _ sum{ R_G,G < F, _ }

So &exist. some G < F such that Q_G ≠ R_G, i.e. G &in. &Theta.. So either &Theta. contains the minimal element of &Delta. or &Theta. is empty. Let M be the minimal element then Q_M = P_M = R_M , so M &nin. &Theta. &imply. &Theta. is empty.

Alternative Expression for Q_G

Q_G _ = _ P_G prod{(I &minus. P_F),F!&ge.G, _ } _ = _ P_G prod{(P_G &minus. P_{F&min.G}),F!&ge.G, _ }

Now if _ F < G _ then _ P_{F&min.G} = P_F.

If _ F !< G _ then _ F&min.G = H , _ where H < G, _ so _ P_G &minus. P_H _ will be a factor of the above product.

Note also that if _ F < G , _ then _ ( P_G - P_F ) ( P_G - P_F ) _ = _ P_G - P_F. _ So:

Q_G _ = _ P_G prod{( P_G - P_F ),F<.G, _ }

This can sometimes make calculations simpler, e.g.

Q_A _ = _ P_A ( P_A - P_{A&min.B} ) _ = _ P_A - P_{A&min.B}

Design

A ~#{design} is a set of factors on the same finite observation set Y.

In terms of the statistical analysis, the design is the set of factors that we decide that we are going to consider before the experiment starts, or before the observations are made. Subsequently we will look at hypotheses that involve a subset of the design. So the design is the maximal set of factors that will be considered in an experiment or survey.

Orthogonal Design

A design, &Delta., on Y is an ~#{orthogonal design} if it is an orhtogonal set as defined above, i.e.

  1. I &in. &Delta.
  2. F, G &in. &Delta. &imply. F &perp. G
  3. F, G &in. &Delta. &imply. F &min. G &in. &Delta.

Example: Two Sided ANOVA

Consider a model with two factors, A and B, which is orthogonal, e.g. there are an equal number of observations in each cross-cell, i.e. in each level of A # B.

Let &Delta. _ = _ \{ I, A # B, A, B, A&min.B \}

Q_{A&min.B} _ = _ P_{A&min.B}

Q_A _ = _ P_A ( I - P_B ) ( I - P_{A&min.B} )

_ _ _ = _ P_A _ - _ P_A P_B _ - _ P_A P_{A&min.B} _ + _ P_A P_B P_{A&min.B} _ _ = _ _ P_A _ - _ P_{A&min.B}

[ remembering that _ P_AP_B _ = _ P_{A&min.B} _ for orthogonal factors. ]

Q_B _ = _ P_B ( I - P_A ) ( I - P_{A&min.B} ) _ _ = _ _ P_B _ - _ P_{A&min.B}

Q_{A # B} _ = _ P_{A # B} ( I - P_A ) ( I - P_B ) ( I - P_{A&min.B} )

_ _ _ = _ P_{A # B} _ - _ P_{A # B} P_A _ - _ P_{A # B} P_B _ - _ P_{A # B} P_{A&min.B}

_ _ _ _ _ + _ P_{A # B} P_A P_B _ + _ P_{A # B} P_B P_{A&min.B} _ + _ P_{A # B} P_B P_{A&min.B} _ - _ P_{A # B} P_A P_B P_{A&min.B}

_ _ _ = _ P_{A # B} _ - _ P_A _ - _ P_B _ - _ P_{A&min.B} _ + _ P_{A&min.B} _ + _ P_{A&min.B} _ + _ P_{A&min.B} _ - _ P_{A&min.B}

_ _ _ = _ P_{A # B} _ - _ P_A _ - _ P_B _ + _ P_{A&min.B}


In fact, for any factor, G, in an orthogonal design, Q_G can be expressed in the form

Q_G _ = _ sum{&alpha._G^F P_F, F &le. G, _ }, _ _ _ _ _ _ where &alpha._G^F &in. \{ -1 , +1 \}