One method for determining correct denominators in analysis of variance is the Cornfield-Tukey method. This FAQ presents a modified version of the Cornfield-Tukey method for manually deriving the symbolic values for the expected mean squares. It is from these expected mean squares that one can determine appropriate error terms.

Please note that this approach to deriving expected mean squares assumes that the interaction of the fixed and random effects sum to zero over the fixed effect levels. This approach can be found in a number of classical ANOVA textbooks, such as, Kirk, Winer and Keppel. This assumption, however, is not universal and is not used in most mixed programs (proc mixed, xtmixed, etc).

If you would like to try a program that automates much of the computation for this algorithm, go to How can I determine the correct term in an anova using Stata?.

## Steps in deriving expected mean squares

Step 1 - Write the linear model for the design. Step 2 - Construct a table with three parts. Step 3 - The row headings in part 1 contain each of the terms from the linear model including their subscripts but leaving out μ. Step 4 - The column heading in part 2 contain the subscripts from the linear model, the symbol for the number of levels along with the sampling coefficient. Sampling coefficients are coded 1 for random variables and 0 for fixed. Step 5 - If a column heading appears as a row subscript in parentheses enter a 1 in part 2. Step 6 - If a column heading appears as a row subscript, not in parentheses, enter the appropriate sampling coefficient (0 or 1). Step 7 - If a column heading does not appear as a row subscript enter the letter for the number of levels Step 8 - In part 3 list a variance for each term in the linear model that contains all the row subscripts. Step 9 - Coefficients for variances are obtained by covering the column headed by subscripts that appear in the row but not including subscripts in parentheses. Obviously, terms with zero coefficients drop out.

##
**Example three-way factorial design**

**Example three-way factorial design**

In this example, A & C are fixed and B is random. The subscript for ε is
i(jkl) because the subjects are nested in the A*B*C cells. The subjects themselves
are also random. The term, ε_{i(jkl)}, is known as error, within cell
or residual.

Step 1 - Y_{ijkl}= μ + α_{j}+ β_{k}+ γ_{l}+ αβ_{jk}+ αγ_{jl}+ βγ_{kl}+ αβγ_{jkl}+ ε_{i(jkl)}Part 1 Part 2 Part 3 subscript i j k l levels n p q m sampling coef 1 0 1 0 ------------------------------------------------------------------- α_{j}n 0 q m σ^{2}_{ε}+ 0σ^{2}_{αβγ}+ 0σ^{2}_{αγ}+ nmσ^{2}_{αβ}+ nqmσ^{2}_{α}σ^{2}_{ε}+ nmσ^{2}_{αβ}+ nqmσ^{2}_{α}β_{k}n p 1 m σ^{2}_{ε}+ 0σ^{2}_{αβγ}+ 0σ^{2}_{βγ}+ 0σ^{2}_{αβ}+ npmσ^{2}_{β}σ^{2}_{ε}+ npmσ^{2}_{β}γ_{l}n p q 0 σ^{2}_{ε}+ 0σ^{2}_{αβγ}+ npσ^{2}_{βγ}+ 0σ^{2}_{αγ}+ npqσ^{2}_{γ}σ^{2}_{ε}+ npσ^{2}_{βγ}+ npqσ^{2}_{γ}αβ_{jk}n 0 1 m σ^{2}_{ε}+ 0σ^{2}_{αβγ}+ nmσ^{2}_{αβ}σ^{2}_{ε}+ nmσ^{2}_{αβ}αγ_{jl}n 0 q 0 σ^{2}_{ε}+ nσ^{2}_{αβγ}+ nqσ^{2}_{αγ}βγ_{kl}n p 1 0 σ^{2}_{ε}σ^{2}_{ε}+ 0σ^{2}_{αβγ}+ npσ^{2}_{βγ}σ^{2}_{ε}+ npσ^{2}_{βγ}αβγ_{jkl}n 0 1 0 σ^{2}_{ε}+ nσ^{2}_{αβγ}ε_{i(jkl)}1 1 1 1 σ^{2}_{ε}

A correctly formed F-ratio will have one more term in the numerator than in the denominator. The additional
term in the numerator is the effect of interest. Thus, the F-ration for A main effect would
look something like this:

σ^{2}_{ε}+ nmσ^{2}_{αβ}+ nqmσ^{2}_{α}MS(A) F(A) = ------------------------ = --------- σ^{2}_{ε}+ nmσ^{2}_{αβ}MS(A*B)

Here are the terms that go into each of the F-ratios for the above model:

Effect Error Term numerator denominator MS(A) MS(A*B) MS(B) MS(residual) MS(C) MS(B*C) MS(A*B) MS(residual) MS(A*C) MS(A*B*C) MS(B*C) MS(residual) MS(A*B*C) MS(residual)

**Reference**

Kirk, Roger E. (1998) *Experimental Design: Procedures for the Behavioral Sciences,
Third Edition.* Monterey, California: Brooks/Cole Publishing. ISBN 0-534-25092-0