Now that Stata 11 has been released, you be wondering if there is an easy way to compute tests of simple main effects? Yes there is and this FAQ will show you how to get tests of simple main effects using the new margins command.
Consider a two-factor design using a dataset called crf24.
use https://stats.idre.ucla.edu/stat/stata/faq/crf24, clear anova y a##b Number of obs = 32 R-squared = 0.9214 Root MSE = .877971 Adj R-squared = 0.8985 Source | Partial SS df MS F Prob > F -----------+---------------------------------------------------- Model | 217 7 31 40.22 0.0000 | a | 3.125 1 3.125 4.05 0.0554 b | 194.5 3 64.8333333 84.11 0.0000 a#b | 19.375 3 6.45833333 8.38 0.0006 | Residual | 18.5 24 .770833333 -----------+---------------------------------------------------- Total | 235.5 31 7.59677419
As you can see the a#b interaction is statistically significant.
Next we will use the margins command on the a#b interaction. We will include the post option and if the design were unbalanced or included a covariate we would also include the asbalanced option.
margins a#b, post Adjusted predictions Number of obs = 32 Expression : Linear prediction, predict() ------------------------------------------------------------------------------ | Delta-method | Margin Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- a#b | 1 1 | 3.75 .4389856 8.54 0.000 2.889604 4.610396 1 2 | 4 .4389856 9.11 0.000 3.139604 4.860396 1 3 | 7 .4389856 15.95 0.000 6.139604 7.860396 1 4 | 8 .4389856 18.22 0.000 7.139604 8.860396 2 1 | 1.75 .4389856 3.99 0.000 .8896041 2.610396 2 2 | 3 .4389856 6.83 0.000 2.139604 3.860396 2 3 | 5.5 .4389856 12.53 0.000 4.639604 6.360396 2 4 | 10 .4389856 22.78 0.000 9.139604 10.8604 ------------------------------------------------------------------------------
The margins command has given us a list of the cell means and their standard errors. We can use combinations of these means to compute the test of simple maun effects. For example, a1 versus a2 at b1 would test cell a1,b1 versus cell a2,b1. Here is what the tests of simple main effects for a each level of b would look like.
test 1.a#1.b == 2.a#1.b ( 1) 1bn.a#1bn.b - 2.a#1bn.b = 0 chi2( 1) = 10.38 Prob > chi2 = 0.0013 test 1.a#2.b == 2.a#2.b ( 1) 1bn.a#2.b - 2.a#2.b = 0 chi2( 1) = 2.59 Prob > chi2 = 0.1072 test 1.a#3.b == 2.a#3.b ( 1) 1bn.a#3.b - 2.a#3.b = 0 chi2( 1) = 5.84 Prob > chi2 = 0.0157 test 1.a#4.b == 2.a#4.b ( 1) 1bn.a#4.b - 2.a#4.b = 0 chi2( 1) = 10.38 Prob > chi2 = 0.0013
Each one of the above tests used only one degree of freedom so the terms in the test were simple. We will next move on to test of b at each level of a. Each of these tests have three degrees of freedom and so, will will involve a more complex test command. We will start off showing the accum approach and then show a cleaner way to get the result with a single test command.
/* test of b at a==1 */ test 1.a#1.b == 1.a#2.b test 1.a#1.b == 1.a#3.b, accum test 1.a#1.b == 1.a#4.b, accum ( 1) 1bn.a#1bn.b - 1bn.a#2.b = 0 ( 2) 1bn.a#1bn.b - 1bn.a#3.b = 0 ( 3) 1bn.a#1bn.b - 1bn.a#4.b = 0 chi2( 3) = 70.95 Prob > chi2 = 0.0000 /* the same test using a single test command */ test (1.a#1.b == 1.a#2.b)(1.a#1.b == 1.a#3.b)(1.a#1.b == 1.a#4.b) ( 1) 1bn.a#1bn.b - 1bn.a#2.b = 0 ( 2) 1bn.a#1bn.b - 1bn.a#3.b = 0 ( 3) 1bn.a#1bn.b - 1bn.a#4.b = 0 chi2( 3) = 70.95 Prob > chi2 = 0.0000 /* convert chi2 to approximate F-value */ display r(chi2)/r(df) 23.648649
We will finish up with tests of b at a==2 just by changing all of the 1.a terms to 2.a
/* test of b at a==1 */ test (2.a#1.b == 2.a#2.b)(2.a#1.b == 2.a#3.b)(2.a#1.b == 2.a#4.b) ( 1) 2.a#1bn.b - 2.a#2.b = 0 ( 2) 2.a#1bn.b - 2.a#3.b = 0 ( 3) 2.a#1bn.b - 2.a#4.b = 0 chi2( 3) = 206.51 Prob > chi2 = 0.0000 /* convert chi2 to approximate F-value */ display r(chi2)/r(df) 68.837838
This FAQ only covers the computation of the tests of simple main effects using the margins command. The FAQ does not cover computing the critical values of these tests. There is a user written ado-program smecriticalvalue which can assist in this process (search smecriticalvalue).
Let’s try a little more challenging example, one that is unbalanced with a covariate.
use https://stats.idre.ucla.edu/stat/data/hsbdemo, clear anova write female##prog c.read Number of obs = 200 R-squared = 0.4889 Root MSE = 6.88105 Adj R-squared = 0.4730 Source | Partial SS df MS F Prob > F ------------+---------------------------------------------------- Model | 8740.54801 6 1456.758 30.77 0.0000 | female | 1680.08743 1 1680.08743 35.48 0.0000 prog | 632.133455 2 316.066727 6.68 0.0016 female#prog | 301.770797 2 150.885399 3.19 0.0435 read | 4110.18709 1 4110.18709 86.81 0.0000 | Residual | 9138.32699 193 47.3488445 ------------+---------------------------------------------------- Total | 17878.875 199 89.843593
The female#prog interaction is significant but we’ll go ahead and compute the tests of simple main effects. This time because the model is both unbalanced and includes a covariate we will include the asbalanced option which is equivalent to least squares means (lsmeans) or expected marginal means (emmeans) in SAS and SPSS.
margins female#prog, asbalanced post Predictive margins Number of obs = 200 Expression : Linear prediction, predict() ------------------------------------------------------------------------------ | Delta-method | Margin Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- female#prog | 0 1 | 48.7854 1.502058 32.48 0.000 45.84142 51.72938 0 2 | 52.61465 1.026455 51.26 0.000 50.60284 54.62647 0 3 | 45.08098 1.476716 30.53 0.000 42.18667 47.97529 1 1 | 55.85857 1.432221 39.00 0.000 53.05146 58.66567 1 2 | 55.68658 .926245 60.12 0.000 53.87117 57.50199 1 3 | 53.71585 1.356821 39.59 0.000 51.05653 56.37517 ------------------------------------------------------------------------------ /* tests of female at each level of prog */ test 0.female#1.prog == 1.female#1.prog ( 1) 0bn.female#1bn.prog - 1.female#1bn.prog = 0 chi2( 1) = 11.56 Prob > chi2 = 0.0007 test 0.female#2.prog == 1.female#2.prog ( 1) 0bn.female#2.prog - 1.female#2.prog = 0 chi2( 1) = 5.17 Prob > chi2 = 0.0229 test 0.female#3.prog == 1.female#3.prog ( 1) 0bn.female#3.prog - 1.female#3.prog = 0 chi2( 1) = 19.54 Prob > chi2 = 0.0000 /* tests of prog at each level of female */ test (0.female#1.prog == 0.female#2.prog)(0.female#1.prog == 0.female#3.prog) ( 1) 0bn.female#1bn.prog - 0bn.female#2.prog = 0 ( 2) 0bn.female#1bn.prog - 0bn.female#3.prog = 0 chi2( 2) = 17.41 Prob > chi2 = 0.0002 /* convert chi2 to approximate F-value */ display r(chi2)/r(df) 8.7026677 test (1.female#1.prog == 1.female#2.prog)(1.female#1.prog == 1.female#3.prog) ( 1) 1.female#1bn.prog - 1.female#2.prog = 0 ( 2) 1.female#1bn.prog - 1.female#3.prog = 0 chi2( 2) = 1.69 Prob > chi2 = 0.4287 /* convert chi2 to approximate F-value */ display r(chi2)/r(df) .84693839
So that’s how to compute tests of simple main effects using margins in Stata 11.