This FAQ presents an ado program (smecriticalvalue) that can be used to compute critical values for tests of simple interactions and simple main effects. You can find the smecriticalvalue program by typing search smecriticalvalue in the Stata command window (see How can I use the search command to search for programs and get additional help? for more information about using search).
There are at least four different approaches in the literature for computing critical values for test of simple interactions and simple main effects. There is a method attributable to Dunn and one by Marascuilo and Levin. In addition there is the per family error rate and a very conservative method called the simultaneous test procedure that is closely related to the Scheff´e method for multiple comparisons. There does not seem to be a clear favorite among these approaches in the literature.
Let’s say that you have run a 3×4 factorial model with a significant interaction. The original 3×4 model has a total of 11 degrees of freedom for for the anova and 48 for the error term. You follow up the anova by doing four tests of simple main effects. Each of these tests has 2 degrees of freedom in the numerator and 48 in the denominator. Here is how you can compute the critical values using smecriticalvalue.
smecriticalvalue, number(4) df1(2) df2(48) dfmodel(11)
number of tests: 4
numerator df: 2
denominator df: 48
original model df: 11
Critical value of F for alpha = .05 using ...
------------------------------------------------
Dunn's procedure = 4.7033201
Marascuilo & Levin = 5.1923643
per family error rate = 4.8075721
simultaneous test procedure = 8.4455227
If you are using the per family error rate then the critical value would be about 4.81.
If you had done the tests of simple main effects the other way around, that is, three tests with 3 degrees of freedom in the numerator and 48 in the denominator, the critical values would look like this:
smecriticalvalue, number(3) df1(3) df2(48) dfmodel(11)
number of tests: 3
numerator df: 3
denominator df: 48
original model df: 11
Critical value of F for alpha = .05 using ...
------------------------------------------------
Dunn's procedure = 3.9394001
Marascuilo & Levin = 4.3040955
per family error rate = 3.7605452
simultaneous test procedure = 5.6303484
This time the critical value for the per family error rate is 3.76.
Reference
Kirk, R.E. Experimental design: Procedures for the behavioral sciences, 3rd ed. (1995) Pacific Grove, CA: Brooks/Cole