------------------- help for simanova -------------------ANOVA Simulation ----------------
Usage #1
In this usage, you supply the number of groups, the sample sizes of the groups, and the standard deviations of the groups. Data is simulated given these conditions where the null hypothesis is true, and the proportion of significant results are computed and displayed. Under certain conditions, the actual (simulated) p values can differ from the nominal p values.
. simanova , groups(#) n(# #..) s(# #..) [ nomp(numlist) reps(#) level(#) mu(numlist) fstar wtest rreg rse seed(#) ]
Examples
The following example performs 1000 ANOVA simulations with 2 groups, both groups having an N=10 and sd=1. . simanova , groups(2) n(10 10) s(1 1)
This example performs 1000 simulations with 3 groups having Ns of 10 10 and 20 and standard deviations of 1 2 and 1. . simanova , groups(3) n(10 10 20) s(1 2 1)
This example is the same as above, but performs 5000 simulations (repetitions) and displays the results using 99% confidence intervals. . simanova , groups(3) n(10 10 20) s(1 2 1) reps(5000) level(99)
This example is the same as above, but performs shows the p values associated with nominal p values of .10 .08 and .04 . . simanova , groups(3) n(10 10 20) s(1 2 1) nomp(.10 .08 .04)
Usage #1 allows you to show the discrepancy between the nominal alpha level and the actual (simulated) alpha level (which can occur when the standard deviations of the groups are not equal). You can also use the fstar or wtest option to explore how the F* and Wtest tests perform. (See Wilcox (1986) referenced below for more information about the F* and Wtest, and for more about simulation techniques). You can also use the mu( ) option to examine the power of a test.
The following example uses the fstar and wtest options to include the F* and Wtest tests in addition to the standard ANOVA test. (If you decide you wish to use the F* and/or Wtest, you can use the fstar and/or wtest commands to perform the F* and Wtest tests, respectively.
. simanova , groups(2) n(10 10) s(1 1) fstar wtest
The following example uses robust standard errors (using the rse option) and robust regression (using the rreg option).
. simanova , groups(2) n(10 10) s(1 1) rse rreg
The following example uses the seed option to specify the starting seed used by the random number generator. By choosing the starting seed, you can replicate results exactly.
. simanova , groups(2) n(10 10) s(1 1) seed(43876)
The following example uses mu option to assess the power of the test to detect the difference between mu1 and mu2 where mu1 is 0 and mu2 is .5.
. simanova , groups(2) n(10 10) s(1 1) mu(0 .5)
.-
Usage #2
. simanova depvar indvar , [ reps(#) level(#) ]
With this usage, simanova computes a one way anova of depvar by indvar and computes the p value using a traditional ANOVA. It then determines the number of groups in indvar and computes the sample sizes and standard deviations for the groups, and then performs ANOVA simulations where the null hypothesis is true, and the number of simulated p values that exceed the p value from the traditional ANOVA are counted. You can then compare the nominal p value from the traditional ANOVA with the p value from the simulations.
Examples
. use https://stats.idre.ucla.edu/stat/stata/notes/hsb2 . simanova science gender . simanova science gender , reps(5000) . simanova science gender , fstar wtest reps(5000)
Usage #2 allows you to use monte carlo simulation to assess the significance of your results computed using a traditional ANOVA. You can use the fstar and wtest options to also assess the significance of your results using simulated F* and Wtest tests. This would be most useful when your sample sizes for the groups are different, and the standard deviations are also different.
For more information about simulation techniques, and about the F* and Wtest tests, see Wilcox, R, Charlin, V, Thompson, K. (1986). Communications in Statistical Simulation and Computation. 15(4) 933-943.
.- Return values
The following values are returned. Each contains the proportion of values which are significant at the levels specified in the nomp option. If more than one level is specified, multiple values are given separated by spaces and can be separated using the tokenize command. The return values differ based on the test used, summarized below.
r(_fp) - using the usual F test. r(_wstatp) - using the W statistic. r(_fstarp) - using the F* statistic. r(rsep) - using robust standard errors. r(_rrp) - using robust regression.
.- Simulation Studies
You can perform simulation studies using simanova, varying the Ns, standard deviations, and examining how the different techniques work under various conditions. Here is an example do file.
set more off
postfile simrse n1 n2 n3 s1 s2 s3 fp rsep rrp wstatp /* */ fstarp using c:simrse , replace
local n2 = 30 local s2 = 5
foreach n1 of numlist 10 30 50 { foreach n3 of numlist 10 30 50 { foreach s1 of numlist 1 5 10 { foreach s3 of numlist 1 5 10 { simanova , groups(3) n(`n1' `n2' `n3') s(`s1' `s2' `s3') /* */ fstar wtest rse rreg nomp(0.05) reps(1000) post simrse (`n1') (`n2') (`n3') (`s1') (`s2') (`s3') /* */ (`r(_fp)') (`r(rsep)') (`r(_rrp)') (`r(_wstatp)') (`r(_fstarp)') } } } }
postclose simrse
You can then summarize the results like this...
use simrse, clear gen str8 s = string(s1,"%02.0f") + "," + string(s2,"%02.0f") /* */ + "," + string(s3,"%02.0f") gen str8 n = string(n1,"%02.0f") + "," + string(n2,"%02.0f") /* */ + "," + string(n3,"%02.0f") display "regular f, wstat, fstar" tabdisp s n, cellvar(fp wstatp fstar) display "regular f, rse, rreg" tabdisp s n, cellvar(fp rsep rrp) display "regular f, wstat, ftar, rse" tabdisp s n, cellvar(fp wstatp fstar rsep)
.-
If you have comments or suggestions, please email Michael Mitchell at mnmatucla.edu .
Author ------
Michael N. Mitchell Statistical Computing and Consulting UCLA, Academic Technology Services mnat@ucla.edu Also see -------- fstar net from https://stats.idre.ucla.edu/stat/stata/ado net install fstar wtest net from https://stats.idre.ucla.edu/stat/stata/ado net install wtest
Statistical Methods and Data Analytics
