In version 9, SAS introduced two new procedures on power and sample size analysis, proc power and proc glmpower. Proc power covers a variety of statistical analyses: tests on means, one-way ANOVA, proportions, correlations and partial correlations, multiple regression and rank test for comparing survival curves. Proc glmpower covers tests related to experimental design models. In this seminar, we will cover various sample power and sample size analysis problems and how to perform the analyses with the two new procedures.
Tests on means
Example 1. One sample mean t-test
Let’s first take a look at the t-test for one sample means. One needs to specify the distribution of the population. SAS can handle two different types of distributions, namely the normal distribution and the lognormal distribution. The null hypothesis can be written as the population mean μ = μ0. The alternative hypothesis can be either one-sided or two sided. The default is two-sided, namely μ ≠ μ0.
In this example, our null hypothesis is that the population mean is zero. If we take a random sample of size 10 with the sample mean is 1 and the sample standard deviation is 1. What is the probability that we will reject the null hypothesis? This is the same as asking what the power of the test is.
We will calculate the power using proc power. The statement is onesamplemeans. The default test is the t-test. The default value for the null hypothesis is zero. We then specify the sample mean, the sample standard deviation and the sample size, i.e., the total number of observations. Since we are to calculate the power of our t-test, we will specify power = . to indicate that it should be calculated. The output below shows that the power is .803. This means that the probability to reject the null hypothesis is about 80%.
proc power; onesamplemeans test=t mean = 1 stddev = 1 ntotal = 10 power = .; run;
Fixed Scenario Elements
Distribution Normal Method Exact Mean 1 Standard Deviation 1 Total Sample Size 10 Number of Sides 2 Null Mean 0 Alpha 0.05
Computed Power
Power
0.803
Example 2. Another example on one-sample mean t-test
What if our null hypothesis is that the population mean is .6 and the sample standard deviation is .5? We can add the option nullmean = .6 as shown below. As we can see from the output, the power is much lower than the previous example, since the difference of the hypothesized mean and the sample mean is much smaller.
proc power; onesamplemeans test=t nullmean = .6 mean = 1 stddev = .5 ntotal = 10 power = .; run;
Fixed Scenario Elements
Distribution Normal Method Exact Null Mean 0.6 Mean 1 Standard Deviation 0.5 Total Sample Size 10 Number of Sides 2 Alpha 0.05
Computed Power
Power
0.616
Example 3. Calculating the sample size for a given power for a one sample mean analysis
proc power; onesamplemeans test=t mean = 1 stddev = 1 ntotal = . power = .90; run;
Fixed Scenario Elements
Distribution Normal Method Exact Mean 1 Standard Deviation 1 Nominal Power 0.9 Number of Sides 2 Null Mean 0 Alpha 0.05
Computed N Total
Actual N Power Total
0.911 13
Example 4: Plotting power vs. sample size for one sample mean analysis
proc power plotonly; ods output plotcontent=PlotData; onesamplemeans test=t nullmean = .6 mean = 1 stddev = 1 ntotal = 10 power = .; plot x=n min=10 max=100 npoints=20; run;
Example 5: Pair-wise t-test with equal variance
proc power;
pairedmeans test=diff
meandiff = 1.5
corr = 0.4
stddev = 2
npairs = 20
power = .;
run;
Paired t Test for Mean Difference
Fixed Scenario Elements
Distribution Normal Method Exact Mean Difference 1.5 Standard Deviation 2 Correlation 0.4 Number of Pairs 20 Number of Sides 2 Null Difference 0 Alpha 0.05
Computed Power
Power
0.827
Example 6: Pair-wise t-test with unequal variances
proc power; pairedmeans test=diff meandiff = 1.5 corr = 0.4 pairedstddevs = (3 1) npairs = 20 power = .; run;
Paired t Test for Mean Difference
Fixed Scenario Elements
Distribution Normal Method Exact Mean Difference 1.5 Standard Deviation 1 3 Standard Deviation 2 1 Correlation 0.4 Number of Pairs 20 Number of Sides 2 Null Difference 0 Alpha 0.05
Computed Power
Power
0.637
Example 7: Two sample means with equal variance
proc power; twosamplemeans test=diff groupmeans = 1.0 | .3 stddev = .9 ntotal = 60 power = .; run;
Two-sample t Test for Mean Difference
Fixed Scenario Elements
Distribution Normal Method Exact Group 1 Mean 1 Group 2 Mean 0.3 Standard Deviation 0.9 Total Sample Size 60 Number of Sides 2 Null Difference 0 Alpha 0.05 Group 1 Weight 1 Group 2 Weight 1
Computed Power
Power
0.842
Another way of performing the same analysis is to specify the number of subjects per group as shown below.
proc power; twosamplemeans test=diff groupmeans = 1.0 | .3 stddev = .9 npergroup = 30 power = .; run;
Fixed Scenario Elements
Distribution Normal Method Exact Group 1 Mean 1 Group 2 Mean 0.3 Standard Deviation 0.9 Sample Size Per Group 30 Number of Sides 2 Null Difference 0 Alpha 0.05
Computed Power
Power
0.842
Example 8: Two sample mean t-test with unequal variance
proc power; twosamplemeans test=diff_satt groupmeans = 1.0 | .3 groupstddevs = 1.5 | .7 ntotal = 60 power = .; run;
Distribution Normal Method Exact Group 1 Mean 1 Group 2 Mean 0.3 Group 1 Standard Deviation 1.5 Group 2 Standard Deviation 0.7 Total Sample Size 60 Number of Sides 2 Null Difference 0 Nominal Alpha 0.05 Group 1 Weight 1 Group 2 Weight 1
Computed Power
Actual Alpha Power
0.0501 0.619
Example 9: Two sample mean t-test with unequal variance and unbalanced groups
proc power; twosamplemeans test=diff_satt groupmeans = 1.0 | .3 groupstddevs = 1.5 | .7 groupns = (40 20) power = .; run;
Fixed Scenario Elements
Distribution Normal Method Exact Group 1 Mean 1 Group 2 Mean 0.3 Group 1 Standard Deviation 1.5 Group 2 Standard Deviation 0.7 Group 1 Sample Size 40 Group 2 Sample Size 20 Number of Sides 2 Null Difference 0 Nominal Alpha 0.05
Computed Power
Actual Alpha Power
0.0499 0.678
Example 10: Two sample mean t-test by specifying the difference in means
proc power; twosamplemeans test=diff_satt meandiff = 3 groupstddevs = 5 | 8 groupweights = (1 2) ntotal = 60 power = .; run;
Fixed Scenario Elements
Distribution Normal Method Exact Mean Difference 3 Group 1 Standard Deviation 5 Group 2 Standard Deviation 8 Group 1 Weight 1 Group 2 Weight 2 Total Sample Size 60 Number of Sides 2 Null Difference 0 Nominal Alpha 0.05
Computed Power
Actual Alpha Power
0.0498 0.415
Tests on correlations and partial correlations
proc power; onecorr dist=fisherz corr = 0.35 nullcorr = .15 ntotal = 180 power = .; run;
Fisher's z Test for Pearson Correlation
Fixed Scenario Elements
Distribution Fisher's z transformation of r Method Normal approximation Null Correlation 0.15 Correlation 0.35 Total Sample Size 180 Number of Sides 2 Nominal Alpha 0.05 Number of Variables Partialled Out 0
Computed Power
Actual Alpha Power
0.05 0.816
proc power; onecorr dist=t npartialvars = 4 corr = 0.45 ntotal = . power = 0.85; run;
t Test for Pearson Correlation
Fixed Scenario Elements
Distribution t transformation of r Method Exact Number of Variables Partialled Out 4 Correlation 0.45 Nominal Power 0.85 Model Random X Number of Sides 2 Alpha 0.05
Computed N Total
Actual N Power Total
0.858 45
Test on proportions
*one sample proportion; proc power; onesamplefreq test=exact /*default*/ nullproportion = 0.2 proportion = 0.3 ntotal = 100 power = .; run;
Fixed Scenario Elements
Method Exact Null Proportion 0.2 Binomial Proportion 0.3 Total Sample Size 100 Number of Sides 2 Nominal Alpha 0.05
Computed Power
Lower Upper Crit Crit Actual Val Val Alpha Power
11 29 0.0326 0.623
proc power; onesamplefreq test=adjz /*normal approximate z test with continuity adjustment*/ nullproportion = 0.3 proportion = 0.2 ntotal = 100 power = .; run;
Z Test for Binomial Proportion with Continuity Adjustment
Fixed Scenario Elements
Method Exact Null Proportion 0.3 Binomial Proportion 0.2 Total Sample Size 100 Number of Sides 2 Nominal Alpha 0.05
Computed Power
Lower Upper Crit Crit Actual Val Val Alpha Power
20 40 0.0375 0.559
*two group chi-square test;
proc power;
twosamplefreq test=pchi
groupproportions = (.3 .2)
npergroup = 50
power = .;
run;
Pearson Chi-square Test for Two Proportions
Fixed Scenario Elements
Distribution Asymptotic normal Method Normal approximation Group 1 Proportion 0.3 Group 2 Proportion 0.2 Sample Size Per Group 50 Number of Sides 2 Null Proportion Difference 0 Alpha 0.05
Computed Power
Power
0.210
Anova
Example 1:
proc power; onewayanova test = overall groupmeans = 5 | 7 | 8 stddev = 6 npergroup = 50 power = .; run;
The POWER Procedure Overall F Test for One-Way ANOVA
Fixed Scenario Elements
Method Exact Group Means 5 7 8 Standard Deviation 6 Sample Size Per Group 50 Alpha 0.05
Computed Power
Power
0.610
Example 2:
proc power; onewayanova test=contrast contrast = (1 0 -1) groupmeans = 5 | 7 | 8 stddev = 6 npergroup = 50 power = .; run;
Single DF Contrast in One-Way ANOVA
Fixed Scenario Elements
Method Exact Contrast Coefficients 1 0 -1 Group Means 5 7 8 Standard Deviation 6 Sample Size Per Group 50 Number of Sides 2 Null Contrast Value 0 Alpha 0.05
Computed Power
Power
0.700
Example 3: Two-factor anova with only main effects
data test2; input a b y; datalines; 1 1 12.4 1 2 10.1 1 3 14.2 2 1 10.1 2 2 9.1 2 3 11.4 ; run; proc means data = test2 mean; class a ; var y; run;
The MEANS Procedure
Analysis Variable : y
N
a Obs Mean
-----------------------------------
1 3 12.2333333
2 3 10.2000000 -----------------------------------
proc means data = test2 mean; class b ; var y; run;
The MEANS Procedure
Analysis Variable : y
N
b Obs Mean
-----------------------------------
1 2 11.2500000
2 2 9.6000000
3 2 12.8000000 -----------------------------------
proc glmpower data = test2; class a b; model y = a b; power stddev = 3.5 ntotal = 120 power = .; run;
Fixed Scenario Elements
Dependent Variable y Error Standard Deviation 3.5 Total Sample Size 120 Alpha 0.05 Error Degrees of Freedom 116
Computed Power
Test Index Source DF Power
1 a 1 0.884
2 b 2 0.960
Example 4: Two-factor ANOVA with interaction effects
proc glmpower data = test2; class a b; model y = a|b; power stddev = 3.5 ntotal = 120 power = .; run;
Fixed Scenario Elements
Dependent Variable y Error Standard Deviation 3.5 Total Sample Size 120 Alpha 0.05 Error Degrees of Freedom 114
Computed Power
Test Index Source DF Power
1 a 1 0.884
2 b 2 0.960
3 a*b 2 0.167
Example 5: More on contrast statement
proc glmpower data=test2;
class a b;
model y = a b;
contrast "b1 vs. other b's " b 2 -1 -1;
contrast "b1 vs. b2" b 1 -1 0;
contrast "b2 vs. b3" b 0 1 -1;
power
stddev = 3.5
ntotal = 120
power = .;
run;
The GLMPOWER Procedure
Fixed Scenario Elements
Dependent Variable y Error Standard Deviation 3.5 Total Sample Size 120 Alpha 0.05 Error Degrees of Freedom 116
Computed Power
Test Index Type Source DF Power
1 Effect a 1 0.884
2 Effect b 2 0.960
3 Contrast b1 vs. other b's 1 0.051
4 Contrast b1 vs. b2 1 0.552
5 Contrast b2 vs. b3 1 0.982
ANCOVA
Example 1:
data test; input a y; datalines; 1 14.5 2 16.8 ; run; proc glmpower data = test; class a ; model y = a ; power stddev = 3.5 nocovariate = 1 corrxy = .2 nototal = 80 power = .; run;
The GLMPOWER Procedure
Fixed Scenario Elements
Dependent Variable y Source a Number of Covariates 1 Corr (Covariates, Response) 0.2 Std Dev Without Covariate Adjustment 3.5 Total Sample Size 80 Alpha 0.05 Std Dev Adjusted for Covariates 3.429286 Test Degrees of Freedom 1 Error Degrees of Freedom 77
Computed Power
Power
0.842
A variation of doing the same analysis is via r-squares.
proc glmpower data = test;
class a ;
model y = a ;
power
stddev = 3.5
nocovariate = 1
pvred = .04
nototal = 80
power = .;
run;
Fixed Scenario Elements
Dependent Variable y Source a Number of Covariates 1 Proportional Variance Reduction From Covariates 0.04 Std Dev Without Covariate Adjustment 3.5 Total Sample Size 80 Alpha 0.05 Std Dev Adjusted for Covariates 3.429286 Test Degrees of Freedom 1 Error Degrees of Freedom 77
Computed Power
Power
0.842
Regression analysis through R-squares
Survival Analysis
Log-rank test is a nonparametric test for comparing two survival curves.
Cantor, A. B. (1997), Extending SAS Survival Analysis Techniques for Medical Research, Cary, NC: SAS Institute Inc.
Lachin, J. (1981), Introduction to Sample Size Determination and Power Analysis for Clinical Trials, Controlled Clinical Trials, 2, 93-113.
B. Brown, J. Lovato, and K. Russell, Asymptotic Power Calculations: Description, examples, computer code, Statist. Med. 18, (1999)