The purpose of this seminar is to explore how to analyze survey data collected under different sampling plans using Stata 9. Other examples, including those using other survey data analysis packages, can be found at Choosing the Correct Analysis for Various Survey Designs. Before we begin looking at examples in Stata, we will quickly review some basic issues and concepts in survey data analysis.
Why do we need survey data analysis software?
Regular statistical software (that is not designed for survey data) analyzes data as if the data were collected using simple random sampling. For experimental and quasi-experimental designs, this is exactly what we want. However, when surveys are conducted, a simple random sample is rarely collected. Not only is it nearly impossible to do so, but it is not as efficient (both financially and statistically) as other sampling methods. When any sampling method other than simple random sampling is used, we usually need to use survey data analysis software to take into account the differences between the design that was used and simple random sampling. This is because the sampling design affects the calculation of the standard errors of the estimates. If you ignore the sampling design, e.g., if you assume simple random sampling when another type of sampling design was used, the standard errors will likely be underestimated, possibly leading to results that seem to be statistically significant, when in fact, they are not. The difference in point estimates and standard errors obtained using non-survey software and survey software with the design properly specified will vary from data set to data set, and even between variables within the same data set. While it may be possible to get reasonably accurate results using non-survey software, there is no practical way to know beforehand how far off the results from non-survey software will be.
Sampling designs
Most people do not conduct their own surveys. Rather, they use survey data that some agency or company collected and made available to the public. The documentation must be read carefully to find out what kind of sampling design was used to collect the data. This is very important because many of the estimates and standard errors are calculated differently for the different sampling designs. Hence, if you mis-specify the sampling design, the point estimates and standard errors will likely be wrong.
Below are some common features of many sampling designs.
Weights: There are many types of weights that can be associated with a survey. Perhaps the most common is the sampling weight, sometimes called a pweight, which is used to denote the inverse of the probability of being included in the sample due to the sampling design (except for a certainty PSU, see below). The pweight is calculated as N/n, where N = the number of elements in the population and n = the number of elements in the sample. For example, if a population has 10 elements and 3 are sampled at random with replacement, then the pweight would be 10/3 = 3.33. In a two-stage design, the pweight is calculated as f1f2, which means that the inverse of the sampling fraction for the first stage is multiplied by the inverse of the sampling fraction for the second stage. Under many sampling plans, the sum of the pweights will equal the population total.
PSU: This is the primary sampling unit. This is the first unit that is sampled in the design. For example, school districts from California may be sampled and then schools within districts may be sampled. The school district would be the PSU. If states from the US were sampled, and then school districts from within each state, and then schools from within each district, then states would be the PSU. One does not need to use the same sampling method at all levels of sampling. For example, probability-proportional-to-size sampling may be used at level 1 (to select states), while cluster sampling is used at level 2 (to select school districts). In the case of a simple random sample, the PSUs and the elementary units are the same.
Strata: Stratification is a method of breaking up the population into different groups, often by demographic variables such as gender, race or SES. Once these groups have been defined, one samples from each group as if it were independent of all of the other groups. For example, if a sample is to be stratified on gender, men and women would be sampled independent of one another. This means that the pweights for men will likely be different from the pweights for the women. In most cases, you need to have two or more PSUs in each stratum. The purpose of stratification is to improve the precision of the estimates, and stratification works most effectively when the variance of the dependent variable is smaller within the strata than in the sample as a whole.
FPC: This is the finite population correction. This is used when the sampling fraction (the number of elements or respondents sampled relative to the population) becomes large. The FPC is used in the calculation of the standard error of the estimate. If the value of the FPC is close to 1, it will have little impact and can be safely ignored. In some survey data analysis programs, such as SUDAAN, this information will be needed if you specify that the data were collected without replacement (see below for a definition of “without replacement”). The formula for calculating the FPC is ((N-n)/(N-1))1/2, where N is the number of elements in the population and n is the number of elements in the sample. To see the impact of the FPC for samples of various proportions, suppose that you had a population of 10,000 elements.
Sample size (n) FPC 1 1.0000 10 .9995 100 .9950 500 .9747 1000 .9487 5000 .7071 9000 .3162
Sampling with and without replacement
Most samples collected in the real world are collected “without replacement”. This means that once a respondent has been selected to be in the sample and has participated in the survey, that particular respondent cannot be selected again to be in the sample. Many of the calculations change depending on if a sample is collected with or without replacement. Hence, programs like SUDAAN request that you specify if a survey sampling design was implemented with our without replacement, and an FPC is used if sampling without replacement is used, even if the value of the FPC is very close to one.
Examples
In the examples that follow, we have data that represent a population, and we will discuss the analysis of these survey data as if they had been collected under five sampling plans: simple random sampling, stratified random sampling, systematic sampling, one-stage cluster sampling and two-stage cluster sampling with stratification. The Stata code necessary to generate the samples using each of these sampling plans is shown here. The variables from the data set with which we will be working include api00 and api99, which is an aggregate of student test scores for each school, for the years 2000 and 1999, respectively; yr_rnd, which is a 0/1 variable indicating if the school is on a year-round calendar; awards, which indicates whether or not the school met their target; meals, which indicates the percentage of children receiving free or reduced-priced meals at school; both, which indicates that the school met both targets; and growth, which is the difference between the api scores in the current year and those of the last year.
One of the most important points to remember is that all svy commands can be used with any sampling plan. To help illustrate this, we will use the svy: mean and the svy: total commands with each sampling plan. Another important point is that the interpretation of the results from the svy commands is usually no different than the interpretation that you would have if you had used the equivalent non-survey command. For example, there is no special interpretation of regression coefficients just because you obtained them using svy: reg instead of regress.
Simple random sample
We will start by showing how you can take a simple random sample (SRS) from you data file. While we will not go through the commands necessary for obtaining any other type of sample, we will go over how to draw an SRS. Simple random samples are very rare in actual practice; however, researchers will often draw an SRS of their data set so that they can work out their data analysis programs on a relatively small data file. This saves computing time and resources, as the analysis program may have to be run many times before it is satisfactory.
set mem 5m use https://stats.idre.ucla.edu/stat/stata/seminars/svy_stata_intro/apipop, clear count 6194 set seed 1003002849 sample 5 count 310
Because we have eliminated elements of our population to create our sample, we need to create pweights (probability weights). We selected 5% of the elements in our population into our sample, so our sampling fraction is 310/6194. The pweight is the inverse of the sampling fraction, or N/n, where N is the population total (6194) and n is the number of elements selected into the sample (310). Another way to think of this is: “How many elements (schools, people, whatever) in the population should each element in the sample represent?” Clearly, each school in our current sample should represent twenty schools in the population, so all of the p-weights will be the same; approximately 20.
gen pw = 6194/310
Next, we need to consider how large our sample is relative to our population to determine if we need to use a finite population correction. (For a quick review of FPCs, please see the summary at the beginning of this handout.) In Stata, we only need to give the population total, and Stata will make the necessary calculations to obtain the correct FPC. Note that the svyset command is very different in Stata 8 than it was in Stata 7.
gen fpc = 6194
We use the svyset command to tell Stata about the features of the sampling design that we have. In this case, we only need to specify the pweight and the FPC.
svyset [pweight=pw], fpc(fpc)
pweight: pw
VCE: linearized
Strata 1: <one>
SU 1: <observations>
FPC 1: fpc
Next, we will use the svydes command to display the information Stata has regarding our sampling plan. As you can see, the number of PSUs and observations is the same, which reassures us that Stata understands that we have a simple random sample. We also see that there is only one strata, which is correct for this type of sampling plan. Note that once you have used the svyset command, Stata will remember this information for your entire session; you do not need to reissue this command (unless you want to change something). Also, if you save your data, Stata will save the survey information with the data set, so that when you open the data in your next session, the survey information will be used when you issue svy commands.
svydesSurvey: Describing stage 1 sampling units pweight: pw VCE: linearized Strata 1: <one> SU 1: <observations> FPC 1: fpc #Obs per Unit ---------------------------- Stratum #Units #Obs min mean max -------- -------- -------- -------- -------- -------- 1 310 310 1 1.0 1 -------- -------- -------- -------- -------- -------- 1 310 310 1 1.0 1
We will start our analysis of these data with some basic descriptive statistics. We will use the svy: mean and svy: total commands. The svy: mean command is used to estimate the mean of a variable in the population. In our example, we will estimate the mean for api00 and growth. Please note that svy: mean is an estimation command, and Stata will do a listwise deletion of missing data. For example, if we had missing data on api00, we would probably get a different mean for growth than if we issued the command svy: mean growth because there would a different number of cases used in the calculation of the mean.
svy: mean api00 growth(running mean on estimation sample) Survey: Mean estimation Number of strata = 1 Number of obs = 310 Number of PSUs = 310 Population size = 6194 Design df = 309 -------------------------------------------------------------- | Linearized | Mean Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ api00 | 663.2645 7.21478 649.0682 677.4608 growth | 33.84516 1.667394 30.56428 37.12604 --------------------------------------------------------------
The svy: total command is used to get estimates of population totals. In our example, we will get an estimate of how many schools are on a year-round calendar. From the output of the svy: total command, we can see that approximately 719 schools are on a year-round calendar.
svy: total yr_rnd(running total on estimation sample) Survey: Total estimation Number of strata = 1 Number of obs = 310 Number of PSUs = 310 Population size = 6194 Design df = 309 -------------------------------------------------------------- | Linearized | Total Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ yr_rnd | 719.3032 110.0291 502.8022 935.8042 --------------------------------------------------------------
We will now do a multiple regression. We will use api00 as the dependent variable and award and meals as independent variables. We can see from the output that the model is statistically significant (F = 464.21, p < .000), and that each of the predictors is also statistically significant. You can interpret the output from the svy commands in the same way that you would the non-svy command. In this example, you interpret the output from the svy: reg command in the same way that you would the output from the regress command. Remember that the difference between the svy: reg and the regress commands is how the standard errors are calculated. The svy: reg command takes into account the survey sampling plan, while the regress command does not.
svy: reg api00 awards meals(running regress on estimation sample) Survey: Linear regression Number of strata = 1 Number of obs = 310 Number of PSUs = 310 Population size = 6193.9997 Design df = 309 F( 2, 308) = 464.21 Prob > F = 0.0000 R-squared = 0.7124 ------------------------------------------------------------------------------ | Linearized api00 | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- awards | 53.37164 9.047051 5.90 0.000 35.57002 71.17326 meals | -3.329605 .1285952 -25.89 0.000 -3.582638 -3.076571 _cons | 738.9207 19.47419 37.94 0.000 700.6019 777.2395 ------------------------------------------------------------------------------
Stratified random sampling
The difference between the example above and the next example is that stratification has been added to the sampling design. For this example, we have calculated the mean of api99 and stratified schools based on this. Schools that were above the mean were placed into one strata, and schools that were below the mean were placed in the other strata. Simple random samples of schools were then drawn from each strata. Although we have created only two strata, in many public-use data sets, you can have dozens of strata.
We have used the svyset, clear(all) command here to show how it is used. After issuing the svyset command, we again use the svydes command to ensure that Stata is handling the survey design appropriately. Next, we use the svymean to obtain the estimated means of api00 and api99. We can compare these estimates to those obtained from the SRS above. (Please see the table at the end of this handout.)
use https://stats.idre.ucla.edu/stat/stata/seminars/svy_stata_intro/strsrs, clear svyset, clear(all) svyset [pweight = pw], strata(strat) fpc(fpc)pweight: pw VCE: linearized Strata 1: strat SU 1: <observations> FPC 1: fpc svydes Survey: Describing stage 1 sampling units pweight: pw VCE: linearized Strata 1: strat SU 1: <observations> FPC 1: fpc #Obs per Unit ---------------------------- Stratum #Units #Obs min mean max -------- -------- -------- -------- -------- -------- 1 310 310 1 1.0 1 2 310 310 1 1.0 1 -------- -------- -------- -------- -------- -------- 2 620 620 1 1.0 1
Below we use the svy: mean command to get the population estimate of the mean of api00. We can use the estat effects command to get the design effect. Notice the value of the design effect, labeled Deff in the output. The design effect compares the current sampling design (in this case, stratified random sampling) with simple random sampling. Design effects of 1 (or close to 1) indicate that the current sampling design is about as efficient as a simple random sample. Design effects that are smaller than 1 indicate that the current design is more efficient than simple random sampling, while design effects that are larger than 1 indicate that the current sampling design is less efficient than simple random sampling. Here, we can see the benefit of the stratification: the design effect for api00 is .35, well below 1. However, you will remember that we stratified on the mean of api99, which is closely related to api00, the variable for which we are getting an estimate.
svy: mean api00 growth(running mean on estimation sample) Survey: Mean estimation Number of strata = 2 Number of obs = 620 Number of PSUs = 620 Population size = 6194 Design df = 618 -------------------------------------------------------------- | Linearized | Mean Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ api00 | 665.6216 2.957053 659.8145 671.4287 growth | 33.26666 1.10437 31.09788 35.43543 --------------------------------------------------------------estat effects ---------------------------------------------------------- | Linearized | Mean Std. Err. Deff Deft -------------+-------------------------------------------- api00 | 665.6216 2.957053 .346875 .558707 growth | 33.26666 1.10437 .962983 .930909 ---------------------------------------------------------- Note: Weights must represent population totals for deff to be correct when using an FPC; however, deft is invariant to the scale of weights.
In the results of the svy: total shown below, you will see that the design effect is not much smaller than 1; in other words, we get relatively little benefit from the stratification. That is because there is not much of a relationship between api99 and yr_rnd. The point here is that to be genuinely useful, you need to stratify on variable(s) closely related to the variable of interest. In many cases, this will mean that while stratification will make some estimates more efficient, it will not do so for others.
svy: total yr_rnd(running total on estimation sample) Survey: Total estimation Number of strata = 2 Number of obs = 620 Number of PSUs = 620 Population size = 6194 Design df = 618 -------------------------------------------------------------- | Linearized | Total Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ yr_rnd | 789.5516 76.59607 639.1314 939.9717 --------------------------------------------------------------
When estimates are made for each strata, they are made independently of all other strata. In other words, the estimate of yr_rnd for strata 1 was made independently of the estimate for strata 2. Also note that the sum of the estimates for strata 1 and strata 2 equals the value shown above.
svy: total yr_rnd, over(strat)
(running total on estimation sample)
Survey: Total estimation
Number of strata = 2 Number of obs = 620
Number of PSUs = 620 Population size = 6194
Design df = 618
1: strat = 1
2: strat = 2
--------------------------------------------------------------
| Linearized
Over | Total Std. Err. [95% Conf. Interval]
-------------+------------------------------------------------
yr_rnd |
1 | 639.7935 67.69423 506.8549 772.7321
2 | 149.7581 35.83923 79.37663 220.1395
--------------------------------------------------------------
Systematic sampling
Systematic sampling is just that: drawing a sample from elements that are ordered in a systematic way. For example, you might take a systematic sample of library books by selecting every k-th book from the books on the shelf. (Remember that librarians hate when people actually do this!) Of course, first you need to determine how large of a sample you want to select. There are 6194 schools in our sample, and we would like to use systematic sampling to select a sample of size 500. First, we need to determine the “rate” at which schools should be selected. We do this by dividing the number of elements (e.g., schools) by the number desired in the sample. Therefore, k = 6194/500 = 12.38, which we will round to 13. Hence, we will select every 13th school. We will also randomly select a number from 1 to 13 and start counting from there. In our example, we selected the number 4. Hence, we ordered the schools from lowest id number to highest id number, started with school number 4, and then selected into our sample every 13th school. After creating our sample, we follow the same procedure as before: open the correct data file, issue the svyset command, check to see that everything is OK with the svydes command, and then begin our analysis with descriptive statistics.
use https://stats.idre.ucla.edu/stat/stata/seminars/svy_stata_intro/systematic.dta, clear svyset [pweight = pw], fpc(fpc)pweight: pw VCE: linearized Strata 1: <one> SU 1: <observations> FPC 1: fpc svydes Survey: Describing stage 1 sampling units pweight: pw VCE: linearized Strata 1: <one> SU 1: <observations> FPC 1: fpc #Obs per Unit ---------------------------- Stratum #Units #Obs min mean max -------- -------- -------- -------- -------- -------- 1 477 477 1 1.0 1 -------- -------- -------- -------- -------- -------- 1 477 477 1 1.0 1
Below we get the population estimates for the mean of api00 and growth.
svy: mean api00 growth(running mean on estimation sample) Survey: Mean estimation Number of strata = 1 Number of obs = 477 Number of PSUs = 477 Population size = 6194 Design df = 476 -------------------------------------------------------------- | Linearized | Mean Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ api00 | 656.3061 5.655353 645.1935 667.4186 growth | 33.08595 1.226588 30.67576 35.49615 --------------------------------------------------------------estat effects ---------------------------------------------------------- | Linearized | Mean Std. Err. Deff Deft -------------+-------------------------------------------- api00 | 656.3061 5.655353 1 .960724 growth | 33.08595 1.226588 1 .960724 ---------------------------------------------------------- Note: Weights must represent population totals for deff to be correct when using an FPC; however, deft is invariant to the scale of weights.
Notice that the design effect for all variables is 1. This is not necessarily because systematic sampling is always just as efficient as simple random sampling. Rather, it has to do with the information that you have given to Stata. The design effect is influenced by setting the strata and PSU. In both simple random sampling and systematic sampling, we set neither the strata or PSU. Hence, Stata “can’t tell the two sampling plans apart.” Because the specification of the sampling design is exactly the same as with simple random sampling, the design effect is 1. However, you can calculate the design effect by hand by dividing the variance of the variable of interest under the current sampling design by the variance of the same variable under simple random sampling. We did this and found that the design effects were very close to 1. We found them to be .96 for api00, .93 for growth and 1.2 for yr_rnd.
svy: total yr_rnd(running total on estimation sample) Survey: Total estimation Number of strata = 1 Number of obs = 477 Number of PSUs = 477 Population size = 6194 Design df = 476 -------------------------------------------------------------- | Linearized | Total Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ yr_rnd | 779.1195 90.44644 601.3958 956.8432 --------------------------------------------------------------
Below we show the use of the svy: tab command. This can be used to make one- and two-way crosstabulations. Here we will make a crosstab of both and awards. The values in the cells are proportions. You can use the count option (as shown below) to obtain the counts in each cell. The svy: tab command also gives us the chi-square test for these two variables. We can see that the relationship between them is statistically significant.
svy: tab both awards(running tabulate on estimation sample) Number of strata = 1 Number of obs = 477 Number of PSUs = 477 Population size = 6194 Design df = 476 ------------------------------- met both | eligible for awards targets | no yes Total ----------+-------------------- No | .3019 0 .3019 Yes | .0503 .6478 .6981 | Total | .3522 .6478 1 ------------------------------- Key: cell proportions Pearson: Uncorrected chi2(1) = 379.3900 Design-based F(1, 476) = 427.4673 P = 0.0000svy: tab both awards, count (running tabulate on estimation sample) Number of strata = 1 Number of obs = 477 Number of PSUs = 477 Population size = 6194 Design df = 476 ------------------------------- met both | eligible for awards targets | no yes Total ----------+-------------------- No | 1870 0 1870 Yes | 311.6 4012 4324 | Total | 2182 4012 6194 ------------------------------- Key: weighted counts Pearson: Uncorrected chi2(1) = 379.3900 Design-based F(1, 476) = 427.4673 P = 0.0000svy: reg api00 award meals (running regress on estimation sample) Survey: Linear regression Number of strata = 1 Number of obs = 477 Number of PSUs = 477 Population size = 6194 Design df = 476 F( 2, 475) = 679.67 Prob > F = 0.0000 R-squared = 0.6967 ------------------------------------------------------------------------------ | Linearized api00 | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- awards | 46.30969 7.237096 6.40 0.000 32.08908 60.53029 meals | -3.406531 .1056495 -32.24 0.000 -3.614128 -3.198934 _cons | 791.0985 9.321325 84.87 0.000 772.7825 809.4146 ------------------------------------------------------------------------------
One-stage cluster sampling in Stata
In a one-stage cluster sample, the data are divided into two “levels”, one “nested” in the other. At the first level, the data are grouped into clusters. In a one-stage cluster sample, clusters are selected first and are called primary sampling units, or PSUs. All of the elements in each selected cluster are selected into the sample. These elements represent the second “level” of the data. In our one-stage cluster sample, the districts will be the clusters and the schools will be the elementary or sampling units. Hence, we randomly select school districts and then select all schools within each selected district. You can use any sampling plan to select the clusters; we have used SRS only for the sake of simplicity.
Typically, data values in one cluster are more similar to one another than data values in another cluster. For example, if we surveyed people in households (e.g., people nested within households), we would expect that people in one household would be more similar to one another than they would be to people in another household. Unfortunately, this feature makes our standard errors less efficient. However, because of financial and/or logistical considerations, most surveys employ some sort of cluster sampling.
use https://stats.idre.ucla.edu/stat/stata/seminars/svy_stata_intro/oscs1, clear svyset dnum [pweight = pw], fpc(fpc) pweight: pw VCE: linearized Strata 1: <one> SU 1: dnum FPC 1: fpc svydes Survey: Describing stage 1 sampling units pweight: pw VCE: linearized Strata 1: <one> SU 1: dnum FPC 1: fpc #Obs per Unit ---------------------------- Stratum #Units #Obs min mean max -------- -------- -------- -------- -------- -------- 1 189 1463 1 7.7 100 -------- -------- -------- -------- -------- -------- 1 189 1463 1 7.7 100svy: mean api00 growth(running mean on estimation sample) Survey: Mean estimation Number of strata = 1 Number of obs = 1463 Number of PSUs = 189 Population size = 5859.74 Design df = 188 -------------------------------------------------------------- | Linearized | Mean Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ api00 | 670.5202 11.09702 648.6295 692.4108 growth | 32.85783 1.440905 30.01541 35.70025 --------------------------------------------------------------svy: total yr_rnd(running total on estimation sample) Survey: Total estimation Number of strata = 1 Number of obs = 1463 Number of PSUs = 189 Population size = 5859.74 Design df = 188 -------------------------------------------------------------- | Linearized | Total Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ yr_rnd | 797.0529 176.0585 449.7489 1144.357 --------------------------------------------------------------
As you can see, the standard errors for these estimates are much larger than they were for any of the previous sampling plans. Although we don’t show an example here, you can easily combine stratification with cluster sampling, and this will help to make the standard errors more efficient.
Two-stage cluster sampling with stratification
In this last example, we will take a stratified two-stage cluster sample. As with the stratified random sample illustrated above, the sampling for each strata will be done independent of every other strata. A two-stage cluster sample means that clusters will be sampled (using whatever sampling plan the researcher chooses), and then elements within each of the selected clusters will also be sampled. This is different from what we did above in that, in a one-stage cluster sample, all of the elements in each selected cluster are selected into the sample. In a two-stage cluster sample, (usually) only some of the elements are selected into the sample. In our example, we will take an SRS of school districts (clusters), and then we will take an SRS of schools (elements). In the same way that you can use pretty much any sampling plan to select clusters, you can use pretty much any sampling plan to select elements from within the selected clusters; the sampling plan for selecting the clusters does not have to be the same as the one for selecting the elements. Also, you do not have to use the same sampling plan from one strata to the next, as the sampling between strata is independent. To obtain the sample used below, we first used the stratification that we used before, stratifying schools based on their mean api99 score. Next, we randomly selected 25% of the school districts from each strata. Finally, we randomly selected three schools from each selected district. The choice to select three schools, as opposed to selecting two or four schools, was rather arbitrary. However, when deciding how many elements to select from a cluster, remember that you need to have a sufficient number to get stable estimates; however, because data values within each cluster are likely correlated, taking lots of them is often a waste of resources: 200 elements probably won’t be any more informative than 100. (This, of course, depends on how strong the correlation is.)
use https://stats.idre.ucla.edu/stat/stata/seminars/svy_stata_intro/strataboth, clear svyset dnum [pweight = pwt], fpc(fpc) strata(strata)pweight: pwt VCE: linearized Strata 1: strata SU 1: dnum FPC 1: fpc svydes Survey: Describing stage 1 sampling units pweight: pwt VCE: linearized Strata 1: strata SU 1: dnum FPC 1: fpc #Obs per Unit ---------------------------- Stratum #Units #Obs min mean max -------- -------- -------- -------- -------- -------- 1 94 227 1 2.4 3 2 95 239 1 2.5 3 -------- -------- -------- -------- -------- -------- 2 189 466 1 2.5 3svy: mean api00 growth(running mean on estimation sample) Survey: Mean estimation Number of strata = 2 Number of obs = 466 Number of PSUs = 189 Population size = 6032.9 Design df = 187 -------------------------------------------------------------- | Linearized | Mean Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ api00 | 681.84 10.44856 661.2278 702.4522 growth | 30.71763 2.22572 26.32688 35.10838 --------------------------------------------------------------svy: total yr_rnd(running total on estimation sample) Survey: Total estimation Number of strata = 2 Number of obs = 466 Number of PSUs = 189 Population size = 6032.9 Design df = 187 -------------------------------------------------------------- | Linearized | Total Std. Err. [95% Conf. Interval] -------------+------------------------------------------------ yr_rnd | 718.9149 214.9205 294.9345 1142.895 --------------------------------------------------------------svy: reg api00 awards meals(running regress on estimation sample) Survey: Linear regression Number of strata = 2 Number of obs = 466 Number of PSUs = 189 Population size = 6032.9042 Design df = 187 F( 2, 186) = 556.68 Prob > F = 0.0000 R-squared = 0.7114 ------------------------------------------------------------------------------ | Linearized api00 | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- awards | 66.19885 5.867421 11.28 0.000 54.624 77.77369 meals | -3.192264 .1135934 -28.10 0.000 -3.416353 -2.968175 _cons | 772.7654 6.72774 114.86 0.000 759.4934 786.0374 ------------------------------------------------------------------------------
We have seen examples of how to do OLS regression with survey data, so now let’s do a logistic regression. First, we need to recode our dependent variable so that is 0/1. Next, we issue the svy: logit command. If you want odds ratios, you can use the or option with svy: logit. In this example, we use some new variables. The variable comp_imp1 is coded 0/1 and indicates if the school met a comparable improvement target; growth is the difference between the current year’s api score and last year’s api score; ell is the percent of English language learners; and mobility is the percent of students for whom this is their first year at the school.
svy: logit comp_imp1 growth ell mobility(running logit on estimation sample) Survey: Logistic regression Number of strata = 2 Number of obs = 466 Number of PSUs = 189 Population size = 6032.9042 Design df = 187 F( 3, 185) = 20.80 Prob > F = 0.0000 ------------------------------------------------------------------------------ | Linearized comp_imp1 | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- growth | .1213203 .0159442 7.61 0.000 .0898667 .1527739 ell | -.0702944 .0119777 -5.87 0.000 -.0939231 -.0466657 mobility | -.0781154 .0202496 -3.86 0.000 -.1180624 -.0381684 _cons | .6391637 .3169899 2.02 0.045 .013828 1.264499 ------------------------------------------------------------------------------
Now we will use a three-level variable to show the use of the test command. Please note that “svytest” is an out-of-date command. As you can see, the xi prefix works with the svy commands (and so does xi3). However, you need to use the prefixes in the correct order: “svy: xi: logit” does not work.
xi: svy: logit comp_imp1 growth ell mobility i.meals3 i.meals3 _Imeals3_1-3 (naturally coded; _Imeals3_1 omitted) (running logit on estimation sample) Survey: Logistic regression Number of strata = 2 Number of obs = 466 Number of PSUs = 189 Population size = 6032.9042 Design df = 187 F( 5, 183) = 14.28 Prob > F = 0.0000 ------------------------------------------------------------------------------ | Linearized comp_imp1 | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- growth | .1333139 .0177526 7.51 0.000 .0982928 .168335 ell | -.0335437 .0134298 -2.50 0.013 -.0600371 -.0070503 mobility | -.0528434 .0194839 -2.71 0.007 -.09128 -.0144068 _Imeals3_2 | -1.976366 .3789415 -5.22 0.000 -2.723916 -1.228817 _Imeals3_3 | -2.54474 .9051281 -2.81 0.005 -4.330314 -.7591659 _cons | .5236906 .2685344 1.95 0.053 -.0060555 1.053437 ------------------------------------------------------------------------------test _Imeals3_2 _Imeals3_3Adjusted Wald test
( 1) _Imeals3_2 = 0 ( 2) _Imeals3_3 = 0
F( 2, 186) = 15.94
Prob > F = 0.0000
Summary of population values, estimates, standard errors, design effects and estimated population totals for each sampling plan
The table below summarizes the values obtained from the descriptive statistics that we ran under each of the sampling plans, as well as the estimated population size. It also contains the population values, which, of course, are not estimates, and hence do not have standard errors or design effects associated with them. (To obtain the design effects, you will need to issue the estat effects command after the analysis command.) Design effects are the ratio of the variance of the variable under the current sampling design to the estimated variance under simple random sampling. In other words, it is an estimate of efficiency of the current sampling design relative to simple random sampling. As you can see, the standard errors and the design effects for the stratified simple random sample are the smallest, followed closely by those for the simple random sample. The design effects obtained under the systematic sample are slightly larger, and they become even larger when cluster sampling is used. The largest design effects are obtained using stratified one-stage cluster sampling. Also notice that cluster sampling yields estimates of the population size that are considerably different from those obtained using other types of sampling plans. You should not assume that this pattern of results will be obtained every time these sampling plans are compared. Some plans that look relatively inefficient in this example may appear to be more efficient with other samples and/or other data.
| mean api00 | mean growth | total yr_rnd | estimated population size | |||||||
| estimate | standard error | design effect | estimate | standard error | design effect | estimate | standard error | design effect | ||
| population values | 664.71 | N/A | N/A | 32.80 | N/A | N/A | 874 | N/A | N/A | 6194 |
| SRS | 663.26 | 7.21 | 1 | 33.85 | 1.67 | 1 | 719.30 | 110.03 | 1 | 6194 |
| Stratified SRS | 665.62 | 2.96 | .35 | 33.27 | 1.10 | .96 | 789.55 | 76.60 | .95 | 6194 |
| Systematic | 656.31 | 5.66 | 1 | 33.09 | 1.23 | 1 | 779.12 | 90.45 | 1 | 6194 |
| One-stage cluster | 670.52 | 11.10 | 15.28 | 32.86 | 1.44 | 4.55 | 797.05 | 176.06 | 14.97 | 5860 |
| Stratified two-stage cluster | 681.84 | 10.45 | 3.90 | 30.72 | 2.23 | 3.01 | 818.92 | 214.92 | 6.09 | 6033 |