This chapter uses data file on the effect of social training on socially anxious children. We created an ascii data file https://stats.idre.ucla.edu/wp-content/uploads/2016/02/meta20.txt (shown below) for it. The data file contains six variables: study, d, varofd, rii, ntot and weeks. The variable study is the identification variable and has to be the first column. The second column is the effect size variable and the third is the variance. The rest of the columns will be the level-2 predictor variables.
1 -.264 .086 .9 47 3 2 -.23 .106 .75 38 1 3 .166 .055 .75 74 2 4 .173 .084 .9 48 4 5 .225 .071 .75 57 3 6 .291 .078 .75 53 6 7 .309 .051 .9 80 7 8 .435 .093 .9 51 9 9 .476 .149 .75 32 3 10 .617 .095 .75 46 6 11 .651 .11 .75 56 6 12 .718 .054 .9 79 7 13 .74 .081 .75 55 9 14 .745 .084 .9 51 5 15 .758 .087 .9 59 6 16 .922 .103 .9 56 5 16 .938 .113 .75 45 5 18 .962 .083 .9 54 7 19 1.522 .100 .900 66 9 20 1.844 .141 .75 45 9
The meta-analysis is a V-known model and is done through the command interface. In order to issue HLM commands via the command line from any directory, we need to add HLM to the path. Click here for detailed instructions.
Table 8.3 on page 148.This type of analysis is done through command
line interactively. We will show the detailed steps here. At the end of this
interactive process, we name our .ssm file to be meta20.ssm.
Step 1: Creating the SSM File
HLM creates .ssm (sufficient statistics matrix) file from a raw data file.
E:hoxascii>hlm2 Will you be starting with raw data? y Is the input file a v-known file? y How many level-1 statistics are there? 1 How many level-2 predictors are there? 3 Enter 8 character name for level-1 variable number 1: d Enter 8 character name for level-2 variable number 1: rii Enter 8 character name for level-2 variable number 2: ntot Enter 8 character name for level-2 variable number 3: weeks Input format of raw data file (the first field must be the character ID) format: (a2, 3F11.3, 2F8.0) What file contains the data?meta20.csv Enter name of SSM file: meta20.ssm 20 groups have been processed
Step 2: Estimating a V-known model
After creating the .ssm file, we are ready to perform the analysis.
Part 1: Intercept only.
E:hoxascii>hlm2 meta20.ssm SPECIFYING AN HLM2 MODEL Level-1 predictor variable specification Which level-1 predictors do you wish to use? The choices are: For D enter 1 level-1 predictor? (Enter 0 to end) 1 Level-2 predictor variable specification Which level-2 variables do you wish to use? The choices are: For RII enter 1 For NTOT enter 2 For WEEKS enter 3 Which level-2 predictors to model D? Level-2 predictor? (Enter 0 to end) 0 ADDITIONAL PROGRAM FEATURES Select the level-2 variables that you might consider for inclusion as predictors in subsequent models. The choices are: For RII enter 1 For NTOT enter 2 For WEEKS enter 3 Which level-2 variables to model D? Level-2 variable? (Enter 0 to end) 0 Do you want to run this analysis with a heterogeneous sigma^2? n Do you wish to use any of the optional hypothesis testing procedures? n Do you want to do a latent variable regression? n OUTPUT SPECIFICATION Do you want a residual file? n How many iterations do you want to do? 1000 Do you want to see OLS estimates for all of the level-2 units? n Enter a problem title: Intercept Only Model Enter name of output file: meta20_m1.txt Computing . . ., please wait
The output is included below.
Problem Title: Intercept Only Model Summary of the model specified (in equation format) --------------------------------------------------- Level-1 Model Y1 = B1 + E1 Level-2 Model B1 = G10 + U1 Tau D,B1 0.14461 Tau (as correlations) D,B1 1.000 ---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- D, B1 0.620 ---------------------------------------------------- The value of the likelihood function at iteration 11 = -4.416426E+001 Final estimation of fixed effects: ---------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------------------------- INTRCPT2, G10 0.580146 0.108002 5.372 19 0.000 ---------------------------------------------------------------------------- Final estimation of variance components: ----------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ----------------------------------------------------------------------------- D, U1 0.38027 0.14461 19 49.79738 0.000 ----------------------------------------------------------------------------- Statistics for current covariance components model -------------------------------------------------- Deviance = 88.328524 Number of estimated parameters = 2
Table 8.3 Part 2: The variable time is included.
E:hoxascii>hlm2 meta20.ssm SPECIFYING AN HLM2 MODEL Level-1 predictor variable specification Which level-1 predictors do you wish to use? The choices are: For D enter 1 level-1 predictor? (Enter 0 to end) 1 Level-2 predictor variable specification Which level-2 variables do you wish to use? The choices are: For RII enter 1 For NTOT enter 2 For WEEKS enter 3 Which level-2 predictors to model D? Level-2 predictor? (Enter 0 to end) 3 Level-2 predictor? (Enter 0 to end) 0 ADDITIONAL PROGRAM FEATURES Select the level-2 variables that you might consider for inclusion as predictors in subsequent models. The choices are: For RII enter 1 For NTOT enter 2 For WEEKS enter 3 Which level-2 variables to model D? Level-2 variable? (Enter 0 to end) 0 Do you want to run this analysis with a heterogeneous sigma^2? n Do you wish to use any of the optional hypothesis testing procedures? n Do you want to do a latent variable regression? n OUTPUT SPECIFICATION Do you want a residual file? n How many iterations do you want to do? 10000 Do you want to see OLS estimates for all of the level-2 units? n Enter a problem title: Variable Time is included Enter name of output file: meta20_m2.txt
The major part of the output is shown below.
Problem Title: Variable Time is included Summary of the model specified (in equation format) --------------------------------------------------- Level-1 Model Y1 = B1 + E1 Level-2 Model B1 = G10 + G11*(WEEKS) + U1 Tau D,B1 0.03663 Tau (as correlations) D,B1 1.000 ---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- D, B1 0.297 ---------------------------------------------------- The value of the likelihood function at iteration 11 = -3.777495E+001 Final estimation of fixed effects: ---------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------------------------- INTRCPT2, G10 -0.216959 0.204343 -1.062 18 0.303 WEEKS, G11 0.139929 0.033791 4.141 18 0.001 ---------------------------------------------------------------------------- Final estimation of variance components: ----------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ----------------------------------------------------------------------------- D, U1 0.19139 0.03663 18 26.45735 0.090 ----------------------------------------------------------------------------- Statistics for current covariance components model -------------------------------------------------- Deviance = 75.549906 Number of estimated parameters = 2
Table 8.4 on page 151. The first model and the fourth model are
the models from Table 8.3 and we omit them here.
Model 1: The variable
ntot is included in the model.
The setup of the model is similar to the example above and we omit it here.
We only show the output below.
Problem Title: Ntot is included Level-1 Model Y1 = B1 + E1 Level-2 Model B1 = G10 + G11*(NTOT) + U1 Tau D,B1 0.15921 Tau (as correlations) D,B1 1.000 ---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- D, B1 0.642 ---------------------------------------------------- The value of the likelihood function at iteration 9 = -5.353864E+001 Final estimation of fixed effects: ---------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------------------------- INTRCPT2, G10 0.442971 0.512570 0.864 18 0.399 NTOT, G11 0.002489 0.009006 0.276 18 0.785 ---------------------------------------------------------------------------- Final estimation of variance components: ----------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ----------------------------------------------------------------------------- D, U1 0.39901 0.15921 18 49.92053 0.000 ----------------------------------------------------------------------------- Statistics for current covariance components model -------------------------------------------------- Deviance = 107.077277 Number of estimated parameters = 2
Model 2: The variable rii is included in the model.
The setup of the model is similar to the example above and we omit it here.
We only show the output below.
Problem Title: Variable RII is included Level-1 Model Y1 = B1 + E1 Level-2 Model B1 = G10 + G11*(RII) + U1 Tau D,B1 0.15648 Tau (as correlations) D,B1 1.000 ---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- D, B1 0.638 ---------------------------------------------------- The value of the likelihood function at iteration 11 = -4.328604E+001 Final estimation of fixed effects: ---------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------------------------- INTRCPT2, G10 0.160176 1.227267 0.131 18 0.898 RII, G11 0.508698 1.477294 0.344 18 0.734 ---------------------------------------------------------------------------- Final estimation of variance components: ----------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ----------------------------------------------------------------------------- D, U1 0.39557 0.15648 18 49.23675 0.000 ----------------------------------------------------------------------------- Statistics for current covariance components model -------------------------------------------------- Deviance = 86.572080 Number of estimated parameters = 2
Model 3: All of the variables are included.
The setup of the model is similar to the example above and we omit it here.
We only show the output below.
Problem Title: All Variables Are Included Level-1 Model Y1 = B1 + E1 Level-2 Model B1 = G10 + G11*(RII) + G12*(NTOT) + G13*(WEEKS) + U1 Tau D,B1 0.04934 Tau (as correlations) D,B1 1.000 ---------------------------------------------------- Random level-1 coefficient Reliability estimate ---------------------------------------------------- D, B1 0.362 ---------------------------------------------------- The value of the likelihood function at iteration 11 = -4.683295E+001 Final estimation of fixed effects: ---------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ---------------------------------------------------------------------------- INTRCPT2, G10 0.384280 0.923448 0.416 16 0.682 RII, G11 -0.550999 1.201005 -0.459 16 0.652 NTOT, G12 -0.003571 0.007035 -0.508 16 0.618 WEEKS, G13 0.150605 0.037551 4.011 16 0.001 ---------------------------------------------------------------------------- Final estimation of variance components: ----------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ----------------------------------------------------------------------------- D, U1 0.22213 0.04934 16 25.43980 0.062 ----------------------------------------------------------------------------- Statistics for current covariance components model -------------------------------------------------- Deviance = 93.665910 Number of estimated parameters = 2