Proc Logistic | SAS Annotated Output

Background

This page shows an example of logistic regression with footnotes explaining the output. The data were collected on 200 high school students, with measurements on various tests, including science, math, reading and social studies. The response variable is high writing test score (honcomp), where a writing score greater than or equal to 60 is considered high, and less than 60 considered low; from which we explore its relationship with gender (female), reading test score (read), and science test score (science). The dataset used in this page can be downloaded from SAS Web Books Regression with SAS.

Model code

data logit;
 set "c:\temp\hsb2";
 honcomp = (write >= 60);
run;

proc logistic data= logit descending; 
 model honcomp = female read science;
run;

Annotated Output

Model Information

• Data Set: This the data set used in this procedure.
• Response Variable: This is the response variable in the logistic regression.
• Number of Response Levels: This is the number of levels our response variable has.
• Model: This is the type of regression model that was fit to our data. The term logit and logistic are exchangeable.
• Optimization Technique: This refers to the iterative method of estimating the regression parameters. In SAS, the default is method is Fisher’s scoring method. For further discussion, see Regression Models for Categorical and Limited Dependent Variables by J. Scott Long (page 56).

Number of Observations Read

• Number of Observations Read: number of observations read from the data set
• Number of Observations Used: number of observations used for estimation, which can be less than the number read because observations with missing are typically dropped during estimation

Response Profile

Frequencies of outcome values. Columns are:

• Ordered Value: rank order of value; in PROC LOGIT, SAS models the value of the first ordered value
• honcomp: the actual values of outcome honcomp that correspond to the ordered values. Because we used descending on the PROC LOGIT statement, the higher value of honcomp, 1, becomes the first ordere value.
• Total Frequency: number of observations (used) with corresponding value of honcomp

Probability modeled is honcomp=1: This is a note informing which level of the response variable we are modeling. .

Model Convergence Status

Describes whether the maximum-likelihood algorithm has converged or not, and what kind of convergence criterion is used to assess convergence. The default criterion is the relative gradient convergence criterion (GCONV), and the default precision is 10^-8.

Model Fit Statistics

Relative fit statistics, with columns:

• Criterion: fit statistic label
• Intercept Only: fit statistic value for model with only an intercept
• Intercept and Covariates: fit statistic for fitted model; if this value is close to the value in the Intercept Only column, the model covariates do not explain much variation in the outcome.

The rows of this table are:

• AIC:This is the Akaike Information Criterion. It is calculated as $AIC = -2LogLik + 2((k-1) + s)$, where $k$ is the number of levels of the dependent variable and $s$ is the number of predictors in the model. AIC is used for the comparison of nonnested models on the same sample. Ultimately, the model with the smallest AIC is considered the best.
• SC: This is the Schwarz Criterion. It is defined as $ SC= – 2LogLik + ((k-1) + s)*log(\sum f_{i})$, where $f_i$’s are the frequency values of the $i$th observation, and $k$ and $s$ were defined previously. Like AIC, SC penalizes for the number of predictors in the model and the smallest SC is most desirable and the value itself is not meaningful.
• -2 Log L: This is negative two times the log-likelihood. The -2 Log L is used in hypothesis tests for nested models and the value in itself is not meaningful.

Testing Global Null Hypothesis: BETA=0

Provides three asymptotically equivalent Chi-Square tests of the null hypothesis that all of the predictors’ regression coefficients are simultaneously equal to zero in the model. Rejection of the null suggests that at least one of the coefficients is different from zero. The difference between the various tests are where on the log-likelihood function they are evaluated. For further discussion, see Categorical Data Analysis, Second Edition, by Alan Agresti (pages 11-13). The columns are:

• Test: name of test
• Chi-square: value of test statistic
• DF: degrees of freedom for null Chi-square distribution, typically equals the number of estimated parameters
• Pr > ChiSq: p-value corresponding to test of null hypothesis that all coefficients are zero; a small p-value suggests that at least one covariate may be important in explaining variation in the outcome

The specific tests are:

• Likelihood Ratio: Likelihood Ratio (LR) Chi-Square test. The LR Chi-Square statistic measures how much the likelihood is improved by adding the predictors to the model and can be calculated by  $LR=-2LogLik_{null} – 2LogLik_{fitted} = 231.289-160.236 = 71.05$, where $LogLik_{null}$ is the log-likelihood of model with only and intercept, and $LogLik_{fitted}$ is the log-likelihood fo the fitted model.
• Score: Score Chi-Square Test, which evaluates the slope/curvature of the log-likelihood function with respect to the coefficients under the null hypothesis that the coefficients are zero. A large statistic reflects that the slope is steep and that the log-likelihood is rapidly changing as the coefficient values move away from zero, i.e., that non-zero coefficient values improve the fit of the model to the data.
• Wald: Wald Chi-Square Test, which measures the coefficient values relative to their estimated variability, while accounting for their correlations. A large statistic suggests that data provide evidence that at least one coefficient is non-zero after taking into consideration their variability.

All three tests suggest that at least one parameter related to the predictors is non-zero in the population.

Analysis of Maximum Likelihood Estimates

Logistic regression parameter estimates table, where the columns are:

• Parameter: parameter label
• DF: model degrees of freedom used to estimate corresponding parameter(s), and degrees of freedom for null Wald chi-square distribution
• Estimate: estimated parameters in log-odds (logit) units
• Standard Error: standard error of estimate, an estimate of sampling variability of the parameter
• Wald Chi-square: test statistic assessing null hypothesis that the parameter is zero in the population, given the other parameters in the model, and is calculated as the square of the Estimate divided by its Standard Error. Under the null hypothesis, this statistic should be chi-square distributed with 1 degree of freedom.
• Pr > ChiSq: p-value for the Wald Chi-square, which is the probability of observing a Wald-statistic equal or greater than the fitted Wald-statistic in repeate samples. Low p-value suggest rejection of the null hypothesis.

Specific interpretations of the parameter estimates:

• Intercept: Estimated log-odds that honcomp=1 when all variables in the model are evaluated at zero. For males (the variable female evaluated at zero) with zero read and science test scores, the log-odds for high write score is -12.777. Note that evaluating read and science at zero is out of the range of plausible test scores. If the test scores were mean-centered, the intercept would have a natural interpretation: the expected log-odds for high write score for males with an average read and science test score.
• female: The difference in log-odds of honcomp=1 is expected to be 1.4825 units higher for females compared to males, while holding the other variables constant in the model.
• read: If a student were to increase read score by one point, the log-odds for honcomp=1 is expected to increase by 0.10 unit, given the other variables in the model are held constant.
• science: If a student were to increase science score by one point, the log-odds of honcomp=1 is expected to increase by 0.095 unit, given the other variables in the model are held constant.

The data provide evidence that all the coefficients for female, read and science are all non-zero in the population.

Odds Ratio Estimates

Exponentiated coefficients for predictors are interpreted as odds ratios. The columns of this table are:

• Effect: predictor
• Point Estimate: estimated odds ratio, obtained by exponentiating the corresponding Estimate from the Maximum Likelihood Estimates table, e.g., $exp(1.4825) = 4.404$
• 95% Wald Confidence Limits: lower and upper limit of 95% confidence interval for the odds ratio, which gives a range of values of the odds ratio that are reasonably consistent with the data. In repeated samples, assuming the model is correct, we would expect that 95% of such confidence intervals would contain the true population odds ratio. These confidence intervals are obtained by exponentiating the confidence limits of the corresponding log-odds coefficient. Because exponentiation is a non-linear transformation, the 95% confidence interval is not symmetric around the odds ratio.

To interpret specific odds ratios (we use different styles of interpretations for each odds ratio):

• female: females are expected to more than 4 times the odds of honcomp=1 than males, holding read an science constant. Note, however, that the 95% confidence interval suggests that a wide range of odds ratios for the female effect are consistent with the data.
• read: If a student were to increase read score by one point, the odds of honcomp=1 are expected to increase by 10.9% unit, given the other variables in the model are held constant.
• science: If a student were to increase science score by one point, the odds of honcomp=1 are expected to increase by a factor of 1.099, given the other variables in the model are held constant.

Association of Predicted Probabilites and Observed Responses

• Percent Concordant: A pair of observations with different observed responses is said to be concordant if the observation with the lower ordered response value (honcomp = 0) has a lower predicted mean score (predicted probability) than the observation with the higher ordered response value (honcomp = 1). A high value near 100 suggests that the model predictions fit the data well. See Pairs for what defines a pair.
• Percent Discordant: If the observation with the lower ordered response value has a higher predicted mean score than the observation with the higher ordered response value, then the pair is discordant.
• Percent Tied: If a pair of observations with different responses is neither concordant nor discordant, it is a tie.
• Pairs: This is the total number of distinct pairs of observations in which one case has an observed outcome different from the other member of the pair (one has honcomp=0, the other honcomp=1). In the Response Profile table in the Model Information section above, we see that there are 53 observations with honcomp=1 and 147 observations with honcomp=0. Thus the total number of pairs with different outcomes is 53*147=7791.
• Somers’ D: Somer’s D is used to determine the strength and direction of relation between pairs of variables. Its values range from -1.0 (all pairs disagree) to 1.0 (all pairs agree). It is defined as $D=\frac{n_c-n_d}{t}$ where $n_c$ is the number of pairs that are concordant, $n_d$ the number of pairs that are discordant, and t is the number of total number of pairs with different responses. In our example, it equals the difference between the percent concordant and the percent discordant divided by 100: (85.6-14.2)/100 = 0.714.
• Gamma: The Goodman-Kruskal Gamma method does not penalize for ties on either variable. Its values range from -1.0 (no association) to 1.0 (perfect association). Because it does not penalize for ties, its value will generally be greater than the values for Somer’s D.
• Tau-a: Kendall’s Tau-a is a modification of Somer’s D that takes into the account the difference between the number of possible paired observations and the number of paired observations with a different response. It is defined to be the ratio of the difference between the number of concordant pairs and the number of discordant pairs to the number of possible pairs, $\frac{2(n_c-n_d)}{(N(N-1)}$. Usually Tau-a is much smaller than Somer’s D since there would be many paired observations with the same response.
• c: c is essentially the percent concordant, but also giving half-credit to tied observations. It is calucated as $c=\frac{n_c + 0.5n_t}{n}$, where $n_c$ is the number of discordant pairs and $n_t$ is the number of tied pairs. Equivalently, $c=\frac{percent\ concordant}{100} + 0.5\frac{percent\ tied}{100}$. The c -statistic is equivalent to the well known measure, area under the ROC curve. c ranges from 0.5 to 1, where 0.5 corresponds to the model randomly predicting the response, and a 1 corresponds to the model perfectly discriminating the response.