Background
This page shows an example of logistic regression with footnotes explaining the output. This example is from the Mplus User’s Guide (example 3.5) and we suggest that you see the Mplus User’s Guide for more details about this example. We thank the kind people at Muthén & Muthén for permission to use examples from their manual.
By default, Mplus models the logit of the probability of the latter category of a binary outcome.
Note: While most software packages use this typical parameterization of the logistic regression model:
Mplus uses this parameterization of the model:
The parameter $\tau$ is often referred to as the “threshold” and equals the negative of the intercept $\beta_0$ in the more common parameterization, i.e., $\tau=-\beta_0$. The coefficients for the predictors $\beta_1,…,\beta_p$ will have the same interpretation in the two parameterizations.
This example uses the https://stats.idre.ucla.edu/wp-content/uploads/2016/02/ex3.5.dat data file. We model the binary outcome u1, which has values 0 and 1 so Mplus will be modeling the probability of 1. The predictors are x1 and x3.
Input file
TITLE:
this is an example of a logistic
regression for a categorical observed
dependent variable with two covariates
DATA:
FILE = https://stats.idre.ucla.edu/wp-content/uploads/2016/02/ex3.5.dat;
VARIABLE:
NAMES = u1 x1 x3;
CATEGORICAL = u1;
ANALYSIS:
ESTIMATOR = ML;
! need to use estimator = ml to make this a logistic model;
MODEL:
u1 ON x1 x3;
Annotated Output
Note: Much of the output at the top of the output file has been omitted for space.
MODEL FIT INFORMATION
Number of Free Parameters 3
Loglikelihood
H0 Value -202.620
Information Criteria
Akaike (AIC) 411.240
Bayesian (BIC) 423.884
Sample-Size Adjusted BIC 414.362
(n* = (n + 2) / 24)
MODEL FIT INFORMATION
Number of Free Parameters
- number of parameters estimated in the model, here the intercept and coefficients for x1 and x3
Loglikelihood
- H0 Value: log-likelihood value associated with the model
Information Criteria
- Akaike (AIC): Akaike Information Criterion, a relative measure of model fit, calculated as $AIC=-2logLik + 2p$, where $p$ is the number of free parameters
- Bayesian (BIC): Bayesian Information Criterion, another relative measure of model fit that penalizes model complexity more than AIC, and is calculated as $BIC=-2logLik + log(n)p$, where $n$ is the sample size and $p$ is the number of free parameters
- Sample-size adjusted BIC: similar to BIC, but provides a milder penalty for model complexity and is calculated as $ssaBIC=-2logLik + log(\frac{n+2}{24})p$
MODEL RESULTS
Two-Tailed
Estimate S.E. Est./S.E. P-Value
U1 ON
X1 1.072 0.143 7.502 0.000
X3 1.839 0.179 10.243 0.000
Thresholds
U1$1 1.026 0.137 7.492 0.000
MODEL RESULTS
Estimated model parameters, where the columns are:
- Estimate: parameter estimate
- S.E.: standard error of estimate
- Est/S.E.: test statistic (z-statistic), calculated as estimate divided by standard error
- Two-tailed P-value: p-value assessing null hypothesis that parameter is zero in the population; test-statistic is assumed to follow a standard normal distribution under the null hypothesis
The rows of the parameter estimate table are:
- X1: a one-unit increase in x1 is associated with an increase of 1.072 in the log odds of U1=1, and the data provide substantial evidence that this parameter is not zero in the population (p < 0.001)
- X3: a one-unit increase in x3 is associated with an increase of 1.839 in the log odds of U1=1, and the data provide substantial evidence that this parameter is not zero in the population (p < 0.001)
- U1$1: in binary logistic regression, the threshold is interpreted as the log odds of the lower category (here U1=0) when all predictors equal 0. The notation U1\$1 signifies that this is the first threshold for U1. For ordinal outcomes with $k$ categories, including binary outcomes, the number of thresholds will be $k-1$, so there is only one threshold for U1. Tests of null hypotheses are generally not of substantive interest for thresholds, so we typically ignore their associated p-values.
QUALITY OF NUMERICAL RESULTS
Condition Number for the Information Matrix 0.314E+00
(ratio of smallest to largest eigenvalue)
QUALITY OF NUMERICAL RESULTS
- Condition Number for the Information Matrix: the condition number is a diangostic that summarizes the numeric stability of the parameter estimation. The condition number ranges from 0 to 1 and is defined as $C=\frac{\lambda_{min}}{\lambda_{max}}$, where $\lambda_{min}$ and $\lambda_{max}$ are the smallest and largest eigenvalues of the information matrix, respectively. Values close to 1 signify stability, while values near 0 signify numerical instability, where small changes in the data can produce large changes in the parameter estimates, often caused by multicollinearity or weak parameter identification. Small condition numbers near 0 do not necessarily indicate that model results are invalid or untrustworthy.
RESULTS IN PROBABILITY SCALE
Estimate
U1
Category 1 0.654
Category 2 0.346
RESULTS IN PROBABILITY SCALE
- Category 1: overall probability of first category of outcome, here U1=0
- Category 2: overall probability of second category of outcome, here U1=1
LOGISTIC REGRESSION ODDS RATIO RESULTS FOR OBSERVED VARIABLES
95% C.I.
Estimate S.E. Lower 2.5% Upper 2.5%
U1 ON
X1 2.921 0.417 2.207 3.864
X3 6.288 1.129 4.423 8.939
LOGISTIC REGRESSION ODDS RATIO RESULTS FOR OBSERVED VARIABLES
Odds ratios are the exponentiated regression coefficients, and the columns are:
- Estimate: Odds ratio estimate, which is the exponentiation of the coefficients in the Model Results section above, e.g. $exp(1.072)=2.921$
- S.E.: standard error of odds ratio
- 95% C.I., Lower 2.5% and Upper 2.5%: lower and upper limits of 95% confidence intervals for the odds ratios, which give ranges of odds ratios that are reasonable consistent with the data. Assuming the logistic regression model is correct, over repeated samples, 95% of these confidence intervals should contain the true parameter value.
The rows of the table are:
- X1: the odds that U1=1 increases by a factor of 2.921 per unit-increase in X1
- X3: the odds that U1=1 increases by a factor of 6.288 per unit-increase in X3