Skip to Main Content

Regression Analysis by Example, Third Edition Chapter 3: Multiple Linear Regression | Stata Textbook Examples

Table 3.3, page 54.

use https://stats.idre.ucla.edu/stat/stata/examples/chp/p054, clear
list y-x3

             y         x1         x2         x3 
  1.        43         51         30         39  
  2.        63         64         51         54  
  3.        71         70         68         69  
  4.        61         63         45         47
..
  [remainder of output omitted]

list x4-x6

            x4         x5         x6 
  1.        61         92         45  
  2.        63         73         47  
  3.        76         86         48  
  4.        54         84         35
..
  [remainder of output omitted]

Coefficients for equation 3.12, page 57.

regress y x1 x2

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  2,    27) =   29.10
   Model |  2935.10281     2  1467.55141               Prob > F      =  0.0000
Residual |  1361.86385    27  50.4394019               R-squared     =  0.6831
---------+------------------------------               Adj R-squared =  0.6596
   Total |  4296.96667    29  148.171264               Root MSE      =  7.1021

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .7803434   .1193876      6.536   0.000       .5353802    1.025307
      x2 |  -.0501598   .1299192     -0.386   0.702       -.316732    .2164124
   _cons |   15.32762   7.160234      2.141   0.041       .6360331    30.01921
------------------------------------------------------------------------------

Coefficients for equation 3.13, page 57.

regress y x1

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  1,    28) =   59.86
   Model |  2927.58425     1  2927.58425               Prob > F      =  0.0000
Residual |  1369.38241    28  48.9065148               R-squared     =  0.6813
---------+------------------------------               Adj R-squared =  0.6699
   Total |  4296.96667    29  148.171264               Root MSE      =  6.9933

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .7546098   .0975329      7.737   0.000       .5548227    .9543969
   _cons |   14.37632   6.619986      2.172   0.039       .8158929    27.93675
------------------------------------------------------------------------------

Compute residuals for this model.

Note: The variable eyx1 is set to type double for greater precision.

predict double eyx1, residual

Coefficients for equation 3.14, page 57.

regress x2 x1

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  1,    28) =   12.68
   Model |  1353.17314     1  1353.17314               Prob > F      =  0.0013
Residual |  2988.29352    28  106.724769               R-squared     =  0.3117
---------+------------------------------               Adj R-squared =  0.2871
   Total |  4341.46667    29  149.705747               Root MSE      =  10.331

------------------------------------------------------------------------------
      x2 |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |    .513032   .1440789      3.561   0.001       .2178997    .8081642
   _cons |    18.9654   9.779267      1.939   0.063      -1.066517    38.99732
------------------------------------------------------------------------------

Compute residuals for this model.

predict double ex2x1, residual

Table 3.4, page 58.

list eyx1 ex2x1

          eyx1      ex2x1 
  1. -9.861421  -15.13003  
  2.  .3286522  -.7994502  
  3.  3.800993   13.12236  
  4.  -.916738  -6.286418  
  5.  7.764115  -2.981898  
  6. -12.87986   1.817838  
  7. -6.935177  -11.33855  
  8.  .0279442  -7.442802  
  9. -4.254324   10.96597  
 10.  6.592482  -5.260354  
 11.   9.62936   6.843902  
 12.  7.347092  -2.747322  
 13.  7.837872   6.226614  
 14. -9.008934   21.45294  
 15.  4.518724  -4.468866  
 16. -1.291203  -15.13828  
 17. -4.518154   1.426878  
 18.  5.347092   15.25268  
 19. -2.199007  -8.877642  
 20. -8.143689   19.27874  
 21.  5.439288  -6.486683  
 22.  3.592482   1.739646  
 23. -11.18057  -.8255141  
 24. -2.296883   4.052413  
 25.  7.874751   -4.66913  
 26. -6.481276   7.531134  
 27.  7.027944    .557198  
 28. -9.389079  -4.208226  
 29.  6.481846   8.426878  
 30.  5.745676  -22.03403 

Coefficients for equation 3.15, page 58.

Note: Values such as -9.19e-15 are zero as represented by Stata in double
precision.

regress eyx1 ex2x1
      Source |       SS       df       MS              Number of obs =      30
-------------+------------------------------           F(  1,    28) =    0.15
       Model |  7.51856161     1  7.51856161           Prob > F      =  0.6972
    Residual |  1361.86385    28  48.6379947           R-squared     =  0.0055
-------------+------------------------------           Adj R-squared = -0.0300
       Total |  1369.38241    29  47.2200832           Root MSE      =  6.9741

------------------------------------------------------------------------------
        eyx1 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       ex2x1 |  -.0501598   .1275781    -0.39   0.697    -.3114917    .2111721
       _cons |  -9.19e-15    1.27329    -0.00   1.000    -2.608216    2.608216
------------------------------------------------------------------------------

Equation 3.25, page 62 and table 3.5, page 63.

regress y x1 x2 x3 x4 x5 x6

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  6,    23) =   10.50
   Model |  3147.96634     6  524.661057               Prob > F      =  0.0000
Residual |  1149.00032    23  49.9565359               R-squared     =  0.7326
---------+------------------------------               Adj R-squared =  0.6628
   Total |  4296.96667    29  148.171264               Root MSE      =   7.068

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .6131876   .1609831      3.809   0.001       .2801687    .9462065
      x2 |  -.0730501   .1357247     -0.538   0.596       -.353818    .2077178
      x3 |   .3203321   .1685203      1.901   0.070      -.0282787    .6689429
      x4 |   .0817321   .2214777      0.369   0.715      -.3764293    .5398936
      x5 |   .0383814   .1469954      0.261   0.796      -.2657018    .3424647
      x6 |  -.2170567   .1782095     -1.218   0.236       -.585711    .1515977
   _cons |   10.78708   11.58926      0.931   0.362      -13.18713    34.76128
------------------------------------------------------------------------------ 

Table 3.7, page 67.

test x1 x2 x3 x4 x5 x6

 ( 1)  x1 = 0.0
 ( 2)  x2 = 0.0
 ( 3)  x3 = 0.0
 ( 4)  x4 = 0.0
 ( 5)  x5 = 0.0
 ( 6)  x6 = 0.0

       F(  6,    23) =   10.50
            Prob > F =    0.0000

Equation 3.40, page 68.

test x2 x4 x5 x6

 ( 1)  x2 = 0.0
 ( 2)  x4 = 0.0
 ( 3)  x5 = 0.0
 ( 4)  x6 = 0.0

       F(  4,    23) =    0.53
            Prob > F =    0.7158

Table 3.8, page 69.

regress y x1 x3

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  2,    27) =   32.74
   Model |   3042.3177     2  1521.15885               Prob > F      =  0.0000
Residual |  1254.64897    27  46.4684804               R-squared     =  0.7080
---------+------------------------------               Adj R-squared =  0.6864
   Total |  4296.96667    29  148.171264               Root MSE      =  6.8168

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .6435176   .1184774      5.432   0.000        .400422    .8866132
      x3 |   .2111918   .1344037      1.571   0.128      -.0645818    .4869655
   _cons |    9.87088   7.061224      1.398   0.174      -4.617553    24.35931
------------------------------------------------------------------------------

Equation following 3.48, page 71.

generate w = x1 + x3
regress y w

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  1,    28) =   56.46
   Model |  2872.37206     1  2872.37206               Prob > F      =  0.0000
Residual |  1424.59461    28  50.8783788               R-squared     =  0.6685
---------+------------------------------               Adj R-squared =  0.6566
   Total |  4296.96667    29  148.171264               Root MSE      =  7.1329

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
       w |   .4443897   .0591439      7.514   0.000       .3232388    .5655406
   _cons |   9.988214    7.38841      1.352   0.187      -5.146257    25.12268
------------------------------------------------------------------------------

F-ratio middle of page 71.

regress y x1 x3 

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  2,    27) =   32.74
   Model |   3042.3177     2  1521.15885               Prob > F      =  0.0000
Residual |  1254.64897    27  46.4684804               R-squared     =  0.7080
---------+------------------------------               Adj R-squared =  0.6864
   Total |  4296.96667    29  148.171264               Root MSE      =  6.8168

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .6435176   .1184774      5.432   0.000        .400422    .8866132
      x3 |   .2111918   .1344037      1.571   0.128      -.0645818    .4869655
   _cons |    9.87088   7.061224      1.398   0.174      -4.617553    24.35931
------------------------------------------------------------------------------

test x1 = x3

 ( 1)  x1 - x3 = 0.0

       F(  1,    27) =    3.66
            Prob > F =    0.0665            

F-ratio near the top of page 72.

Note: The accum option allows you to accumulate the simultaneous results of
of consecutive test commands.

regress y x1 x2 x3 x4 x5 x6

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  6,    23) =   10.50
   Model |  3147.96634     6  524.661057               Prob > F      =  0.0000
Residual |  1149.00032    23  49.9565359               R-squared     =  0.7326
---------+------------------------------               Adj R-squared =  0.6628
   Total |  4296.96667    29  148.171264               Root MSE      =   7.068

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .6131876   .1609831      3.809   0.001       .2801687    .9462065
      x2 |  -.0730501   .1357247     -0.538   0.596       -.353818    .2077178
      x3 |   .3203321   .1685203      1.901   0.070      -.0282787    .6689429
      x4 |   .0817321   .2214777      0.369   0.715      -.3764293    .5398936
      x5 |   .0383814   .1469954      0.261   0.796      -.2657018    .3424647
      x6 |  -.2170567   .1782095     -1.218   0.236       -.585711    .1515977
   _cons |   10.78708   11.58926      0.931   0.362      -13.18713    34.76128
------------------------------------------------------------------------------

test x1=x3

 ( 1)  x1 - x3 = 0.0

       F(  1,    23) =    1.21
            Prob > F =    0.2821


test x2 x4 x5 x6, accum
 ( 1)  x1 - x3 = 0.0
 ( 2)  x2 = 0.0
 ( 3)  x4 = 0.0
 ( 4)  x5 = 0.0
 ( 5)  x6 = 0.0

       F(  5,    23) =    1.10
            Prob > F =    0.3857

Method 1: Equations at the bottom of page 72 and top of page 73.

generate yprime = y - x3
generate v = x1 - x3
regress yprime v

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  1,    28) =   37.79
   Model |  1794.31921     1  1794.31921               Prob > F      =  0.0000
Residual |  1329.54746    28  47.4838377               R-squared     =  0.5744
---------+------------------------------               Adj R-squared =  0.5592
   Total |  3123.86667    29   107.71954               Root MSE      =  6.8909

------------------------------------------------------------------------------
  yprime |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
       v |   .6938232   .1128683      6.147   0.000        .462623    .9250233
   _cons |   1.166543   1.707882      0.683   0.500      -2.331896    4.664982
------------------------------------------------------------------------------

display 1 - .6938232
.3061768

Method 2: Equations at the bottom of page 72 and top of page 73. cnsreg,
constrained linear regression, computes the coefficients for both x1 and x3 in one command.

constraint define 1  x1 + x3 = 1
cnsreg y x1 x3, c(1)

Constrained linear regression                          Number of obs =      30
                                                       Root MSE      =  6.8909
 ( 1)  x1 + x3 = 1.0
------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .6938232   .1128683      6.147   0.000        .462623    .9250233
      x3 |   .3061768   .1128683      2.713   0.011       .0749767     .537377
   _cons |   1.166543   1.707882      0.683   0.500      -2.331896    4.664982
------------------------------------------------------------------------------

F-ratio at the top of page 73.

regress y x1 x3

  Source |       SS       df       MS                  Number of obs =      30
---------+------------------------------               F(  2,    27) =   32.74
   Model |   3042.3177     2  1521.15885               Prob > F      =  0.0000
Residual |  1254.64897    27  46.4684804               R-squared     =  0.7080
---------+------------------------------               Adj R-squared =  0.6864
   Total |  4296.96667    29  148.171264               Root MSE      =  6.8168

------------------------------------------------------------------------------
       y |      Coef.   Std. Err.       t     P>|t|       [95% Conf. Interval]
---------+--------------------------------------------------------------------
      x1 |   .6435176   .1184774      5.432   0.000        .400422    .8866132
      x3 |   .2111918   .1344037      1.571   0.128      -.0645818    .4869655
   _cons |    9.87088   7.061224      1.398   0.174      -4.617553    24.35931
------------------------------------------------------------------------------

test x1 + x3 = 1

 ( 1)  x1 + x3 = 1.0

       F(  1,    27) =    1.61
            Prob > F =    0.2151