1. Introduction
This module illustrates (1) how to create and recode
variables manually and (2) how to use foreach to ease the process of
creating and recoding variables.
Consider the sample program below, which reads
in income data for twelve months.
input famid inc1-inc12 1 3281 3413 3114 2500 2700 3500 3114 3319 3514 1282 2434 2818 2 4042 3084 3108 3150 3800 3100 1531 2914 3819 4124 4274 4471 3 6015 6123 6113 6100 6100 6200 6186 6132 3123 4231 6039 6215 endlist
The output is shown below
list famid inc1-inc12, clean famid inc1 inc2 inc3 inc4 inc5 inc6 inc7 inc8 inc9 inc10 inc11 inc12 1 3281 3413 3114 2500 2700 3500 3114 3319 3514 1282 2434 2818 2 4042 3084 3108 3150 3800 3100 1531 2914 3819 4124 4274 4471 3 6015 6123 6113 6100 6100 6200 6186 6132 3123 4231 6039 6215
2. Computing variables (manually)
Say that we
wanted to compute the amount of tax (10%) paid for each month, the simplest
way to do this is to compute 12 variables (taxinc1-taxinc12)
by multiplying each of the (inc1-inc12) by .10 as
illustrated below. As you see, this requires entering a command
computing the tax for each month of data (for months 1 to 12) via the generate
command.
generate taxinc1 = inc1 * .10 generate taxinc2 = inc2 * .10 generate taxinc3 = inc3 * .10 generate taxinc4 = inc4 * .10 generate taxinc5 = inc5 * .10 generate taxinc6 = inc6 * .10 generate taxinc7 = inc7 * .10 generate taxinc8 = inc8 * .10 generate taxinc9 = inc9 * .10 generate taxinc10= inc10 * .10 generate taxinc11= inc11 * .10 generate taxinc12= inc12 * .10
The output is shown below.
+----------------------------------------------------------------------------------------------+
1. | famid | inc1 | inc2 | inc3 | inc4 | inc5 | inc6 | inc7 | inc8 | inc9 | inc10 | inc11 | inc12 |
| 1 | 3281 | 3413 | 3114 | 2500 | 2700 | 3500 | 3114 | 3319 | 3514 | 1282 | 2434 | 2818 |
|----------------------------------------------------------------------------------------------|
| taxinc1 | taxinc2 | taxinc3 | taxinc4 | taxinc5 | taxinc6 | taxinc7 | taxinc8 | taxinc9 |
| 328.1 | 341.3 | 311.4 | 250 | 270 | 350 | 311.4 | 331.9 | 351.4 |
|----------------------------------------------------------------------------------------------|
| taxinc10 | taxinc11 | taxinc12 |
| 128.2 | 243.4 | 281.8 |
+----------------------------------------------------------------------------------------------+
+----------------------------------------------------------------------------------------------+
2. | famid | inc1 | inc2 | inc3 | inc4 | inc5 | inc6 | inc7 | inc8 | inc9 | inc10 | inc11 | inc12 |
| 2 | 4042 | 3084 | 3108 | 3150 | 3800 | 3100 | 1531 | 2914 | 3819 | 4124 | 4274 | 4471 |
|----------------------------------------------------------------------------------------------|
| taxinc1 | taxinc2 | taxinc3 | taxinc4 | taxinc5 | taxinc6 | taxinc7 | taxinc8 | taxinc9 |
| 404.2 | 308.4 | 310.8 | 315 | 380 | 310 | 153.1 | 291.4 | 381.9 |
|----------------------------------------------------------------------------------------------|
| taxinc10 | taxinc11 | taxinc12 |
| 412.4 | 427.4 | 447.1 |
+----------------------------------------------------------------------------------------------+
+----------------------------------------------------------------------------------------------+
3. | famid | inc1 | inc2 | inc3 | inc4 | inc5 | inc6 | inc7 | inc8 | inc9 | inc10 | inc11 | inc12 |
| 3 | 6015 | 6123 | 6113 | 6100 | 6100 | 6200 | 6186 | 6132 | 3123 | 4231 | 6039 | 6215 |
|----------------------------------------------------------------------------------------------|
| taxinc1 | taxinc2 | taxinc3 | taxinc4 | taxinc5 | taxinc6 | taxinc7 | taxinc8 | taxinc9 |
| 601.5 | 612.3 | 611.3 | 610 | 610 | 620 | 618.6 | 613.2 | 312.3 |
|----------------------------------------------------------------------------------------------|
| taxinc10 | taxinc11 | taxinc12 |
| 423.1 | 603.9 | 621.5 |
+----------------------------------------------------------------------------------------------+
3. Computing variables (using the foreach command)
Another way to compute 12 variables representing
the amount of tax paid (10%) for each month is to use the
foreach command. In the example below we use the foreach command to cycle through the variables
inc1 to inc12
and compute the taxable income as taxinc1 – taxinc12.
foreach var of varlist inc1-inc12 {
generate tax`var' = `var' * .10
}
The initial foreach statement tells Stata that we want to
cycle through the variables inc1 to inc12 using the statements
that are surrounded by the curly braces. The first time we cycle through
the statements, the value of var will be inc1 and the
second time the value of var will be inc2 and so on until
the final iteration where the value of var will be inc12.
Each statement within the loop (in this case, just the one generate statement)
is evaluated and executed. When we are inside the foreach loop, we can
access the value of var by surrounding it with the funny quotation
marks like this `var’ . The ` is the quote
right below the ~ on your keyboard and the ‘ is the quote below the ” on your
keyboard. The first time through the loop, `var’ is replaced with inc1,
so the statement
generate tax`var' = `var' * .10
becomes
generate taxinc1 = inc1 * .10
This is repeated for inc2 and then inc3 and so on
until inc12. So, this foreach loop is the equivalent of
executing the 12 generate statements manually, but much easier and less
error prone.
4. Collapsing across variables (manually)
Often one needs to sum across variables (also
known as collapsing across variables). For example, let’s say the
quarterly income for each observation is desired. In order to get this
information, four quarterly variables incqtr1-incqtr4 need to
be computed. Again, this can be achieved manually or by using the foreach
command. Below is an example of how to compute 4 quarterly income variables incqtr1-incqtr4 by simply adding together the months that
comprise a quarter.
generate incqtr1 = inc1 + inc2 + inc3 generate incqtr2 = inc4 + inc5 + inc6 generate incqtr3 = inc7 + inc8 + inc9 generate incqtr4 = inc10+ inc11+ inc12list incqtr1 – incqtr4
The output is shown below.
+---------------------------------------+
| incqtr1 incqtr2 incqtr3 incqtr4 |
|---------------------------------------|
1. | 9808 8700 9947 6534 |
2. | 10234 10050 8264 12869 |
3. | 18251 18400 15441 16485 |
+---------------------------------------+
5. Collapsing across variables (using the foreach command)
This same result as above can be achieved using
the foreach command. The example below illustrates how to
compute the quarterly income variables incqtr1-incqtr4 using
the foreach command.
foreach qtr of numlist 1/4 {
local m3 = `qtr'*3
local m2 = (`qtr'*3)-1
local m1 = (`qtr'*3)-2
generate incqtr`qtr' = inc`m1' + inc`m2' + inc`m3'
}
list incqtr1 - incqtr4
The output is shown below.
+---------------------------------------+
| incqtr1 incqtr2 incqtr3 incqtr4 |
|---------------------------------------|
1. | 9808 8700 9947 6534 |
2. | 10234 10050 8264 12869 |
3. | 18251 18400 15441 16485 |
+---------------------------------------+
In this example, instead of cycling across variables, the foreach
command is cycling across numbers, 1, 2, 3 then 4 which we refer to as qtr
which represent the 4 quarters of variables that we wish to create. The
trick is the relationship between the quarter and the month numbers that
compose the quarter and to create a kind of formula that relates the quarters
to the months. For example, quarter 1 of data corresponds to months 3, 2
and 1, so we can say that when the quarter (qtr) is 1 we want the months
represented by qtr*3, (qtr*3)-1 and (qtr*3)-2, yielding 3, 2, and 1.
This is what the statements below from the foreach loop are doing.
They are relating the quarter to the months.
local m3 = `qtr'*3 local m2 = (`qtr'*3)-1 local m1 = (`qtr'*3)-2
So, when qtr is 1, the value for m3 is 1*3, the value
for m2 is (1*3)-1 and the value for m1 is (1*3)-2.
Then, imagine all of those values being substituted into the following
statement from the foreach loop.
generate incqtr`qtr' = inc`m1' + inc`m2' + inc`m3'
This then becomes
generate incqtr1 = inc3 + inc2 + inc1
and for the next quarter (when qtr becomes 2) the statement
would become
generate incqtr2 = inc6 + inc5 + inc4
In this example, with only 4 quarters of data, it would probably be easier to
simply write out the 4 generate statements manually, however if you had
40 quarters of data, then the foreach loop can save you considerable
time, effort and mistakes.
6. Identifying patterns across variables (using the foreach command)
The foreach command can also be used to
identify patterns across variables of a dataset. Let’s say, for example,
that one needs to know which months had income that was less than the income of the previous month. To obtain this information, dummy
indicators can be created to indicate in which months this occurred. Note that only 11 dummy indicators are needed
for a 12 month period because the interest is in the change from one month to
the next. When a month has income that is less
than the income of the previous month, the dummy indicators
lowinc2-lowinc12 get assigned a “1”. When this is not the case,
they are assigned a “0”. This program is illustrated below (note
for simplicity we assume no missing data on income).
foreach curmon of numlist 2/12 {
local lastmon = `curmon' - 1
generate lowinc`curmon' = 1 if ( inc`curmon' < inc`lastmon' )
replace lowinc`curmon' = 0 if ( inc`curmon' >= inc`lastmon' )
}
We can list out the original values of inc and lowinc and
verify that this worked properly
list famid inc1-inc12, clean noobs famid inc1 inc2 inc3 inc4 inc5 inc6 inc7 inc8 inc9 inc10 inc11 inc12 1 3281 3413 3114 2500 2700 3500 3114 3319 3514 1282 2434 2818 2 4042 3084 3108 3150 3800 3100 1531 2914 3819 4124 4274 4471 3 6015 6123 6113 6100 6100 6200 6186 6132 3123 4231 6039 6215list famid lowinc2-lowinc12, clean noobs famid lowinc2 lowinc3 lowinc4 lowinc5 lowinc6 lowinc7 lowinc8 lowinc9 lowinc10 lowinc11 lowinc12 1 0 1 1 0 0 1 0 0 1 0 0 2 1 0 0 0 1 1 0 0 0 0 0 3 0 1 1 0 0 1 1 1 0 0 0
This time we used the foreach loop to compare the current month,
represented by curmon, and the prior month, computed as `curmon’-1
creating lastmon. So, for the first pass through the foreach loop
the value for curmon is 2 and the value for lastmon is 1, so the
generate and replace statements become
generate lowinc2 = 1 if ( inc2 < inc1 ) replace lowinc2 = 0 if ( inc2 >= inc1 )
The process is repeated until curmon is 12, and then the generate
and replace statements become
generate lowinc12 = 1 if ( inc12 < inc11 ) replace lowinc12 = 0 if ( inc12 >= inc11 )
If you were using foreach to span a large range of values (say
1/1000) then it is more effcient to use forvalues since it is designed
to quickly increment through a sequential list, for example
forvalues curmon = 2/12 {
local lastmon = `curmon' - 1
generate lowinc`curmon' = 1 if ( inc`curmon' < inc`lastmon' )
replace lowinc`curmon' = 0 if ( inc`curmon' >= inc`lastmon' )
}