There may be times that you receive a file that has many (or all) of the variables defined as strings, that is, character variables. The variables may contain numeric values, but if they are defined as type string, there are very few things you can do to analyze the data. You cannot get means, you cannot do a regression, you cannot do an ANOVA, etc… Sometimes the dataset contains numerical values that are stored as strings. We will address this scenario first. Then we will address the case where the string variables actually contain strings, and the goal is to assign each value the string takes on to a numeric value.
All of the examples on this page use the same dataset, so let’s start by examining the data. The example dataset, hsbs, is a subset of the High School and Beyond data file with all of the variables as string variables.
use https://stats.idre.ucla.edu/stat/stata/faq/hsbs, clear
As you see from the describe command below, the variables are all defined as string variables (e.g., science is str2, a string of length 2).
describe Contains data from hsbs.dta obs: 20 vars: 6 25 Nov 1999 19:09 size: 400 (99.6% of memory free) ------------------------------------------------------------------------------- 1. id str3 %3s 2. gender str1 %5s 3. race str5 %9s 4. schtyp str3 %5s 5. read str2 %5s 6. science str2 %5s ------------------------------------------------------------------------------- Sorted by:
Now that we know the variables are string variables, we can use the list command to see what the strings stored in these variables look like. Although the variable science is defined as str2, you can see from the list below that it contains just numeric values. Even so, because the variable is defined as str2, Stata cannot perform any kind of numerical analysis of the variable science. The same is true for the variable read.
list
id gender race schtyp read science
1. 70 m 1 pub 45 47
2. 121 f 1 pub 68 63
3. 86 m 1 pub 44 58
4. 141 m 1 pub 63 53
5. 172 m 1 pub 47 53
6. 113 m 1 pub 44 63
7. 50 m 3 pub 50 53
8. 11 m 2 pub 34 39
9. 84 m 1 pub 63 .
10. 48 m 3 pub 57 50
11. 75 m 1 pub 60 53
12. 60 m X pub 57 63
13. 95 m 1 pub 73 61
14. 104 m 1 pub 54 55
15. 38 m 3 pub 45 31
16. 115 m 1 pub 42 50
17. 76 m 1 pub 47 50
18. 195 m 1 pri 57
19. 114 m 1 pub 68 55
20. 85 m 1 pub 55 53
Converting string variables with numeric values
One method of converting numbers stored as strings into numerical variables is to use a string function called real that translates numeric values stored as strings into numeric values Stata can recognize as such. The first line of syntax reads in the dataset shown above. The second generates a new variable read_n that is equal to the value of the number stored in the string variable read. The real(s) is the function that translates the values held as strings, where s is the variable containing strings.
gen read_n = real(read)
list read read_n in 1/5
+---------------+
| read read_n |
|---------------|
1. | 45 45 |
2. | 68 68 |
3. | 44 44 |
4. | 63 63 |
5. | 47 47 |
+---------------+
A second method of achieving the same result is the command destring. Let’s try using the destring command and see how it works. The first line of syntax loads the dataset again, so that we are starting with a dataset containing only string variables again. The second line of syntax runs the destring command.
use https://stats.idre.ucla.edu/stat/stata/faq/hsbs, clear destring , replace id has all characters numeric; replaced as int gender contains non-numeric characters; no replace race contains non-numeric characters; no replace schtyp contains non-numeric characters; no replace read has all characters numeric; replaced as byte science has all characters numeric; replaced as byte (2 missing values generated)
As you can see from the describe command below, the destring command converted all of the variables to numeric, except for race, gender and schtyp. Since these variables had characters in them, the destring command left such variables alone. If there had been any numeric variables in the dataset, they would remain unchanged.
describe
Contains data from https://stats.idre.ucla.edu/stat/stata/faq/hsbs.dta
obs: 20
vars: 6 25 Nov 1999 19:09
size: 340 (98.5% of memory free)
-------------------------------------------------------------------------------
storage display value
variable name type format label variable label
-------------------------------------------------------------------------------
id int %10.0g
gender str1 %5s
race str5 %9s
schtyp str3 %5s
read byte %10.0g
science byte %10.0g
-------------------------------------------------------------------------------
Sorted by:
Note: dataset has changed since last saved
Both of the techniques described above have attributes that in some situations are advantages and in other situations may be disadvantages. The command destring can be run on an entire dataset in one step, the method using the function “real” requires issuing a command for each variable to be converted (although this can be done with a loop rather than typing out the syntax for each variable). One potential advantage to using the function “real” (the first method) is that if the function “real” encounters a non-numeric value, it sets the variable equal to missing in that case and moves on. To some extent destring can be made to behave similarly, but not identically. In order to convert a string variable containing any non-numeric value using destring one must list the characters that should be ignored (e.g. “,” or “.”). Additionally, rather than setting values for those cases containing non-numeric values to missing (what the function “real” does), destring removes the specified non-numeric characters. destring will extract the specified strings and then convert, meaning that “a4” can be converted to “4”. destring‘s behavior is very good if one has numeric values stored as strings that occasionally contain things like commas (e.g. 4,354), but there may be situations where this behavior is undesirable.
Converting string variables with non-numeric values into numeric values
How do we convert gender and schtyp into numeric values? We can use the encode command as shown below. These commands create gender2 and schtyp2.
encode gender, generate(gender2) encode schtyp, generate(schtyp2)
Notice in the describe command below that gender2 and schtyp2 are numeric variables and they have labels associated with them (called gender2 and schtyp2).
describe
Contains data from https://stats.idre.ucla.edu/stat/stata/faq/hsbs.dta
obs: 20
vars: 8 25 Nov 1999 19:09
size: 560 (99.8% of memory free)
-------------------------------------------------------------------------------
storage display value
variable name type format label variable label
-------------------------------------------------------------------------------
id str3 %3s
gender str1 %5s
race str5 %9s
schtyp str3 %5s
read str2 %5s
science str2 %5s
gender2 long %8.0g gender2
schtyp2 long %8.0g schtyp2
-------------------------------------------------------------------------------
Sorted by:
Note: dataset has changed since last saved
If we list out the data, it appears that gender2 and schtyp2 are identical to gender and schtyp, however they are really numeric and what you are seeing are the value labels associated with the variables.
list
id gen~r race sch~p read sci~e gender2 schtyp2
1. 70 m 1 pub 45 47 m pub
2. 121 f 1 pub 68 63 f pub
3. 86 m 1 pub 44 58 m pub
4. 141 m 1 pub 63 53 m pub
5. 172 m 1 pub 47 53 m pub
6. 113 m 1 pub 44 63 m pub
7. 50 m 3 pub 50 53 m pub
8. 11 m 2 pub 34 39 m pub
9. 84 m 1 pub 63 . m pub
10. 48 m 3 pub 57 50 m pub
11. 75 m 1 pub 60 53 m pub
12. 60 m X pub 57 63 m pub
13. 95 m 1 pub 73 61 m pub
14. 104 m 1 pub 54 55 m pub
15. 38 m 3 pub 45 31 m pub
16. 115 m 1 pub 42 50 m pub
17. 76 m 1 pub 47 50 m pub
18. 195 m 1 pri 57 m pri
19. 114 m 1 pub 68 55 m pub
20. 85 m 1 pub 55 53 m pub
Below we use the nolabel option and you see that gender2 and schtyp2 are really numeric.
list, nolabel
id gen~r race sch~p read sci~e gender2 schtyp2
1. 70 m 1 pub 45 47 2 2
2. 121 f 1 pub 68 63 1 2
3. 86 m 1 pub 44 58 2 2
4. 141 m 1 pub 63 53 2 2
5. 172 m 1 pub 47 53 2 2
6. 113 m 1 pub 44 63 2 2
7. 50 m 3 pub 50 53 2 2
8. 11 m 2 pub 34 39 2 2
9. 84 m 1 pub 63 . 2 2
10. 48 m 3 pub 57 50 2 2
11. 75 m 1 pub 60 53 2 2
12. 60 m X pub 57 63 2 2
13. 95 m 1 pub 73 61 2 2
14. 104 m 1 pub 54 55 2 2
15. 38 m 3 pub 45 31 2 2
16. 115 m 1 pub 42 50 2 2
17. 76 m 1 pub 47 50 2 2
18. 195 m 1 pri 57 2 1
19. 114 m 1 pub 68 55 2 2
20. 85 m 1 pub 55 53 2 2
What about the variable race? It is still a character variable because our prior destring command saw the X in the data and did not attempt to convert it because it had non-numeric values. Below we can convert it to numeric by include the ignore(X) option that tells destring to convert the variable to numeric and when it encounters X to convert that to a missing value. You can see the results in the list command below.
destring race, replace ignore(X)
race: characters X removed; replaced as byte
(1 missing value generated)
list
id gen~r race sch~p read sci~e gender2 schtyp2
1. 70 m 1 pub 45 47 m pub
2. 121 f 1 pub 68 63 f pub
3. 86 m 1 pub 44 58 m pub
4. 141 m 1 pub 63 53 m pub
5. 172 m 1 pub 47 53 m pub
6. 113 m 1 pub 44 63 m pub
7. 50 m 3 pub 50 53 m pub
8. 11 m 2 pub 34 39 m pub
9. 84 m 1 pub 63 . m pub
10. 48 m 3 pub 57 50 m pub
11. 75 m 1 pub 60 53 m pub
12. 60 m . pub 57 63 m pub
13. 95 m 1 pub 73 61 m pub
14. 104 m 1 pub 54 55 m pub
15. 38 m 3 pub 45 31 m pub
16. 115 m 1 pub 42 50 m pub
17. 76 m 1 pub 47 50 m pub
18. 195 m 1 pri 57 m pri
19. 114 m 1 pub 68 55 m pub
20. 85 m 1 pub 55 53 m pub
As you have seen, we can use destring to convert string variables that contain numbers into numeric variables, and it can handle situations where some values are stored as a character (like the X we saw with race). If you have a character variable that is stored as all characters, you can use encode to convert the character variable to numeric and it will create value labels that have the values that were stored with the character variable.
Fore more information, see the help or reference manual about the destring and encode commands.