Version info: Code for this page was tested in R version 3.0.2 (2013-09-25)
On: 2013-11-19
With: lattice 0.20-24; foreign 0.8-57; knitr 1.5
1. Reading in data from the console using the scan function
For very small data vectors it is sometimes handy to read in data directly from the
prompt. This can be accomplished using the scan function from the
command line. The scan function reads the fields of
data in the file as specified by the what option, with the default being numeric. If
the what option is specified to be what=character() or what=” ” then all the
fields will be read as strings. If the data are a mix of numeric, string or complex data, then
a list can be used in the what option. The default separator for the scan
function is any white space (single space, tab, or new line). Because the default is space delimiting, you
can enter data on separate lines. When all the data have been entered,
just hit the enter key twice which will terminate the scanning.
# Reading in numeric data > x <- scan() 1: 3 5 6 4: 3 5 78 29 8: 34 5 1 78 12: Read 11 items > x [1] 3 5 6 3 5 78 29 34 5 1 78 > mode(x) [1] "numeric" # Reading in string data # empty quotes indicates character input > y <- scan(what=" ") 1: red blue 3: green red 5: blue yellow 7: Read 6 items > y [1] "red" "blue" "green" "red" "blue" "yellow" > mode(y) [1] "character"
2. Importing data files using the scan function
The scan function is an extremely flexible tool for
importing data. Unlike the read.table
function, however, which returns a data frame, the scan function returns a list or
a vector. This makes the scan function less useful for inputting
“rectangular” data such as the car data set that will been seen in later examples.
In the previous example we input first numeric data and then string data directly from the console;
in the following example, we input a text file. For the what
option, we use list and then list the variables, and after each variable, we
tell R what type of variable (e.g., numeric, string) it is. In the first
example, the first variable is age, and we tell R that age is a
numeric variable by setting it equal to 0. The second variable is called
name, and it is denoted as a string variable by the empty quote marks.
In the second example, we list NULL first, indicating that we do not want the
first variable to be read. After using the scan function, we use
the sapply function, which makes a list out of a vector of names in x.
# inputting a text file and outputting a list (x <- scan("https://stats.idre.ucla.edu/stat/data/scan.txt", what = list(age = 0, name = "")))
## $age ## [1] 12 24 35 20 ## ## $name ## [1] "bobby" "kate" "david" "michael"
# using the same text file and saving only the names as a vector x <- scan("https://stats.idre.ucla.edu/stat/data/scan.txt", what = list(NULL, name = character())) (x <- x[sapply(x, length) > 0])
## $name ## [1] "bobby" "kate" "david" "michael"
is.vector(x)
## [1] TRUE
3. Reading in free formatted data from an ASCII file using the read.table function
The read.table function will let you read in any type of delimited ASCII file. It can read in
both numeric and character values. The default is for it to read in everything as numeric data, and
character data is read in as numeric, it is easiest to change that once the data has been read in using the mode
function. This is by far the easiest and most reliable method of entering data into R.
# complete data, space delimited, variable names in first row (test <- read.table("https://stats.idre.ucla.edu/stat/data/test.txt", header = TRUE))
## prgtype gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
The default delimiter in read.table is the space delimiter, but this
could create problems if there are missing data. The function will not work unless every
data line has the same number of values. Thus, if there are missing data, the data lines
will have different number of values, and you will receive an error. If there are missing
values the easiest way to fix this problem is to change the type of delimiter.
In the read.table function the sep argument is used to specify the delimiter.
# showing the file with missing values, space delimited (test_missing.txt data file) prgtype gender id ses schtyp level general 0 70 4 1 1 vocati 1 121 4 1 general 0 86 1 vocati 0 141 4 3 1 academic 0 172 4 2 1 academic 0 113 4 2 1 general 0 50 3 2 1 academic 0 11 1 2 1
test.missing <- read.table("https://stats.idre.ucla.edu/stat/data/test_missing.txt", header = TRUE)
## Error: line 2 did not have 6 elements
# showing the file with missing data, comma delimited (test_missing_comma.txt data file) prgtype, gender, id, ses, schtyp, level general, 0, 70, 4, 1, 1 vocati, 1, 121, 4, , 1 general, 0, 86, , , 1 vocati, 0, 141, 4, 3, 1 academic, 0, 172, 4, 2, 1 academic, 0, 113, 4, 2, 1 general, 0, 50, 3, 2, 1 academic, 0, 11, 1, 2, 1
(test.missing <- read.table("https://stats.idre.ucla.edu/stat/data/test_missing_comma.txt", header = TRUE, sep = ","))
## prgtype gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 NA 1 ## 3 general 0 86 NA NA 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
The read.table function is very useful when reading in ASCII files
that contain rectangular data. As mentioned above, the default delimiter is blank space;
other delimiters must be specified by using the sep option and setting it equal to
the delimiter in quotes (i.e., sep=”;” for the semicolon delimited data file).
Another very common type of file is the comma delimited file.
The next file has been saved out of Excel as a comma delimited file.
This file can be read in by the read.table function by using the sep option,
but it can also be read in by the read.csv function which was written specifically for
comma delimited files. We use the print function to display the
contents of the object test.csv just to show its use.
test.csv <- read.csv("https://stats.idre.ucla.edu/stat/data/test.csv", header = TRUE) print(test.csv)
## prgtype gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
test.csv1 <- read.table("https://stats.idre.ucla.edu/stat/data/test.csv", header = TRUE, sep = ",") print(test.csv1)
## prgtype gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
It is, of course, also possible to use the read.table function for
reading in files with other delimiters. The next file has semicolon delimiters, followed by
a dataset that uses the letter “z” as a delimiter,
both of which are acceptable delimiters in R.
test.semi <- read.table("https://stats.idre.ucla.edu/stat/data/testsemicolon.txt", header = TRUE, sep = ";") print(test.semi)
## prgtype gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
test.z <- read.table("https://stats.idre.ucla.edu/stat/data/testz.txt", header = TRUE, sep = "z") print(test.z)
## prgtype gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
4. Reading in fixed formatted files
We use the read.fwf function to read in data with fixed
formats, and we use the width argument to indicate the width (number of columns)
of each variable. In a fixed format file we do not have the names of the variables on
the first line, and therefore they must be added after we have read in the data. We add the
variable names using the dimnames function and the bracket notation to indicate that
we are attaching names to the variables (columns) of the data file. Please
note that there are several different ways to accomplish this task; this is just one of them.
test.fixed <- read.fwf("https://stats.idre.ucla.edu/stat/data/test_fixed.txt", width = c(8, 1, 3, 1, 1, 1)) dimnames(test.fixed)[[2]] <- c("prgtyp", "gender", "id", "ses", "schtyp", "level") test.fixed
## prgtyp gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
For fixed format files the variables names are often in a separate file from the data.
In this example the variable names are in a file called “names.txt” and the
data are again in “test_fixed.txt”. This is especially
convenient when the fixed format file is very large and has many variables; then it becomes
rather impractical to type in all the variable names. In this
situation the width option is used to specify the width of each
variable and the col.name option specifies the file containing the
variable names. So, first we read in the file for the names using the
scan function. We specify that file contains character values by
setting the what option to equal character(). By using the
col.names option in the read.fwf function, the object names will supply
the variables names.
(names <- scan("https://stats.idre.ucla.edu/stat/data/names.txt", what = character()))
## [1] "prgtyp" "gender" "id" "ses" "schtyp" "level"
(test.fixed <- read.fwf("https://stats.idre.ucla.edu/stat/data/test_fixed.txt", col.names = names, width = c(8, 1, 3, 1, 1, 1)))
## prgtyp gender id ses schtyp level ## 1 general 0 70 4 1 1 ## 2 vocati 1 121 4 2 1 ## 3 general 0 86 4 3 1 ## 4 vocati 0 141 4 3 1 ## 5 academic 0 172 4 2 1 ## 6 academic 0 113 4 2 1 ## 7 general 0 50 3 2 1 ## 8 academic 0 11 1 2 1
5. Exporting files using the write.table function
The write.table function outputs data files. The first argument specifies which
data frame in R is to be exported. The next argument specifies the file to be created.
The default separator is a blank space but any separator can be specified in the sep option.
The default value for both the row.names and col.names options is TRUE. In the example we specify
that we do not wish to include row names. The default setting for the quote option is to include
quotes around all the character values, i.e., around values in string variables and around the
column names. As we have shown in the example it is very common not to want the quotes when
creating a text file.
# using the test.csv data frame to write a text file with no row names and # without quotes around the character values (both column names and string # variables) write.table(test.csv, file.path(tempdir(), "test1.txt"), row.names = FALSE, quote = FALSE)
6. Exporting files in Stata 6/7/8/10 format using the write.dta function
The write.dta function is part of the foreign package and writes an
R data frame to a Stata data file in either Stata 6, 7, 8, or 10 format.
Although these are older versions of Stata, Stata has no difficulty reading files written
in older versions. It takes at least two arguments, the first one
being the data frame and the second one being the output Stata data file name.
If you look at the help file for write.dta, you will see
that the function writes out a Stata 7 data file, but there are comments and
options for those using later versions of Stata. In the example below, we use the anscombe
data set that comes with R. It happens that the anscombe data is already a data frame,
this being checked with the is.data.frame function.
library(foreign) data(anscombe) is.data.frame(anscombe)
## [1] TRUE
anscombe
## x1 x2 x3 x4 y1 y2 y3 y4 ## 1 10 10 10 8 8.04 9.14 7.46 6.58 ## 2 8 8 8 8 6.95 8.14 6.77 5.76 ## 3 13 13 13 8 7.58 8.74 12.74 7.71 ## 4 9 9 9 8 8.81 8.77 7.11 8.84 ## 5 11 11 11 8 8.33 9.26 7.81 8.47 ## 6 14 14 14 8 9.96 8.10 8.84 7.04 ## 7 6 6 6 8 7.24 6.13 6.08 5.25 ## 8 4 4 4 19 4.26 3.10 5.39 12.50 ## 9 12 12 12 8 10.84 9.13 8.15 5.56 ## 10 7 7 7 8 4.82 7.26 6.42 7.91 ## 11 5 5 5 8 5.68 4.74 5.73 6.89
write.dta(anscombe, file = file.path(tempdir(), "anscombe.dta"))
Now let’s see an example where the data is not yet a data frame. We can
use function as.data.frame to convert the data into a data frame.
Again, these data come with R.
data(WorldPhones) is.data.frame(WorldPhones)
## [1] FALSE
WorldPhones
## N.Amer Europe Asia S.Amer Oceania Africa Mid.Amer ## 1951 45939 21574 2876 1815 1646 89 555 ## 1956 60423 29990 4708 2568 2366 1411 733 ## 1957 64721 32510 5230 2695 2526 1546 773 ## 1958 68484 35218 6662 2845 2691 1663 836 ## 1959 71799 37598 6856 3000 2868 1769 911 ## 1960 76036 40341 8220 3145 3054 1905 1008 ## 1961 79831 43173 9053 3338 3224 2005 1076
(phones_d <- as.data.frame(WorldPhones))
## N.Amer Europe Asia S.Amer Oceania Africa Mid.Amer ## 1951 45939 21574 2876 1815 1646 89 555 ## 1956 60423 29990 4708 2568 2366 1411 733 ## 1957 64721 32510 5230 2695 2526 1546 773 ## 1958 68484 35218 6662 2845 2691 1663 836 ## 1959 71799 37598 6856 3000 2868 1769 911 ## 1960 76036 40341 8220 3145 3054 1905 1008 ## 1961 79831 43173 9053 3338 3224 2005 1076
is.data.frame(phones_d)
## [1] TRUE
write.dta(phones_d, file = file.path(tempdir(), "phones.dta"))
To give you an idea of what types of data can be read into R using the
foreign package, the help file is shown below.
data.restore Read an S3 Binary or data.dump File
lookup.xport Lookup Information on a SAS XPORT Format
Library
read.arff Read Data from ARFF Files
read.dbf Read a DBF File
read.dta Read Stata Binary Files
read.epiinfo Read Epi Info Data Files
read.mtp Read a Minitab Portable Worksheet
read.octave Read Octave Text Data Files
read.spss Read an SPSS Data File
read.ssd Obtain a Data Frame from a SAS Permanent
Dataset, via read.xport
read.systat Obtain a Data Frame from a Systat File
read.xport Read a SAS XPORT Format Library
write.arff Write Data into ARFF Files
write.dbf Write a DBF File
write.dta Write Files in Stata Binary Format
write.foreign Write Text Files and Code to Read Them