ReadLines {lazy.utf8}

R Documentation

Read Lines or Wite Lines with or without Encoding Option

Description

Read Lines or Wite Lines with or without Encoding Option

Usage

ReadLines(filename, encoding = "Shift-JIS", asDBCS = 0, ...)

WriteLines(text, filename, encoding = "Shift-JIS", ...)

Arguments

filename	an input text file name
encoding	= "Shift-JIS" | "UTF-8" | NULL where NULL is for system default and should be equal to "native.enc".
asDBCS	= 1 to create double byte characters when encoding="Shift-JIS" is specified.
...	aditional parameters to readLines and writeLines
text	a text vector to be written

Details

Note that, as described in the Arguments section of native readLines, specifying encoding option when calling readLines cannot read lines from a file using the specified encoding.
This function specifies encoding when defining the connection and closes it upon return.

The following code

 con <- file( filename, encoding="Shift-JIS" )
 return( readLines( con, ... )  )

makes the Zenkaku characters 3 byte chracters. Therefore, when asDBCS=1, native readLines will be used.

When writing UTF-8 characters which are not included in native encoding, specify useBytes=TRUE option.
Check to see that Encoding(text) returns "UTF-8", not "unknown".

Above can avoid the following default action of writeLines:

 useBytes is for expert use. Normally (when false) character strings with
 marked encodings are converted to the current encoding before being passed
 to the connection (which might do further re-encoding).
 useBytes = TRUE suppresses the re-encoding of marked strings so they are
 passed byte-by-byte to the connection: this can be useful when strings
 have already been re-encoded by e.g. iconv.
 (It is invoked automatically for strings with marked encoding "bytes".)
 

Value

ReadLines returns a vector of strings.

Examples

# In the examples below, TestData1.txt is encoded in Shift-JIS
# and  TestData2.txt, in UTF-8. The last line of TestData2.txt contains
# non Shif-JIS characters.
#
# These files are stored in the folder where lazy.utf8 package is stored.

# reading Shift-JIS files on from R.4.1.3
nobs <- 10
infile1 <- system.file("TestData1.txt",package="lazy.utf8")
if( regexpr("utf8",Sys.getlocale (category = "LC_CTYPE")) > 0 ){
 temp1 <- ReadLines(infile1, n=nobs+2)[-(1:2)]
} else{
 # The result will be Double Byte.
 temp1 <- ReadLines(infile1, n=nobs+2, asDB=1)[-(1:2)]
 # The result will be Triple Byte.
 temp11 <- ReadLines(infile1, n=nobs+2)[-(1:2)]

 nbyte(temp1)
 nbyte(temp11)
 identical(temp1[-nobs],temp11[-nobs])  # true
 identical(nbyte(temp1[-nobs]),nbyte(temp11[-nobs])) # false
}



# reading UTF-8
nobs <- 10
infile2 <- system.file("TestData2.txt",package="lazy.utf8")
temp2 <- ReadLines(infile2, n=nobs+2, encoding="UTF-8")[-(1:2)]
nbyte(temp2)

identical(temp1[-nobs],temp2[-nobs])  # true
if( regexpr("utf8",Sys.getlocale (category = "LC_CTYPE")) <= 0 ){
 identical(temp11[-nobs],temp2[-nobs])  # true
 identical(nbyte(temp11[-nobs]),nbyte(temp2[-nobs]))  # false
}


# reading UTF-8 with BOM
nobs <- 10
infile3 <- system.file("TestData3.txt",package="lazy.utf8")
temp3 <- ReadLines(infile3, n=nobs+2, encoding="UTF-8")[-(1:2)]
nbyte(temp3)
identical(temp2[-nobs],temp3[-nobs])  # true


# These two works
WriteLines( temp1, "temp11.txt" )
WriteLines( temp1, "temp12.txt", encoding="UTF-8" )

# These two works exept for the last obs which has non Shift-JIS characters.
WriteLines( temp2, "temp21.txt" )
WriteLines( temp2, "temp22.txt", encoding="UTF-8"  )

# This works.
con <- file( "temp222.txt", encoding="native.enc" )
writeLines( temp2, con , useBytes=TRUE )
close(con)


# creating data frame using column specifications
# column spec
from <- c(1, 7,25,26,27,28,29,30,31,32,34)
to <-   c(5,23,25,26,27,28,29,30,31,32,39)
name <- c("id", "name",paste("Q",1:8,sep=""),"comment")

# convert to a character matrix by stacking each row (record, observation)
datamat2 <- Reduce( rbind, substr2(temp2, from,to), init=NULL )
colnames(datamat2) <- name
# matrix to data frame
df2 <- as.data.frame(datamat2)

# Alternative way by stacking each column (variable)
datamat22 <- Reduce( cbind, mapply( function(x,y) substr2(temp2,x,y), from, to
                , SIMPLIFY=FALSE ), init=NULL )
colnames(datamat22)=name
# matrix to data frame
df22 <- as.data.frame(datamat22)

identical(datamat2,datamat22)  # true
identical(df2,df22)  # true

print(datamat2)

# character to numeric conversion
df2[,paste("Q",1:8,sep="")] <- apply(df2[,paste("Q",1:8,sep="")],2
                                    ,as.numeric)
str(df2)

# cannto print UTF-8 data frame nicely by native print.data.frame.
print(df2)
print.listof(df2)

---