How to Read Huge Files With R

If you have 4GB of RAM you should easily be able to work with 1GB of data if you use data.table properly.

One thing which may help is avoiding the temp stage:

my_data <- rbindlist(lapply(files, fread))

Also, try to avoid using "data" as a variable name.

3 Likes