How to import multiple CSV files?

[Maria_Phoonsin]

Hi all,

I just want to know if there's a function in R where I can read/import multiple CSV files and combine them all together in 1 Data frame as all the csv files share the same column name but different structure.

TIA

[GreyMerchant]

Hi there!

It is pretty easy to perform. You essentially just need a list of the files (see list.files) and then do a loop or lapply and then do some sort of do.call where you do a rbind so you can combine all of them.

See a full example here: r - Read and rbind multiple csv files - Stack Overflow

Let me know if you need more help/have any questions

[andresrcs]

With the readr package (>=2.0.0) is as simple as this

library(readr)

list_of_files <- list.files(path = "path/to/your/files",
                            recursive = TRUE,
                            pattern = "\\.csv$",
                            full.names = TRUE)

df <- readr::read_csv(list_of_files, id = "file_name")

[Maria_Phoonsin]

Thank you so much for your help!

A quick question, if the data structure is different, say for example - both tables(different file) share the same column name and information. The only difference is data type/structure. Will R automatically converted this to the same data type? one is integer and the other is character

TIA

[Maria_Phoonsin]

Thank you so much for your help.

Can I ask you what is id = "file_name" in this context? does it mean i have to input all the file names?

TIA

[andresrcs]

@Maria_Phoonsin

does it mean i have to input all the file names?

No, that is just the name for the "ID" column which will hold the name of the file so you can identify what file each row comes from, you can omit that argument in case you don't need to keep track of the origin.

[sateeshkumartk]

Hi,

Just click on down 'import data' button in environment panel or output panel and select read r.

[prettygreenparrot]

That is very elegant. Hadn't expected something that neat.

I can now replace my clunky for loop and just process a single, giant df.

#get list of excel XML files from the directory
#root - user directory 
#client - client directory name
file_list<-list.files(path=str_c(root,client,sep="/"),pattern="*.xlsx")

#function for importing and processing files goes here
#requires client directory name, file name for import
#output is "import_TT" tibble

#Create Output_Table to hold the additions of imported files
Output_Table<-tibble()

#Call import function for each excel workbook to import
#Create output table by concatenating processed tables
for(i in 1:length(file_list)){
  Output_Table<-bind_rows(Output_Table,import_TT(client,file_list[i]))
}

#Export the table to an excel workbook naming it after the first workbook in the list
fileloc<-str_c(root,client,str_c("TT_Processed",file_list[1],today(),".xlsx",sep="-"), sep="/")
write_xlsx(Output_Table,fileloc,format_headers = TRUE,col_names = TRUE)