Looping folder and sub-folders with txt files

General
,
1

I am new to R. I have several txt. files in several sub-folders under one folder. The structuer of the txt files are the same. I would like to iterate the files for each sub-folder and generate a signle file for the whole folder. I coded as follow:

parent.folder <-"C:/.../18_0101"  # Folder containing sub-folders
sub.folders <- list.dirs(parent.folder, recursive=TRUE)[-1] #
Sub-folders r.scripts <- file.path(sub.folders) HR_2018 <- list() for
(j in seq_along(r.scripts)) {   HR_2018[[j]] <- dir(r.scripts[j],"\\.txt$")}

When I checked HR_2018[[1]] , I found only the list of .txt files under the sub-folder. From there, I would like to analyze the files under each sub-folder. And then I would like to iterate the same process for other sub-folders under the folder and generate a single file. Anyone help me?

2

Hi there,

Here is an example with dummy data and folders to merge all files across different subfolders

#Generate some files and folders
dir.create("test", showWarnings = F)
write("info", "test/file.txt")
dir.create("test/test1", showWarnings = F)
write("info1", "test/test1/file1.txt")
dir.create("test/test1/test1.1", showWarnings = F)
write("info1.1", "test/test1/test1.1/file1.1.txt")
dir.create("test/test2", showWarnings = F)
write("info2", "test/test2/file2.txt")

#Find all files
myFiles = list.files("test", pattern = "\\.txt", recursive = T, full.names = T)


#Combine the files
file.copy(myFiles[1], "test/combined.txt") #Copy the first
for(file in myFiles[-1]){
  file.append("test/combined.txt", file) #Append any other files
}

Hope this helps,
PJ

3

Thanks, @pieterjanvc !
I can follow to find all the files.

In this last step, I don't understand that you combined the files by file.copy(), and append any other files by file.append().

What I wanna do is:
I would like to analyze one sub-folder, and then iterate the other sub-folders using for loop ().

Your idea is that you combined the files at first. In my case, if I combined the files like this, the combined file will be extremely large because of the very large data set even in each txt file.

Could you please help me again?

Thanks in advance.

Heiwa

4

Hi,

If you don't want to append the files beforehand, just replace the code in the for loop with whatever you like to do to individual files (and remove the first copy line). That loop effectively calls every file in all subfolders one by one.

PJ