How can I write 3 blocks of code to 3 files when the blocks of code and the file names are in different lists

General
1

Hello,

I have a list of lists. The lists in the list of lists are file names. I use lapply to read and merge the contents of each list in the list of lists (3 merged contents in this case which will be the content of 3 files). Then, I have to change the name of the 3 resulting files and finally I have to write the contents of the files to each file. In this case the files I am reading and merging are the repeated .txt file which only contains the data a b c d e f separated by tabs.

 lc <- list("test.txt", "test.txt", "test.txt", "test.txt")
 lc1 <- list("test.txt", "test.txt", "test.txt")
 lc2 <- list("test.txt", "test.txt")
#list of lists.  The lists contain file names
 lc <- list(lc, lc1, lc2) 
#new names for the three lists in the list of lists
 new_dataFns <- list("name1", "name2", "name3")
 file_paths <- NULL
 new_path <- NULL
#add the file names to the path and read and merge the contents of each list in the list of lists
 lapply(
    lc, 
    function(lc) {
     filenames <- file.path(dataFnsDir, lc)
     dataList= lapply(filenames, function (x) read.table(file=x, header=TRUE))
     Reduce(function(x,y) merge(x,y), dataList)
     #   print(dataList)

    }
  )

#add the new name of the file to the path total will be 3 paths/fille_newname.tsv.

 lapply(new_path, function(new_path){new_path <- file.path(getwd(), new_dataFns)

The statements above work because lc and new_dataFns are global and I can pass them to the lapply function

#Finally, I need to write the merged contents to the corresponding file (p\ath/name.tsv). I tried the following statement, but this does not work. How can I write the content to each file? I was trying to use list <- cbind(dataList, new_path) so that afterwards I can get the merged contents and the file_name from the list and that way write each merged content to the corresponding file, but it seems that the dataList and the newPath are not global and the cbind() function does not work.

2

I'm not sure I understand exactly what you're trying to do, and unfortunately the code example is not easily reproducible since it involves reading in files that aren't created by the code. Could you please turn this question into a reprex?

I'm especially unsure about what you mean by "merging" the files in the sublists. The code is using merge(), but if all the files have the same variables, this doesn't make much sense. Do you maybe mean instead that you want to row-bind the imported data frames (like stacking them on top of each other)?

Here's one example of how you might do what I think you're trying to do. It's very possible that I've misunderstood your problem, so the details of how you want to combine the files in the sublists might be wrong. Still, the general approach should work even if those details change. This is also an example of how you can make a reprex that involves reading and writing files.

# Setup example files and file lists
in_dir <- paste0(tempdir(), "/in_files")
out_dir <- paste0(tempdir(), "/out_files")

dir.create(in_dir)
dir.create(out_dir)

for (x in 1:9) {
  write.table(
    warpbreaks[(x * 3):((x * 3) + 2), ], 
    file = paste0(file.path(in_dir, "warpbreaks_"), x, ".txt"),
    sep = "\t",
    quote = FALSE,
    row.names = FALSE
  )
}

file_list1 <- as.list(paste0("warpbreaks_", 1:4, ".txt"))
file_list2 <- as.list(paste0("warpbreaks_", 5:7, ".txt"))
file_list3 <- as.list(paste0("warpbreaks_", 8:9, ".txt"))

file_lists <- list(file_list1, file_list2, file_list3)
file_lists
#> [[1]]
#> [[1]][[1]]
#> [1] "warpbreaks_1.txt"
#> 
#> [[1]][[2]]
#> [1] "warpbreaks_2.txt"
#> 
#> [[1]][[3]]
#> [1] "warpbreaks_3.txt"
#> 
#> [[1]][[4]]
#> [1] "warpbreaks_4.txt"
#> 
#> 
#> [[2]]
#> [[2]][[1]]
#> [1] "warpbreaks_5.txt"
#> 
#> [[2]][[2]]
#> [1] "warpbreaks_6.txt"
#> 
#> [[2]][[3]]
#> [1] "warpbreaks_7.txt"
#> 
#> 
#> [[3]]
#> [[3]][[1]]
#> [1] "warpbreaks_8.txt"
#> 
#> [[3]][[2]]
#> [1] "warpbreaks_9.txt"


# Import and bind together files in each sublist, then output with new names
new_names <- c("name1.txt", "name2.txt", "name3.txt")

new_dfs <- lapply(
  file_lists,
  function(lst) {
    df_lst <- lapply(
      lst,
      function(fl) { read.table(file.path(in_dir, fl), header = TRUE) }
    )
   do.call(what = rbind, args = df_lst)
  }
)

new_dfs
#> [[1]]
#>    breaks wool tension
#> 1      54    A       L
#> 2      25    A       L
#> 3      70    A       L
#> 4      52    A       L
#> 5      51    A       L
#> 6      26    A       L
#> 7      67    A       L
#> 8      18    A       M
#> 9      21    A       M
#> 10     29    A       M
#> 11     17    A       M
#> 12     12    A       M
#> 
#> [[2]]
#>   breaks wool tension
#> 1     18    A       M
#> 2     35    A       M
#> 3     30    A       M
#> 4     36    A       M
#> 5     36    A       H
#> 6     21    A       H
#> 7     24    A       H
#> 8     18    A       H
#> 9     10    A       H
#> 
#> [[3]]
#>   breaks wool tension
#> 1     43    A       H
#> 2     28    A       H
#> 3     15    A       H
#> 4     26    A       H
#> 5     27    B       L
#> 6     14    B       L

for (index in seq_along(new_dfs)) {
  write.table(
    new_dfs[[index]], 
    file = file.path(out_dir, new_names[[index]]), 
    sep = "\t", quote = FALSE, row.names = FALSE
  )
}

list.files(out_dir, full.names = TRUE)
#> [1] "/tmp/RtmpxLmJym/out_files/name1.txt"
#> [2] "/tmp/RtmpxLmJym/out_files/name2.txt"
#> [3] "/tmp/RtmpxLmJym/out_files/name3.txt"

# `reprex()` can't run this line, but it can be run by hand
if(interactive()) {
  file.show(list.files(out_dir, full.names = TRUE))
}

Created on 2018-08-21 by the reprex package (v0.2.0).

3 Likes
3

Thank you very much for your response. I will work on a reprex.