Creating pipeline for .Rmd Files

[ksingh19]

Hi All,

I like working in .Rmd files to have better understanding of data.
However, when it comes to creating a pipeline, I am not able to figure out as to how .Rmd files can be picked up from source like we can do for .R files.

# For Example, with all ".R" files,
# I can just have a "pipeline.R" file which contains the following
               source("Src/File1.R")
               source("Src/File2.R")

When I run "pipeline.R" file, both the files within it are run in sequential order.

However, I am not able to do the same thing with .Rmd files. How can I create a pipeline to run all .Rmd files in sequential order?

Thanks!

[williaml]

Does this work for you?

Pipeline to run all RMarkdown Files in sequence - R Markdown - Posit Community (rstudio.com)

[ksingh19]

Thanks @williaml

Yes, it does work to some extent!!

Outputs are as expected. However, running the below mentioned code brings duplicates of .Rmd files in the form of .R files in the source folder. So, now we have 2 copies of each - one with original format in .Rmd and another as a new file in format of .R file

How can we avoid this duplication if we consider the following:

library(knitr)
source(knitr::purl("file_one.Rmd"))
source(knitr::purl("file_two.Rmd"))

Thanks again!

[williaml]

One option could be just to remove the copies.

You can even do it using your R script if you wanted, e.g. using some of these: Comparison of fs functions, base R, and shell commands • fs (r-lib.org)

[ksingh19]

Thanks @williaml
I tried using the following and it deleted .R and .Rmd Files, basically all forms of R files. We would need to retain those .Rmd files insated.

mydir <- "/folder/test_files"
files_to_delete <- dir(path = mydir ,pattern = ".R")
file.remove(file.path(mydir, files_to_delete))

[williaml]

You can change the pattern. You might need to look up some regex to do this.

[Rootsyl]

I am confused as to how you are planning on using rmds as source. Source is a file that contains code parts that can be used later, rmds on the other hand are a way of interacting with code. A fileception. I would probably stay far far away from using rmds as source as many packages will raise errors. Its just not worth it. Just copy paste the code you write to a R file and be happy.

[ksingh19]

Thanks @Rootsyl !
I use Rmds over R because its well structured and much more easier to follow along with documentation. Yes, R is the way to go if we can't have any solution to create pipeline , but I would be thrilled to have an option that can take all my Rmd files as the source file to create an automated pipeline.

If anyone, can solve this issue, it would be greatly appreciated. Thanks all!