Best Practices re: Markdown Use with Complex Processing Jobs

Hello. I have been migrating a reasonably complex job process to R that has in the past used SAS. Because the client makes some (usually minor) changes every time the job is processed, I have modularized the code into several sections so any editing that was done to the code was more localized and controlled.

In the R environment, I would like to continue this practice, but I would also like to create Markdown documents for delivery to the client. With that as background, I'd like to pose one very specific question and one more general.

The specific: Should one create a more general "driver" script that is used to define this as an RMarkdown process, and include/source the remaining files in their original R code , or should each of the modular jobs also be written as Markdown jobs?

General Question: I'd like to hear opinions on general best practices for creating and managing a job stream such as this in the world of R?

Thank you in advance.

This is a judgement call. It's hard to know exactly what to advise without seeing your actual code. However, in general, I would advise using R Markdown mainly for the parts of the analysis that you want to present to your client. Other parts of the analysis can be done in R scripts.

There are tons of tricks for re-using code with R Markdown: parameterized reports, child documents, reusable code chunks, and more. Maybe one of these will be helpful for you.

Also, since you mentioned a "driver" script to run this "process", I recommend you take a look at a pipeline management tool like {targets}, which can ensure all the steps have been completed.

This is a huge question :slight_smile: I recently read the blog post Patterns and anti-patterns of data analysis reuse by @milesmcbain. It's full of good ideas of how to manage a code base for repeated data analyses.