Render scripts to HTML with varing data input path in each script

[zhengchen]

Hello,

I am using knitr::spin() to run a script and render it to HTML report/log. This script just takes data from a given path, does some model fitting and plots some results. Then I use another short script to run and render the report, instead of clicking the "Compile report" button on RStudio or "Ctrl / Cmd + Shift + K". I find this is a nice way to keep track of my analyses and in the meantime, nothing is so different than just running a script.

rmarkdown::render(script_file, output_format = "html_document", output_dir = output_dir,
                  output_file = output_file)

This works fine until I want to repeat it for many different data inputs. Currently, I just change the data input/output path in the script_file and create a new script for each case, then run and render it by specifying the script_file name in the render script. However, I have more than a hundred different data inputs and I would like to make this process automatic by just giving a list of data paths. There are two options I could think of:

1. write a code to automatically generate 100 scripts in which the difference is just the data input/output path. And then loop across 100 scripts in the render script.
2. have only one script template where the data path is one input argument, and find a way in the render script to loop across all cases.

I would prefer the second way, especially considering if I want to change some visualization or analyses in the script_file I don't have to redo it for all 100 scripts, but not sure how to make it in practice. I can not find where to populate the case-specific path for the script_file in rmarkdown::render. Could you please give me some advice? Or maybe only 1) is the possible solution? If so, do you know any R function/package that could generate a bunch of scripts based on a template and some varying input/output path?

Thanks a lot,
zcai

[mattwarkentin]

Seems like you should consider using parameterized R Markdown reports. You can use a placeholder in the R code chunks for the path to the data file, and then map over the 100 file paths and run the parameterized report.

For example:

---
output: html_document
params:
  file: NULL
---

```{r}
data_file <- params$file

# code from your current R script that now use `data_file`
```

Then you can iterate over the file paths and pass them in as parameters to be used in the report:

purrr::walk(
  vector_of_file_paths,
  ~ rmarkdown::render(rmd_file, output_format = "html_document", output_dir = output_dir, output_file = output_file, params = list(file = .x))
)

You will need to modify the above code to name output_file differently for each parameterized report so they don't overwrite one another, but this should give a starting template.

Chapter 15 Parameterized reports | R Markdown: The Definitive Guide

[zhengchen]

Thanks a lot @mattwarkentin I will look into the details of this bookdown.