How to source or render an Rmd for each element of a list with arguments ?

[carmeyeii]

I have an .Rmd file which contains commands to build a plot. buildConstellation.Rmd (see here).

I typically refer to this file from another .Rmd file, which acts as a notebook for a specific dataset or analysis I may be doing at a given time, i.e., 20210305_neocortex_Constellation.Rmd

Usually, I just call rmarkdown::render and specify the objects that buildConstellation.Rmd takes as params.

rmarkdown::render(input  = './buildConstellation.Rmd',
params = list(my_object = neocortex,
                           out.dir = '../out'))

This works fine because the script buildConstellation.Rmd writes a PDF with the final plot, and all the intermediate tables that it creates are saved to my global environment.

However, I recently had to do a variation of this, where I need to split the object I pass as argument to buildConstellation.Rmd (neocortex) into three parts, based on a variable stage.

neocortex_split_stages <- neocortex %>% split(.$stage)

Once it is split, I can run buildConstellation.Rmd on each of the three subsets of the original object.

neocortex_split_stages$early
neocortex_split_stages$mid
neocortex_split_stages$late

Now, when I pass the newly created list to rmarkdown::render using map, the intermediate tables don't make it to the .GlobalEnv.

split_stages %>% 
map(~ rmarkdown::render(input  = './buildConstellation.Rmd',
           params = list(my_object = . ,
                        out.dir = '../out'))

The final plots get created, but the only way I can recover the intermediate objects is to write them to a file in buildConstellation.Rmd, and then import them back in my .GlobalEnv.

I turned the entirety of buildConstellation.Rmd into a function and saved in a file called buildConstellation.R, which I can source and call, and which returns those intermediate objects.

source("../R/buildConstellation.R")

constellation_result <- split_stages %>% 
   map(~ buildConstellation(my_object = . ,
                            out.dir = '../out' ))

This works, but is ugly. I really would like to know how I can do this using the R markdown file, without having to glob all those pretty chunks and readable text in between into a gross function filled with comments and no chunks.

Your help is greatly appreciated!

[cderv]

You encounter this behavior for several reasons:

Rendering a Rmd document is not like sourcing R script in the way that it won’t be able to return object, like you return clause in you R script. render() will return the path to the rendered file. That is important to understand to see the difference between your example. Using purrr and map() with render will result in a list of result paths.

Also, know that render will by default use and evaluates inside the parent.frame(). See the envir argument in render. When you are not using map(), this is why you see some objects in your global environment that are created inside your Rmd. This is side effects because it renders in the current env by default. This is not always wanted, all the most when you are running several render in a row.

See this for more info on envir argument

17.3 Render R Markdown with rmarkdown::render() | R Markdown Cookbook

When you are using map(), the parent environment of render is different. So objects are not created in the global environnent. You could try hack around this so that your function in map() returns what you want.
But I find writing the result you want in a file that you re read later is a good option - it is often use as side effects

Regarding using part of your Rmd as a R script, it makes sense. You could be interested in purl() to turn some of your chunk in Rmd to a R script, you would still have one Rmd source but it could generate a R file used differently.

3.4 Convert R Markdown to R script | R Markdown Cookbook

You could also have your object creation in a script that you would source in you Rmd document and use independently too. This recipe show you how a script can be use as a chunk content

16.2 Read external scripts into a chunk | R Markdown Cookbook

So I think there are several solution to do this.
If you are is interested in more examples, I can try make some.

Hope it helps.