Is it possible to save the HTML output in a directory which is not the one where the .Rmd file resides

Andrea · December 13, 2017, 1:05pm

I have an Rstudio project for an R Markdown document, with the following project structure:

project

code\
data\
output\
project.Rproj

The actual code is a bit complicated, but this example should be equivalent:

---
title: "Untitled"
author: "Andrea Panizza"
date: "`r Sys.Date()`"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(readr)
# Set data/output directories
data_dir <- "../data"
output_dir <- "../output"

```

## Read data

```{r read_data}
data_dir <- "../data"
data_filename <- "fakedata.csv"
data_path <- file.path(data_dir, data_filename)
my_data <- read_csv(data_path, col_names = TRUE)
```

## Complex computations

```{r mean, echo=FALSE}
results$average_x <- mean(my_data$x)

```

## Save results

```{r write_data}
output_filename <- ("results.csv")
output_path <- file.path(output_dir, output_filename)
write_csv(results, output_path)

```

As you can see, I write my output files in the output\ directory, which is defined in the first R chunk as

output_dir <- "../output"

and then use file.path to build my path names accordingly. The .Rmd file is in code\ directory, thus the HTML report is saved there. Is this possible to save the HTML output into output\, instead than into code\?

mara · December 13, 2017, 4:20pm

Take a look at the here package. (Jenny has a nice write-up on it here: Ode to the here package).

If you set your project directory (I'd recommend using an .Rproj file to do so— see this Jenny post for more on that) above where you're keeping the Rmd file, you should be able to still set the output directory to whatever path using path = here::here("output", "output_filename") (or some variation thereupon). I wouldn't recommend separating your product documents from their source (I write out results, as you do, to a different folder, too), but it should be possible.

Frank · December 13, 2017, 5:01pm

I use two files...

report.rmd
review.r

... with the second calling rmarkdown::render, which has an output_dir argument, on the first. There's the downside that when working interactively I need to run the second script and can't simply click the RStudio IDE "knit" button when editing the first one.

Interestingly, it looks like @Yihui doesn't endorse this usage, but I guess I'll stick with it until it visibly breaks something.

dylanjm · December 13, 2017, 5:34pm

Would you mind providing small reprex of what you mean? I'm having a hard time understanding what exactly to do. I'm familiar with here::here() maybe a small example .RMD file could help?

Andrea · December 13, 2017, 5:55pm

@mara yes, the project.Rproj file is in the project\ directory, which is one level above the directory code\, where the .Rmd file resides. Probably my first post wasn't clear enough:

project\
- project.Rproj
- code\
  - myreport.Rmd
  - myreport.html
- data\
  - fakedata.csv
- output\
  - results.csv

Hopefully my terrible attempt at representing a folder hierarchy in Markdown clarifies things
I know about here, but I'm not sure how I can use it to tell knitr to put the myreport.html in the output\ folder...can you show me how? I usually use the "Knit" button to render my report, but I can use the CLI if you show me how.

Andrea · December 13, 2017, 6:03pm

@Frank his sounds interesting. However, since I've always used the RStudio "Knit" button to generate my report, I'm not sure how to render it from command line. I guess the review.r should look something like:

library(rmarkdown)
output_dir <- "../output"
render("myreport.Rmd", output_dir = output_dir)

Right? It's annoying to have to define the same variable (output_dir) in two different files (review.r and myreport.Rmd) because surely once I change it in one of them, I'll forget to make the corresponding change in the other one. I guess that if I want to avoid this kind of problem, I'll have to learn about RMarkdown parametric reports....

Andrea · December 13, 2017, 6:22pm

@Frank this is working nicely for me:

review.r file

library(rmarkdown)
output_dir <- "../output"
render("myreport.Rmd", output_dir = output_dir, params = list(output_dir = output_dir))

myreport.Rmd file

---
title: "Untitled"
author: "Andrea Panizza"
date: "`r Sys.Date()`"
output: html_document
params: 
  output_dir: "../output"
---
.
.
.

I understand this is not the preferred approach, but I don't see any other way.

PS just for clarity in case someone else like me is not familar with R Markdown parametric reports the value output_dir: "../output" in the YAML header is just a default. If I change output_dir in review.r I don't need to change it again in the YAML header: the new parameter value is automatically passed to the .Rmd doc.

Frank · December 13, 2017, 7:12pm

Cool. Are you sure the param works there? I haven't tried it myself, and a request for (what seems to be) that functionality got closed as won't-fix.

Yeah, I meant that I only handled the output directory in review.r.

mara · December 13, 2017, 10:42pm

Ah, based on your question, I thought output_dir was something you had figured out, but weren't able to get the correct path there.

I don't know of anything that would allow this to happen by hitting the knit button. The best suggestion I have would be to look into using make files.

http://stat545.com/automation00_index.html

daattali · December 14, 2017, 12:16am

The ezknitr package may do exactly what you want. Look at the motivation and simple use case, I may be wrong but it seems your situation is similar to the exact problem aimed to be solved by that package

Andrea · December 14, 2017, 10:35am

Sh*t! You're right: this doesn't work the way I expected. params is working in the sense that the .Rmd file writes its output files to the correct directory, but the HTML report is still written to the wrong directory! But I think I'm getting there...just you wait

Andrea · December 14, 2017, 11:20am

Ok, now this works and it's fully reproducible. Assume this project directory:

project\
- project.Rproj
- code\
  - render_report.r
  - myreport.Rmd
- output\

Files as follows:

render_report.r file (note the use of file.path() to avoid any dependency on the operative system )

library(rmarkdown)
code_dir <- "code" 
report_filename <- "myreport.Rmd"
report_filename <- file.path(code_dir, report_filename)
output_dir <- "output"
output <- file.path("..",output_dir)
render(report_filename, output_dir = output_dir, params = list(output_dir = output))

myreport.Rmd file

---
title: "myreport"
author: "Andrea Panizza"
date: "`r Sys.Date()`"
output: html_document
params: 
  output_dir: "../foobar"

---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(readr)
# Set output directory
data_dir <- "../data"
output_dir <- params$output_dir

```

## Complex computations

```{r mean, echo=FALSE}
average <- mean(seq(1,10))
results <- data.frame(average = average)

```

## Save results

```{r write_data}
output_filename <- ("results.csv")
output_path <- file.path(output_dir, output_filename)
write_csv(results, output_path)

```

@Frank note that the default value for params$output_dir is now a non-existent directory! So if parameters substitution didn't work, then I would get an error.

After running render_report.r, I get:

project\
- project.Rproj
- code\
  - render_report.r
  - myreport.Rmd
- output\
  - myreport.html
  - results.csv

Phew! It was harder than I thought.
@mara you were right! There's no way to do this with the "Knit" button. However, this solution at least doesn't require me to leave RStudio. Actually, the "Knit" button could place myreport.html in the project\ dir, which is also fine (better than in the code dir, IMO). Still, I have an output dir, why not use it and keep everything nice and tidy ?

davidhen · December 14, 2017, 11:48am

You can use a function to define where the output of of your rendered report goes.

I sometimes use this project workflow by Joris Muller and find it can be really helpful.

The make.reports.R script allows you to choose where the output file goes.

You can also change the output style of the html if you want - an example of my own use of this is here. Note the report_dir = "~\my_reports_folder" argument under the "Main function" heading.

Hope that might help?

Andrea · December 14, 2017, 12:44pm

@daattali cool! I didn't know about your ezknitr. It seems to fit my use case very well. I'm only a bit worried about leaving knitr (and rmarkdown, which depends on it), which are widely adopted, for another package...I'll think about that. Thanks!

Andrea · December 14, 2017, 12:49pm

Wooow! There's even a command to directly open the HTML file in the browser (btw, that would be a useful option for RStudio). At a first look, it seems like a more advanced version of my solution. I'll study that. Thanks!

daattali · December 14, 2017, 4:34pm

It's not leaving knitr and rmarkdown, it uses knitr and rmarkdown (you can
look at the source code), all it does it take care of setting all the
directories correctly. It's very lightweight

Andrea · December 14, 2017, 4:49pm

Ok! I will test it. I hope I didn't sound rude. And thanks for the tip!

daattali · December 14, 2017, 5:02pm

Haha don't worry as a package developer you get a lot of actual rude
feedback, but any feedback is always useful, even just understanding
potential user concerns:)

pavopax · December 15, 2017, 5:38am

This question is also relevant when you are using GitHub pages to publish your HTML output files onto a web page.

With one Github Pages workflow, the HTML files are expected in /docs, but usually, that's not where your Rmd files are.

My current solution to this and related problems is as follows:

knit Rmd to HTML (these files are usually in /notebooks)
write a single command to move HTML and figures files to /docs

For #2, I have Makefile with contents:

pages:
	rm -rf docs/*_files docs/*html
	cp notebooks/*html docs
	cp -r notebooks/*_files docs

Now, from project home directory, I type make pages to clean up and move HTML and figures files to /docs

Also, I have (had a friend write me) a utility unix command to knit Rmd files from command line:

function knit() {
    R -e "rmarkdown::render('$1')"
}

You put this into your .profile or .bashrc file and now from command line you can quickly knit Rmd files like:

knit analysis.Rmd

Andrea · December 15, 2017, 11:50am

I try my best to never be rude to OS developers. I really dislike when people are impolite in GitHub issues. Sometimes it looks like they feel so entitled, instead than grateful as they should be...I hope it doesn't happen often! And thanks again for your package!