Is it possible to save the HTML output in a directory which is not the one where the .Rmd file resides

I have an Rstudio project for an R Markdown document, with the following project structure:

project


code\
data\
output\
project.Rproj

The actual code is a bit complicated, but this example should be equivalent:

---
title: "Untitled"
author: "Andrea Panizza"
date: "`r Sys.Date()`"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(readr)
# Set data/output directories
data_dir <- "../data"
output_dir <- "../output"

```

## Read data

```{r read_data}
data_dir <- "../data"
data_filename <- "fakedata.csv"
data_path <- file.path(data_dir, data_filename)
my_data <- read_csv(data_path, col_names = TRUE)
```

## Complex computations

```{r mean, echo=FALSE}
results$average_x <- mean(my_data$x)

```

## Save results

```{r write_data}
output_filename <- ("results.csv")
output_path <- file.path(output_dir, output_filename)
write_csv(results, output_path)

``` 

As you can see, I write my output files in the output\ directory, which is defined in the first R chunk as

output_dir <- "../output"

and then use file.path to build my path names accordingly. The .Rmd file is in code\ directory, thus the HTML report is saved there. Is this possible to save the HTML output into output\, instead than into code\?

1 Like

Take a look at the here package. (Jenny has a nice write-up on it here: Ode to the here package).

If you set your project directory (I'd recommend using an .Rproj file to do so— see this Jenny post for more on that) above where you're keeping the Rmd file, you should be able to still set the output directory to whatever path using path = here::here("output", "output_filename") (or some variation thereupon). I wouldn't recommend separating your product documents from their source (I write out results, as you do, to a different folder, too), but it should be possible.

4 Likes

I use two files...

  • report.rmd
  • review.r

... with the second calling rmarkdown::render, which has an output_dir argument, on the first. There's the downside that when working interactively I need to run the second script and can't simply click the RStudio IDE "knit" button when editing the first one.

Interestingly, it looks like @Yihui doesn't endorse this usage, but I guess I'll stick with it until it visibly breaks something.

2 Likes

Would you mind providing small reprex of what you mean? I'm having a hard time understanding what exactly to do. I'm familiar with here::here() maybe a small example .RMD file could help?

@mara yes, the project.Rproj file is in the project\ directory, which is one level above the directory code\, where the .Rmd file resides. Probably my first post wasn't clear enough:

  • project\
    • project.Rproj
    • code\
      • myreport.Rmd
      • myreport.html
    • data\
      • fakedata.csv
    • output\
      • results.csv

Hopefully my terrible attempt at representing a folder hierarchy in Markdown clarifies things :slight_smile:
I know about here, but I'm not sure how I can use it to tell knitr to put the myreport.html in the output\ folder...can you show me how? I usually use the "Knit" button to render my report, but I can use the CLI if you show me how.

@Frank his sounds interesting. However, since I've always used the RStudio "Knit" button to generate my report, I'm not sure how to render it from command line. I guess the review.r should look something like:

library(rmarkdown)
output_dir <- "../output"
render("myreport.Rmd", output_dir = output_dir)

Right? It's annoying to have to define the same variable (output_dir) in two different files (review.r and myreport.Rmd) because surely once I change it in one of them, I'll forget to make the corresponding change in the other one. I guess that if I want to avoid this kind of problem, I'll have to learn about RMarkdown parametric reports....

2 Likes

@Frank this is working nicely for me:

review.r file

library(rmarkdown)
output_dir <- "../output"
render("myreport.Rmd", output_dir = output_dir, params = list(output_dir = output_dir))

myreport.Rmd file

---
title: "Untitled"
author: "Andrea Panizza"
date: "`r Sys.Date()`"
output: html_document
params: 
  output_dir: "../output"
---
.
.
.

I understand this is not the preferred approach, but I don't see any other way.

PS just for clarity in case someone else like me is not familar with R Markdown parametric reports :slight_smile: the value output_dir: "../output" in the YAML header is just a default. If I change output_dir in review.r I don't need to change it again in the YAML header: the new parameter value is automatically passed to the .Rmd doc.

1 Like

Cool. Are you sure the param works there? I haven't tried it myself, and a request for (what seems to be) that functionality got closed as won't-fix.

Yeah, I meant that I only handled the output directory in review.r.

1 Like

Ah, based on your question, I thought output_dir was something you had figured out, but weren't able to get the correct path there.

I don't know of anything that would allow this to happen by hitting the knit button. The best suggestion I have would be to look into using make files.

http://stat545.com/automation00_index.html

1 Like

The ezknitr package may do exactly what you want. Look at the motivation and simple use case, I may be wrong but it seems your situation is similar to the exact problem aimed to be solved by that package

4 Likes

Sh*t! You're right: this doesn't work the way I expected. params is working in the sense that the .Rmd file writes its output files to the correct directory, but the HTML report is still written to the wrong directory! But I think I'm getting there...just you wait :wink:

Ok, now this works and it's fully reproducible. Assume this project directory:

  • project\
    • project.Rproj
    • code\
      • render_report.r
      • myreport.Rmd
    • output\

Files as follows:

render_report.r file (note the use of file.path() to avoid any dependency on the operative system :wink: )

library(rmarkdown)
code_dir <- "code" 
report_filename <- "myreport.Rmd"
report_filename <- file.path(code_dir, report_filename)
output_dir <- "output"
output <- file.path("..",output_dir)
render(report_filename, output_dir = output_dir, params = list(output_dir = output))

myreport.Rmd file

---
title: "myreport"
author: "Andrea Panizza"
date: "`r Sys.Date()`"
output: html_document
params: 
  output_dir: "../foobar"

---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(readr)
# Set output directory
data_dir <- "../data"
output_dir <- params$output_dir

```

## Complex computations

```{r mean, echo=FALSE}
average <- mean(seq(1,10))
results <- data.frame(average = average)

```

## Save results

```{r write_data}
output_filename <- ("results.csv")
output_path <- file.path(output_dir, output_filename)
write_csv(results, output_path)

``` 

@Frank note that the default value for params$output_dir is now a non-existent directory! So if parameters substitution didn't work, then I would get an error.

After running render_report.r, I get:

  • project\
    • project.Rproj
    • code\
      • render_report.r
      • myreport.Rmd
    • output\
      • myreport.html
      • results.csv

Phew! It was harder than I thought.
@mara you were right! There's no way to do this with the "Knit" button. However, this solution at least doesn't require me to leave RStudio. Actually, the "Knit" button could place myreport.html in the project\ dir, which is also fine (better than in the code dir, IMO). Still, I have an output dir, why not use it and keep everything nice and tidy :slight_smile: ?

3 Likes

You can use a function to define where the output of of your rendered report goes.

I sometimes use this project workflow by Joris Muller and find it can be really helpful.

The make.reports.R script allows you to choose where the output file goes.

You can also change the output style of the html if you want - an example of my own use of this is here. Note the report_dir = "~\my_reports_folder" argument under the "Main function" heading.

Hope that might help?

1 Like

@daattali cool! I didn't know about your ezknitr. It seems to fit my use case very well. I'm only a bit worried about leaving knitr (and rmarkdown, which depends on it), which are widely adopted, for another package...I'll think about that. Thanks!

Wooow! There's even a command to directly open the HTML file in the browser (btw, that would be a useful option for RStudio). At a first look, it seems like a more advanced version of my solution. I'll study that. Thanks!

1 Like

It's not leaving knitr and rmarkdown, it uses knitr and rmarkdown (you can
look at the source code), all it does it take care of setting all the
directories correctly. It's very lightweight

1 Like

Ok! I will test it. I hope I didn't sound rude. And thanks for the tip!

Haha don't worry as a package developer you get a lot of actual rude
feedback, but any feedback is always useful, even just understanding
potential user concerns:)

1 Like

This question is also relevant when you are using GitHub pages to publish your HTML output files onto a web page.

With one Github Pages workflow, the HTML files are expected in /docs, but usually, that's not where your Rmd files are.

My current solution to this and related problems is as follows:

  1. knit Rmd to HTML (these files are usually in /notebooks)
  2. write a single command to move HTML and figures files to /docs

For #2, I have Makefile with contents:

pages:
	rm -rf docs/*_files docs/*html
	cp notebooks/*html docs
	cp -r notebooks/*_files docs

Now, from project home directory, I type make pages to clean up and move HTML and figures files to /docs

Also, I have (had a friend write me) a utility unix command to knit Rmd files from command line:

function knit() {
    R -e "rmarkdown::render('$1')"
}

You put this into your .profile or .bashrc file and now from command line you can quickly knit Rmd files like:

knit analysis.Rmd

3 Likes

I try my best to never be rude to OS developers. I really dislike when people are impolite in GitHub issues. Sometimes it looks like they feel so entitled, instead than grateful as they should be...I hope it doesn't happen often! And thanks again for your package!