render_book error "stop_if_not_exists" and "some files were not found"

Publishing Your Results
1

Hi all,

This is my first post to this community so I hope I get the format of my question correct.

I have successfully in the past used the render_book command in an Rscript, but today this script is throwing an error about the input file not being found.

For example, the name of the bookdown file I am working on is BasicWebsite.Rmd which lives in a source-posts folder located at home/source-posts

I have an R-Script that lives in the folder home and is shown below:

file_to_render <- "BasicWebsite"
library(bookdown)
render_book(
input = "./source-posts",
output_format = c("bookdown::html_book"),
output_dir = "../output/posts",
output_file = paste0(file_to_render),
envir = parent.frame(),
output_yaml= "_output.yml",
config_file= "_bookdown.yml",
quiet = FALSE,
encoding = "UTF-8"
)

I understand for bookdown that I need the files index.Rmd, _bookdown.yml, and _output.yml which are copied here respectively as

title: ""
site: bookdown::bookdown_site
link-citations: yes

and

rmd_files: "BasicWebsite.Rmd"
delete_merged_file: yes

and

bookdown::html_book:
css: styles.css
split_by: "rmd"
theme: cosmo
number_sections: no
toc: no

(P.S. I am using html_book rather than html_document2 as html_document2 does not seem to respect writing to the output_dir location specified in the render_book script)

Every time in the past I have run this render_book script an .html file is generated in an output folder specifically at home/output/posts/BasicWebsite.html

But today I get the error message:

render_book(
input = "./source-posts",
output_format = c("bookdown::html_book"),
output_dir = "../output/posts",
output_file = paste .... [TRUNCATED]
Error in stop_if_not_exists(input) :
Some files were not found: ./source-posts

I do not understand where I have gone wrong and why render_book is not finding the input file (BasicWebsite.Rmd) particularly when this script has behaved as expected so many times before.

My sessionInfo is

sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin21.3.0 (64-bit)
Running under: macOS Monterey 12.4
other attached packages:
[1] bookdown_0.27
loaded via a namespace (and not attached):
[1] compiler_4.2.0 fastmap_1.1.0 htmltools_0.5.2 tools_4.2.0 yaml_2.3.5
[6] rmarkdown_2.14 knitr_1.39 digest_0.6.29 xfun_0.31 rlang_1.0.2
[11] evaluate_0.15

2

BTW, I can use the Knit button in RStudio on the file BasicWebsite.Rmd and an BasicWebsite.html file is created in home/source-posts/_book/BasicWebsite.html so there is nothing wrong with the .Rmd file itself.

3

BTW, I do not get the error running the same code (with the same file BasicWebsite.Rmd, and render_book script) in RStudio on my MacBook Pro Laptop:

sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14.6
other attached packages:
[1] bookdown_0.26
loaded via a namespace (and not attached):
[1] digest_0.6.29 R6_2.5.1 jsonlite_1.8.0 magrittr_2.0.3 evaluate_0.15 highr_0.9
[7] stringi_1.7.6 rlang_1.0.2 jquerylib_0.1.4 bslib_0.3.1 rmarkdown_2.13 tools_4.1.3
[13] stringr_1.4.0 xfun_0.30 yaml_2.3.5 fastmap_1.1.0 compiler_4.1.3 htmltools_0.5.2
[19] knitr_1.38 sass_0.4.1

Perhaps a bug has been introduced between bookdown_0.26 and 0.27?

4

Yes there was a change in 0.27 to fix an issue found. We added a check that the input provided in render_book() does exists.

Can you tell me what this give you ?

bookdown:::dir_exists("./source-posts")

This is the folder you are passing in render_book()

thanks

Calling render_book() with so many path parameter can have side effect. Paths handling is quite complex and often it is a very good idea (at least a recommandation) to do the moving part of result in a another step

Like

  • your book is in source-docs
  • You render from that folder and it will output in output_dir, by default _book
  • You move the whole folder _book where you need using another command after render_book()

I am not quite sure why you are using output_file when you are passing a book folder in render_book().
I don't think we are checking that, but this seems something that should not be done.
What does it do for you ?

5

In answer to your question:
I typed into the Console of RStudio

bookdown::dir_exists("./source-posts")

and got

Error: 'dir_exists' is not an exported object from 'namespace:bookdown'

In answer to your other questions, I understand also your suggestion to add a move after files have been written to _book. The solution shown in OP was what I found and what seemed to be working and therefore I didn't think an extra move was necessary. It is hard to find best practices for this stuff so all I was going on was the ? render_book help and the options indicated to me there.

I had to use output_file because without it when writing to output_dir I could not control the name of the file (render_book ended up writing something like _main.html). That is, upon rendering the file would not be named by the rmd_files from _bookdown.yml which seemed odd to me, so I had to find the output_file work around with paste0.

Essentially what I was trying to figure out was how to find some version of what I understood and was working from

render_site(
input = "./source-body",
output_format = "all",
envir = parent.frame(),
quiet = FALSE,
encoding = "UTF-8"
)

The goal and intention of the script in the OP was (according to my perhaps inaccurate mental model)

  • take the the rmd_file in _bookdown.yml
  • which is in input folder
  • and render to the output folder
  • with name given by output_file
  • according to the options in _output.yml.

Am I reading the render_book incorrectly or thinking wrong in my mental model?

The biggest difficulty I have had with RMarkdown and Bookdown so far is having rendered files writing with the name I wish and to the location I want which is why I was so happy to find the help file for render_book. For example, from the help file for render_book

output_dir The output directory. If NULL , a field named output_dir in the configuration file ‘_bookdown.yml’ will be used (possibly not specified, either, in which case a directory name ‘_book’ will be used).

I was never able to get output_dir in _bookdown.yml to write to that location so I had to make output_dir in render_book not NULL which meant to me it should write where specified.

BTW, using the Knit button (which I have stopped using mostly in favor of the R scripts) writes files to the _book location mentioned in the help file.

BTW, I have renderscripts for other bookdown formats (like e-pub) that have been working

render_book(
input = "./source-posts",
output_format = "bookdown::epub_book",
output_dir = "../output/posts",
output_file = paste0(file_to_render,"-ebook",".epub"),
envir = parent.frame(),
output_yaml= "_output.yml",
config_file= "_bookdown.yml",
quiet = FALSE,
encoding = "UTF-8"
)

so I really hope we can get this worked out. The tools you all are giving us are fantastic, being able to write LaTeX and export to html, pdf, pub is exactly what I need.

Best,

Jason

6

Actually what I remember now is the when I passed a list of bookdown formats

c("bookdown::html_book","bookdown::pdf_book")

to the output_format of the script the files would come out named _main.html and _main.pdf. Adding

output_file= paste0(file_to_render)

wrote the files to the output_dir as file_to_render.pdf and file_to_render.html

7

Sorry !

I meant ::: as it is indeed not exported

bookdown:::dir_exists("./source-posts")
8

bookdown:::dir_exists("./source-posts")
Error: object 'dir_exists' not found

9

render_book() is used to render a book project, which will be a single HTML split into several. The _main.html is temporary and is not kept as the book will be HTML files for each chapter.

But if you want to change the name this is explained in here:

and here 4.4 Configuration | bookdown: Authoring Books and Technical Documents with R Markdown

You can configure the book_filename option in _bookdown.yml for the K-M approach, but it should be a Markdown filename, e.g., _main.md , although the filename extension does not really matter, and you can even leave out the extension, e.g., just set book_filename: _main . All other configurations work for both M-K and K-M.

Yes this is what bookdown will do, but do you really need to control all this yourself ?

Yes output_dir is the right option. About filename, you are working with book project, so there won't be only one. If you don't want a book project, but bookdown feature for a single document, then see

In this rmarkdown::render() is the way to go, with bookdown::html_document2 - bookdown::html_book() is a book format, so several files.

You may have been using the function not as expected and it was working because we did not added specific checks. Now we have added some checks to be sure that function is used as expected. So maybe this is what cause you're problem.

We may have missed a use case, so please do share a reproducible example or explain generally how you are working and using bookdown

I hope the hints and explanation above will help clarify.

10

Did you use bookdown 0.27 to run this command ?

11

Yes, if by that you mean I ran it from a Console window in RStudio whose sessionInfo shows bookdown_0.27

sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin21.3.0 (64-bit)
Running under: macOS Monterey 12.4
loaded via a namespace (and not attached):
[1] compiler_4.2.0 bookdown_0.27 fastmap_1.1.0 htmltools_0.5.2 tools_4.2.0 yaml_2.3.5
[7] rmarkdown_2.14 knitr_1.39 xfun_0.31 digest_0.6.29 rlang_1.0.2 evaluate_0.15

12

Really sorry I mixed things up. It has moved in xfun

xfun::dir_exists("./source-posts")
13

As mentioned in OP and in reference to you suggestion to look at section 3.4 of bookdown which references html_docuement2

Also I believe I understand that difference between a book project and single bookdown document. To me a book project is a collection of single documents. I figured why not write a single document in the book project (split by .rmd) so that when I do bring the whole project together I was familiar with the book project settings.

I understand that you have given a lot of resources to look at and I have looked at them and decided for myself that the render book script was the way that made the most sense. This code was working until it was not, so I hope we can keep working that solution and figure out how to fix what changed from bookdown_0.26 to 0.27 ....

or tell me how to go back to bookdown_0.26

14

xfun::dir_exists("./source-posts")
[1] FALSE

15

I am sorry...are you saying that it is not appropriate to use render_book and bookdown::html_document with a single input and that I should use render with bookdown::html_document2?

As a user I am trying to find the minimal toolset needed to do multiple things...eventually I will collect the .Rmds into a book but until then I am writing single documents. Why allow users to split by .rmd and why would it not be possible to pass just one .rmd file to _bookdown.yml? This does not really seem like an edge case to me.

16

So you get the issue because somehow the directory ./source-posts does not seem to be seen as existing from your working directory. Where is this directory compated to your R working directory when you are calling render_book()

I am not quite sure to understand here. With bookdown, a book project is a collection of several Rmd document that are supposed to be one big document with Chapter 1 to X, possibly with PARTS and APPENDIX. In HTML, it is supposed to be organized in pages as a website to save loading time and offer navigation option (like sidebar for the TOC).

I understand you are using this a bit differently:

  • You are writing everything in one document - one Rmd file ?
  • You are not splitting the file really as split_by: rmd and you want one single HTML file at the end ?

Is that right ?

I am not sure why you don't use the single document format in this case.

Is this because

output_dir is also an option in rmarkdown::render() but it seems it does not work as you want ?
Can you detail ?

but as I said, moving the output file after rendering is the safest.

To go forward, I need a project example of what you are doing, and what does not work so that I can reproduce on my side. Having a Github repo with a dummy example that reproduce the issue would be best.
Without that, it will be hard for me to consider reverting the change, especially because we did that to fix another issue as a bookdown specific feature was not working.

Can you provide such example ?

You could do that by installing an older version (see remotes::install_version() or pak :package: )

However, we usually update to push some fix and you won't benefit of them if you're stuck with this version

17

Yes, bookdown::render_book() is for book output format (gitbook(), bs4_book(), ...) and for single output format usual rmarkdown::render() is to be used. Difference is about the splitting of files mainly.

But maybe you need a book format and I am missing something. Having a full example I can see would help (as a Github / Gitlab repo or just a zip file you can send me with a full simple example)

18

I may have given all the time I can to this today, and may have to throw in the towel.

Like I said I am new to the RStudio Community, but in some cases chat window does not seem the place to handle some of these nuances.

Particularly when there are new-ish users trying their best to read your documentation and find solutions.

I will summarize,

  • I had a solution that worked correctly in bookdown_0.26 using the input option of render_book (seen in OP)

  • my file structure (typed here because I cannot seem to include a screenshot of my file structure) for that script was

bitbucket-git-folder
...
render_book script file
...
bitbucket.Rproj
|
/output
...
single.html file rendered by script here in output
|
/source-posts
...
single .Rmd file to render in source-posts

  • Clearly on _0.26 the input="./source-posts" found the .Rmd file and wrote the .html file to /output/single.html

  • Now in _0.27 you all changed something so that in render_book the input no longer finds the single file. That seems like a problem for you all to fix independent of my Reproducibility Example.

  • I understand that you want me to use html_document2, but I pointed out my difficulties there. I can play around with the render as you suggest and we'll see but...

  • ...I think I have pointed out something legitimate in _0.26 to _0.27

  • The solution I am trying to find is a way to keep all my .Rmds in /source-posts and the outputs (.html, .gitbook, .epub, .pdf all in a single output folder). My solution worked in _0.26 but not in _0.27 and I still don't know why from this long series of chats.

That's all the energy I have to keep writing...I do wish there was a tech support number for users to voice what they are trying their best to do...

Best,

Jason

19

Thanks for the sum-up. Last questions before I'll try to reproduce on my side your organisation of project and see if I can adapt our fix to this usage.

Do you have an index.Rmd in this folder ? bookdown::render_book() assumes a index.Rmd is in the folder when input is a directory. Maybe we need to relax this in some cases.

From help page of render_book() Render multiple R Markdown documents into a book — render_book • bookdown

input

A directory, an input filename or multiple filenames. For a directory, index.Rmd will be used if it exists in this (book) project directory. For filenames, if preview = TRUE , only files specified in this argument are rendered, otherwise all R Markdown files specified by the book are rendered.

I feel you don't have it and using bookdown project without it, which is not something we expect (bs4_book() or gitbook() expect a index.Rmd with a YAML header).

Hope you'll find time to give me this complement, but I'll take that as th assumption of the source of the issue here.

Sorry for not being clear. Investigating such things is not straightforward, especially without an actual example with files, that explicitly shows how you are using the tools. We are trying our best to help, document and explain, but sometimes there is not specific usage we did not think of and it is hard to fully comprehend without having files and codes and stuff.

Thanks for your time to explain your issue - I'll see what I can do with the information you were able to gave me.

20

Thanks for the file structure clarification. Yes index.Rmd is in the /source-posts folder

title: ""
site: bookdown::bookdown_site
link-citations: yes

along with _bookdown.yml, _output.yml and others

bitbucket-git-folder
...
render_book script file
...
bitbucket.Rproj
|
/output
...
single.html file rendered by script here in output
|
/source-posts
...
single .Rmd file to render in source-posts
index.Rmd
_bookdown.yml
_output.yml
styles.css
toc.css

I understand how hard it is to debug.

Thanks for all your help. Like I said this is a great tool and is really helpful to me, and I am sure others. I was just surprised to find find something I worked so hard to find/develop and thought I understood to find that it had stop working suddenly.

I will try to find time to create a public bitbucket repo about this file structure-use case. I am working on this now.

Best,

Jason