advice for maintaining report with Rmarkdown that has chunks of code with heavy processes

I finally recently made the changeover from developing R scripts in
emacs/ESS to Rmarkdown in RStudio. I love RMarkdown facilitating
creating reports and having a central document where I can comment on
code and analysis attempts (which I used to do as comments within R
scripts). The problem I find with Rmarkdown in RStudio, probably as a
common beginner's fault, is that such reports become unwieldly very
quickly especially when I am running complex analyses -- this quickly
rules out the possibility of creating a single report with all these
heavy chunks of code in it, even if the Rmarkdown file itself is small.
I have also noticed that sometimes the inline output of these chunks
disappear when I run code chunks with heavy processes/when RStudio
crashes/etc. (and hence also from the automatically knitted html file)
-- so as I am working on a complex set of analyses on the same data, I
end up having to rerun all previously run chunks if I want it in the
knitted file -- when some chunks involve particularly heavy processes,
this takes an inordinate about of time! I would really appreciate any
advice as to how to best deal with this situation and in general any
recommendations for good working habits with Rmarkdown -- or maybe is what I am doing simply not recommended in RStudio?

I see some useful advice in the thread

such as Andrea's suggestions

including putting analysis-heavy part of code outside of RMarkdown to an R script and then save to a file which can be loaded in RMarkdown. Would love to learn more from the experiences of RMarkdown users out there!

Thanks very much in advance!

When I have an Rmd file with long-running processes, I put basically the entire script in the setup chunk and test in console. This includes intermediate results that I'm going to put in chunks or formatting or even inline. When I'm ready to prepare the accompanying text. I saveRDS the object that took forever to a public repository, either on GIT or S3, comment out all the program and add

con <- path_to_rds

That gives me everything I need to write/knit/write/knit without waiting forever for the code to run.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.