Advice and best practice for dealing with the 1GB limit when scraping

A post was split to a new topic: rstudio.cloud instance crashing after the following