I have a Shiny app that reads a table from Big Query. This table changes every day. It's 15 MB and takes ~10 seconds to load using bigrquery::bq_table_download. I have it in global.R file so it's outside the reactive context. Can I somehow cache it so the data won't get queried every single time but only once a day? What would be the best practice?
I like to use cachem package for caching, theres a cache_disk you can use if you can have a permanent path for the purpose to point to. I guess you would key on the date, check if the key is in the cache, if it is get it, if it isnt download your file, and set the key with it.
Thank you! Would you be willing to write an actual code for this case? Let's say my data changes each day at 8 UTC and would like to overwrite my cache.
I think the documentation is light but easy to learn from.
Why not start with going through it and then reach out again if you hit a sticking point ?
Cache R Objects with Automatic Pruning • cachem (r-lib.org)
Another approach could be to separate the data gathering process from the app, you could schedule an Rscript to pull the data on a daily basis.
Do you mean like a daily generated csv file that's then read when you run the app? That's probably a bit slower. What's the best/recommended practice between these two?
Not necessarily in csv format, there are several file formats much faster than csv, and also, not the raw data either but preprocesed data ready to be used for outputs.
It depends on the specific use case and the best way to know which is best for you is by profiling
This topic was automatically closed 54 days after the last reply. New replies are no longer allowed.
If you have a query related to it or one of the replies, start a new topic and refer back with a link.