Pre-loading Data into Shiny App

shiny
, , ,
1

I am creating a shiny app that is a CRUD application connected to a MySQL Database. While in development, it is connected to my local instance, but in production, it will connect to an AWS RDS instance and be hosted on ShinyApps.io.

What I want to know are the best practices for pre-loading data (master data) from the database into the shiny app. By pre-loading, I mean making some data available even before the server is started.

  1. Do I connect to DB outside the server and fetch all the data? Won't this increase the app startup time?
  2. Do I create a connection inside the server section and then query only needed data when I am on a particular page? Won't this slow down the individual pages?
  3. I converted a few small tables of data (master data, unchanging data) into YAML and loaded them into the config file, which can be read before starting the app, This works perfectly for small tables but not larger tables.
  4. Do I create an RDS file in a separate process and load the data from the RDS? How do I create this RDS in the first place? Using a scheduled script?
  5. Is there any other better approach?

Any advice or links to other articles will help. Thanks in advance.

2

Hi @vkatti, welcome back to the forum!

I would start by exploring this article in the Shiny docs

By using the promises package, you may be able to offload the expensive database operations into a separate process to speed up the UI. I wouldn't pursue creating separate files or the like, that seems like a nightmare to keep in sync with the database.

Best,
Randy