Loading huge files in background

shiny
1

Hi!

I have an app, where the choices of selectinput look like the following:
choices=read.delim(file="https://storage.googleapis.com/gencode_ch_data/mouse/mouse_searchNames.txt",header=FALSE,stringsAsFactors = FALSE, sep="\t")

So I have 3 species mouse, human drosophila. In my app there are three selectinput ui for each specie, and when I run and deploy the app it takes time initially to load the page, because I have 3 files to be loaded from the google cloud as shown above for three different kind of species.

My question will be is there a way to load these files in the background, or in a way that does not cause initial loading latency issue when I run the app? Because what my app does is to load these files first.

This issue is very obvious on IE11.

I highly appreciate your advice!
Thank you
Best Regards,
Karni

1 Like
2

I have encountered similar issues in terms of slow app startup and trying to find the best ways to at least get the UI in front of the user, even if key components are still processing. Here are a couple of ideas to pursue:

  • A recent new feature in shiny is async programming, which enables you to launch long-running tasks as separate R processes behind the scenes. I haven't used it fully in my applications yet, but I am going to do some prototyping in the near future.
  • Speaking of future, async programming is built on top of the promises and future packages. Before async programming came to be, I actually used the future package directly to offload some background processing (in my case file-based operations) and it worked extremely well.

You'll definitely want to try this out on a simple application to gain familiarity. Let us know if you decide to pursue either of these approaches!

1 Like
3

KB246, I have a very similar question Minimize delay in starting Shiny app

I found this on SO: https://stackoverflow.com/questions/39044169/can-you-put-a-shiny-progress-bar-in-global-r

Carl writes:
Each time someone goes to your URL it initiates a new instance of R, which will have to reload the CSV, so yes the CSV will be reloaded. – Carl Aug 19 '16 at 18:21

Based on Carl's response having the data preloaded in memory and ready to go for everyone who comes along sounds unlikely.

I hope I am proven wrong.

1 Like
4

You may be able to speed up your load times by using a different file format. For non-spatial data feather looks impressive.

1 Like
5

Hi,

Thank you for your advice.

After reading about async programming, I found it useful in my case where I have to read a huge file in background. But it was a little bit vague to me how to use "future" & "promise" packages in my case where I have to read rds file and then update the choices of my selectinput in my shiny server. But just reading rds files creates initial latency when I deploy my app to shinyapps.io, therefore I need to use this concept (future+promise) to solve my issue.

So I have to read the rds file first in the background and then print it in the choices of my selectinput. Can you advice me how to code that?

These are the commands to be used:

readRDS("www/mouse.rds")
updateSelectInput(session = session, inputId = "geneId_mouse", choices = readRDS("www/mouse.rds") )

I highly appreciate your support!
Thank you

6

It looks like you started a new topic for this follow-up question:

That's great — one of our community guidelines is to keep things tidy. However, if you start a new topic after asking the same question in another thread, please say so in the original thread so that people don't waste time answering your question in two different places.

You can also automatically link your new topic to the previous one by clicking on the :link: icon at the bottom of one of your posts and choosing + New Topic from the small popup.

3 Likes