...I'm creating an app in R that calls to OpenAI. I want to make sure that users in my community have speedy response times. When I run the shiny app on my own computer using multiple instances and send the inquiry all at once from 3 windows, the results come back quickly, as if they are all independent. When I do the same using three windows from shinyapps.io, the timing looks as if they are run in sequence. I would like to understand the way users have access to shinyapps.io hosted apps. Do they have to queue up, or are these each executed as separate processes? The time trials that I ran suggest that there is some sort of queuing. I want to learn what is going on, and also find out what kinds of plans offer (perhaps) different levels of service.
Are you on the basic plan ?
if the queries are blocking the instances, it may be that there are too many workers per instance and if you went to 1 worker per instance they would be sufficiently independent for better result ?
shinyapps.io Documentation - Applications (posit.co)
Or perhaps you could try async style promises ?, perhaps mirai might be useful
I'ts just 1 worker per instance as far as I know. when I run this app on my computer, it takes about 1 minute to return. When I run 3 instances, it takes a bit more than 1 minute (all three return at the same time). When I run 3 on the web hosted by shiny, it takes around 3 minutes. When I run just 1 on shinyapps., it finishes in about 1 minute. This is why I thought things are taking place in sequence. Are they? It seems that if you publish an app, it should be able to all run in parallel, subject to the resource constraints the server puts on you. Am I missing something?
I don't want to have to parallelize the code, if the server is already parallelizing service. . . (or if there is another one that does).
For Free and Starter plans, the default and maximum number of worker processes are 1. For the other plans the default is 3 with a maximum of 10.
So i was thinking maybe you have the basic plan and get the default 3 ?
I had moved to basic plan, which reports "performance boost", but I didn't notice any , and I don't know what to expect. it doesn't say how many instances would be accomodated. noticing no change, I cancelled this. but it could be it all happened to quickly? If I had 3 "worker" (I still am not completely sure what that means), should I expect 3 instances to be run in parallel? If so, the time was much longer than that.