AsyncProgress in shinyapps.io

[massa]

Hi there, I have a problem with AsyncProgress when deployed in shinyapps.io.
I can make AsyncProgress work perfectly on my machine but not on shinyapps.io (I am on a paid subscription, so the worker limitation of the free tire is not a problem).
Here is an example of the code:

library(shiny)
library(bslib)
library(bsicons)
library(promises)
library(future)
library(ipc)
plan(multisession)

# Define UI for application that draws a histogram
ui <- page_navbar(
  theme = bs_theme(version = 5,
                   bootswatch = "minty"),
  nav_spacer(),
  nav_panel(
    title = "Home",
    icon = bs_icon("house"),
    tags$p('Push Putton'),
    actionButton("btn1", "with promise"),
    actionButton("btn2", "without promise"),
  ),
  nav_panel(
    title = "Empty page",
    icon = bs_icon("database"),
    tags$p('Empty page')
  ),
  title = "Trial"
  
)

# Define server logic required to draw a histogram
server <- function(input, output, session) {
  observeEvent(input$btn1, {
    progress = AsyncProgress$new(session,
                                 min = 1,
                                 max = 10)

    #progress$set(message = "In execution ...", value = 0)
    
    future_promise({
      for (x in 1:10) {
        Sys.sleep(1)
        progress$set(message = "In execution ...", value = x)
        print(x)
      }
      progress$close()
    }) 
    
    
    
  })
  
  observeEvent(input$btn2, {
    progress = Progress$new(session = getDefaultReactiveDomain(),
                            min = 1,
                            max = 10,
                            style = getShinyOption("progress.style", default = "notification"))
    
    on.exit(progress$close())
    progress$set(message = "In execution ...", value = 0)
    
    
    for (x in 1:10) {
      Sys.sleep(1)
      progress$set(message = "In execution row ...", value = x)
      print(x)
    }

  })
  
  
  
}

# Run the application
shinyApp(ui = ui, server = server)

The interesting thing is that if I set plan(multisession(workers = 3)), the progress bar actually works, but if multiple users call the function at the same time, the app crashes, and I get the following error in the logs:

Warning: Error in : MultisessionFuture () failed to call grmall() on cluster RichSOCKnode #1 (PID 415 on localhost ‘localhost’). The reason reported was ‘error writing to connection’. Post-mortem diagnostic: A process with this PID exists, which suggests that the localhost worker is still alive.

So, it seems, but please correct me if I am wrong, that Shinyapps.io does not pass the correct PID to AsyncProgress. If the number of workers is set, it forces the PID but it crashes if the local worker already exists. Is there a workaround for this problem? Any help would be more than appreciated.

Thank you so much

[ismirsehregal]

I currently can't test it, but as a workaround you could try using library(future.callr) along with plan(callr) instead of plan(multisession).

Another advantage with callr futures compared to multisession futures is that they do not communicate via R (socket) connections. This avoids the limitation in the number of parallel futures that can be active at any time that multisession futures and cluster futures in general have [...]

[massa]

Thank you so much for your suggestion. I believe this might also interest other people, so I am posting some updates. I changed the code a bit to print the PID. Here is the new code:

library(shiny)
library(bslib)
library(bsicons)
library(promises)
library(future)
library(ipc)
plan(multisession, workers = 5) # it works
#plan(multisession) # it does not work

# Define UI for application that draws a histogram
ui <- page_navbar(
  theme = bs_theme(version = 5,
                   bootswatch = "minty"),
  nav_spacer(),
  nav_panel(
    title = "Home",
    icon = bs_icon("house"),
    tags$p('Push Putton'),
    tags$p("available cores =", future::nbrOfFreeWorkers(), inline = TRUE),
    tags$p("worker PID =",   textOutput("wrk_id", inline = TRUE)),
    actionButton("btn1", "with promise"),
    tags$p("Parallel worker PID =", textOutput("wrk_id_parallel", inline = TRUE)),
    actionButton("btn2", "without promise"),
  ),
  nav_panel(
    title = "Empty page",
    icon = bs_icon("database"),
    tags$p('Empty page')
  ),
  title = "Trial"
  
)

# Define server logic required to draw a histogram
server <- function(input, output, session) {
  valPid <- reactiveValues(data="empty")
  
  output$wrk_id <- renderText({ Sys.getpid() })
  
  output$wrk_id_parallel <- renderText({  valPid$data  })
  
  observeEvent(input$btn1, {
    progress = AsyncProgress$new(session,
                                 min = 1,
                                 max = 10)

    #progress$set(message = "In execution ...", value = 0)
    
    future_promise(
      packages = c("ipc"),
      {
      for (x in 1:10) {
        Sys.sleep(1)
        progress$set(message = "In execution ...", value = x)
        print(x)
        
      }
      progress$close()
      valore <- Sys.getpid()
      return(valore)

    })  %...>% {
      valPid$data <- .
      }
    
  })
  
  observeEvent(input$btn2, {
    progress = Progress$new(session = getDefaultReactiveDomain(),
                            min = 1,
                            max = 10,
                            style = getShinyOption("progress.style", default = "notification"))
    
    on.exit(progress$close())
    progress$set(message = "In execution ...", value = 0)
    
    
    for (x in 1:10) {
      Sys.sleep(1)
      progress$set(message = "In execution row ...", value = x)
      print(x)
    }

  })
  
  
  
}

# Run the application
shinyApp(ui = ui, server = server)

If I understand correctly, plan(multisession) not only does not work for the Asyncprogress bar, but it does not even create a separate process as it keeps having the same PID. When the number of workers is set, everything seems to work properly. The only issue is that one can run out of memory (this was the problem I previously had).
I attached two images. One with the not working code and the second one with the working code. As I said, this happens only when deployed on shinyapps.io. On my computer, both approaches seem to work.

[ismirsehregal]

Did you check your config?

It seems availableCores() returns 1 if you aren't explicitly setting the number of workers - accordingly the work is done synchronously.

[massa]

Sure I did. In the shinyapps.io settings advanced panel it is set to 3. See image.
I noticed it, but I do not understand why. If I set it through code, it shows the number set in the code.

[ismirsehregal]

The same was discussed here.

The docs on availableCores states:

Having said this, it is almost always better to do this by explicitly setting the number of workers when specifying the future strategy, e.g. plan(multiprocess, workers = 8).

So I guess this is the way to go.

[massa]

Thank you so much for your help!
Now I am more confident that it is not something that I am missing.

Cheers,