plumber: Return 200 while promise is evaluating

[KoderKow]

Hello!

We have an application setup where users can request data to be pulled and updated. I have a post request setup that I want to act as a trigger for starting a data extraction process that can take upwards of 20-30 minutes. I know it is not practical for the requestor to wait that long.

Is there a way to initiate the process with {plumber} and instantly return a 200 while the process runs separately as a promise in another session/process? I tried to see if I could get {promises} to execute in this way, but it will wait the full period before returning a response.

library(plumber)
library(promises)

#* Promise Example
#* @param id:integer ID.
#* @post /promise_exmaple
function(
  res,
  id
) {
  
  future_promise({
    print("Waiting 30 seconds...")
    Sys.sleep(30)
    print("Waiting is over!")
  }) %>%
    finally(~ {
      ## Email to alert user data is ready
    })
  
  ## While promise is evaluating, return a 200
  # msg <- "The request has been queued. You will receive an email when the process is complete."
  # res$status <- 200
  # return(list(success = msg))
}

Appreciate any help on getting this to work or other solutions for this type of situation!

[Hong]

You probably want a 202, not a 200.

[meztez]

I'm throwing something out there. What if you sent a system call and detached the process? The script would be executed in a separate process and plumber could return immediately.

I haven't tested it, just a thought.

system("RScript yourscript.R &")

[KoderKow]

I do have code written out to return a 202 if the process (not shown) is running from a previous request. I would like plumber to execute a function to pull/clean data in a separate session and instantly return a 200 of "Process started!" without having to wait for the process to finish. Does that make sense?

[KoderKow]

That is an interesting idea! I will try that right now! The process called does have parameters based on what is sent to the plumber API, I will see if I can get that to work!

Thanks for the idea

[meztez]

Check the callr package, it has some nice interface to do basically that.

[KoderKow]

I've been doing a lot of testing and I think I am really close! I have never touched upon background processes and promises before, would there be any benefit for me to wrap the callr::r_bg() in a promises::future_promise()? I feel like I don't need to since the code initiates that background process and then returns a 200. I am just unsure if its best practice to always use promises with a plumber API to handle multiple requests to put them in queue.

[meztez]

I believe you are right. No need for promises in this case.

[PRodriguez]

I think you can still use promises, but you have to first create a plan in order to tell R to launch a different session or thread, something like this:

Something like this:

library(plumber)
library(promises)
library(future)
plan(multisession)

#* Promise Example
#* @param id:integer ID.
#* @post /promise_exmaple
function(
  res,
  id
) {
 
  future_promise({
    print("Waiting 30 seconds...")
    Sys.sleep(30)
    print("Waiting is over!")
  }) %>%
    finally(~ {
      ## Email to alert user data is ready
    })
  
    ## While promise is evaluating, return a 200
    msg <- "The request has been queued. You will receive an email when the process is complete."
    res$status <- 200
    return(list(success = msg))

}