Why parallel running on shinny app take way longer than my local machine?

When my app runs on shinyapp.io it is using 14 cores, but it takes 7 mins for one parallel computation; in contrast, when it runs on my local machine with 6 cores, it only takes less than 1 mins, am I missing something?

it is essentially what I am trying to do and deployed to shinyapp.io

n_cores <- parallel::detectCores() - 2
 cl <- parallel::makeCluster(n_cores, type = "SOCK", methods = FALSE)

    parallel::clusterEvalQ(
        cl = cl,
        expr = source(file.path("functions", "my_func.R"))
    )

    doParallel::registerDoParallel(cl)
    doRNG::registerDoRNG(seed_num)
    cat("=== Start RandomDrugHitMatrix construction, N core: ",  n_cores, "===\n")

    RandomDrugHitMatrix <- foreach(i = 1:999, .combine = cbind) %dopar% {
         rand_value_vector <- get_rand_value_vector(i)
        rand_value_vector
    }
    parallel::stopCluster(cl)
    cat("=== Complete RandomDrugHitMatrix construction ===\n")

Thank you.

High core count server processors do not have high clock speeds most likely the cores in your local machine are considerably faster and your application doesn't favor parallelisation over processing power as much as you expect.

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.