RConnect Deployment 'Compressing the bundle' step takes over 20 minutes, even in subsequent deployments.

[nwheatley]

I am deploying an R Plumber API that calls on python scripts via Reticulate package. It has a virtual environment with 3 python packages : pandas, scikit-learn and prettyprinter. The virtual environment is saved in my project directory, adding 10000+ files to the app.

The deployment is successful, however it takes much longer than expected.

When I deploy the app using the publish button in RStudio Server, the deployment takes 20-30 minutes to complete. When I redeploy, it takes just as long, indicating that the app is not caching the compression of the python packages or scripts.

The step that takes the longest, by far, is called 'Compressing the bundle' in the Deploy window.

----- Deployment log started at  2021-01-22 22:03:52  -----
Deploy command: 
 rsconnect::deployApp(appDir = "~/dm_cytoxtox_v2", appFileManifest = "/tmp/1ad7-e3e0-6944-f9cf",      account = "me", server = "ourserver",      appName = "dm_cytoxtox_v2", appId = 135, contentCategory = "api",      launch.browser = function(url) {         message("Deployment completed: ", url)     }, lint = FALSE, metadata = list(asMultiple = FALSE, asStatic = FALSE,          ignoredFiles = ".Rprofile|local_test.py|README.md|.vscode/settings.json"),      logLevel = "verbose") 

Session information: 
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server 7.8 (Maipo)
...
loaded via a namespace (and not attached):
[1] compiler_4.0.2   rsconnect_0.8.16
Cookies: 
[1] "None"
GET /__api__/applications/135 135ms
----- Bundle upload started at  2021-01-22 22:03:55  -----
[2021-01-22 22:03:55] Inferring App mode and parameters 
[2021-01-22 22:03:55] Bundling app dir 
[2021-01-22 22:04:49] Generate manifest.json 
[2021-01-22 22:05:27] Writing Rmd index if necessary 
[2021-01-22 22:05:27] Compressing the bundle 
POST /__api__/applications/135/upload 957916ms
[2021-01-22 22:23:44] Saving deployment record for dm_cytoxtox_v2 - me 
----- Server deployment started at  2021-01-22 22:23:44  -----
POST /__api__/applications/135/deploy 103ms
GET /__api__/tasks/NhmhIz1i5cvQbEzc?first_status=0 9ms
[Connect] Building Plumber API...
[Connect] Bundle requested R version 4.0.2; using /opt/R/4.0.2/bin/R which has version 4.0.2
...

The above deploy (a first deployment) took 20 min.
I then redeployed, and it took 17 minutes, which isn't a big savings in time.

I have a good internet connection, and otherwise things are running quickly.

How can I cut down my deploy time? How can I get the compressed python packages cached/remembered by r studio connect?

Thanks for your help.

[cole]

Thanks so much for reaching out to us on this!! It is definitely surprising that things are taking so long! Package versions definitely should be cached.

For it to take that long on upload suggests that this bundle is LARGE. Is the deployment including your virtualenv? You should definitely exclude that directory from being published!

I would recommend going to the content on Connect and looking at the Source Versions to see how big the bundles are.

If they are big, I would definitely look at trying to remove some of the bigger directories from your bundle process! If you can share with us the size / the directories being included, it would definitely be helpful for us to evaluate whether this is something we can improve in the product!

[nwheatley]

Thank you so much! I was under the impression (from previous other venv errors) that I needed to include the folder in the publish.

But, I excluded the python environment folder from the bundle everything went much faster without errors, publishing in less than 2 min, which puts me back on track to make some progress.

[cole]

Woohoo! So glad to hear - well done! Please keep us posted if you run into any further issues!