RSPM - Package storage

Posit Professional & Hosted Products Package Management
1

Dear Community,

I am currently testing the RStudio Server Package Manager but struggling to retain the fetched packages on the server. RSPM does not seem to keep any packages.
For exemple, I download 'shiny' on RSW using my RSPM. Everything work fine, I have 'shiny' on my RSW user library. Then I remove the library of the user. Shutdown the internet connection on RSPM. Try to redownload 'shiny' on RSW using my RSPM. Fail.
In my RSPM config file, I only have set (default value for the rest):

RetainFetchedPackages = yes

Packages (at least some) tar.gz are stored in /var/lib/rstudio-pm/cran but I am not able to install them without internet.

When Internet access is cut and try to catch the package using wget, I get the following:

sudo wget http://0.0.0.0:4242/prod-cran/latest/src/contrib/shiny_1.7.0.tar.gz
--2021-09-27 07:06:42--  http://0.0.0.0:4242/prod-cran/latest/src/contrib/shiny_1.7.0.tar.gz
Connecting to 0.0.0.0:4242... connected.
HTTP request sent, awaiting response...

Does anyone know what I am missing?
My goal is to use as minimum bandwidth as possible and I have enough disk storage. But I do not want to use the air-gapped solution as it does not work for Python.
Thank you! :smiley:

RStudio Package Manager Client Version: 1.2.2.1-17; Built on May 13, 2021
RStudio Package Manager Server Version: 1.2.2.1-17; Built on May 13, 2021

> install.packages("shiny")
Installing package into ‘/home/user1/R/x86_64-pc-linux-gnu-library/4.0’
(as ‘lib’ is unspecified)
also installing the dependencies ‘fs’, ‘rappdirs’, ‘Rcpp’, ‘digest’, ‘base64enc’, ‘magrittr’, ‘sass’, ‘jquerylib’, ‘httpuv’, ‘mime’, ‘jsonlite’, ‘xtable’, ‘fontawesome’, ‘htmltools’, ‘R6’, ‘sourcetools’, ‘later’, ‘promises’, ‘crayon’, ‘rlang’, ‘fastmap’, ‘withr’, ‘commonmark’, ‘glue’, ‘bslib’, ‘cachem’, ‘ellipsis’, ‘lifecycle’

trying URL 'http://0.0.0.0:4242/prod-cran/latest/src/contrib/fs_1.5.0.tar.gz'
Warning in install.packages :
  cannot open URL 'http://0.0.0.0:4242/prod-cran/latest/src/contrib/fs_1.5.0.tar.gz': HTTP status was '500 Internal Server Error'
Error in download.file(url, destfile, method, mode = "wb", ...) : 
  cannot open URL 'http://0.0.0.0:4242/prod-cran/latest/src/contrib/fs_1.5.0.tar.gz'
Warning in install.packages :
  download of package ‘fs’ failed
[... for all packages ...]
2

Actually, I found this line in the logs:

2021/09/27 07:21:14 Package Request 500 error: Head "https://rspm-sync.rstudio.com/v3/1/packages/1c87e2362e3fea420d02e79cc630f96445180c370c2fc02c1ee09a890b064ba6.tar.gz": dial tcp: lookup rspm-sync.rstudio.com on 1.1.1.1:53: read udp 0.0.0.0:54964->1.1.1.1:53: i/o timeout

Does RSPM really needs to lookup at every package installation?
And I am not able to figure out who is 1.1.1.1...

3

Hey Luigi,

I'm a developer on the RStudio package manager product, so I'd be happy to answer some of your questions here.

I am currently testing the RStudio Server Package Manager but struggling to retain the fetched packages on the server

Fetched packages will retain on the server for 30 days after last being accessed, you can configure this option by changing the EvictionPolicy documented here: Configuration - Posit Package Manager

Does RSPM really needs to lookup at every package installation?

Yes, this is because in online mode RSPM performs lazy downloads, we don't pre-download every CRAN package on disc by default because of the number of packages and snapshots stored on the server.

But I do not want to use the air-gapped solution as it does not work for Python

We're definitely working through limitations with the air-gapped offerings, but unfortunately, for online configurations of RSPM we recommend a continuous internet connection for the server. Would you mind reaching out to support with your specific use-case?

And I am not able to figure out who is 1.1.1.1...

1.1.1.1 is Cloudflare's DNS service (more information here: https://www.cloudflare.com/learning/dns/what-is-1.1.1.1/), this is specific to your configuration, but in general the error message you're seeing is telling you that the package manager product is having difficulties reaching the internet.

Hope that helps,

Tyler

1 Like