RStudio Connect and lmod

Posit Professional & Hosted Products Posit Connect
1

Is anyone loading R using lmod or another environment module system with RStudio Connect?

We're using lmod with our RStudio sever, and I was hoping to use the same R packages on our RStudio Connect server for consistency.

2

Currently you cannot use environment modules together with RStudio Connect. However, the way artifacts are deployed on Connect ensures a high degree of consistency:

  • The version of R (or Python) is matched to the version used by the developer as close as possible.
  • The version of R (and Python) packages are matched exactly to the versions used by the developer.

These steps ensure that the execution environment on Connect resembles the environment used by the developer. What sort of (additional) consistency do you want to achieve using environment modules?

3

Thanks for replying!

Here are some reasons I'd prefer to use modules:

  • We use these environment modules in RStudio Server (where modules are supported), and our compute cluster. So it would be less work for me to deploy new versions of R across our servers with modules.
  • Last compared the performance of the binary distribution of R to our module version compiled with EasyBuild, our compiled version had much better performance. That was a couple of years ago, I'm not sure if that's still true.
  • Our module version of R includes a lot more libraries than the default R (thanks to Easybuild). This saves our users time installing libraries.
  • With the module version, system dependencies for R libraries are reduced, since nearly all dependencies are built through Easybuild and distributed as modules. This has made it easier for us to provide dependencies that are outdated in the OS's package management system.

Despite that long list, this isn't a big deal for us. I was really happy to see how easy it to use environment modules in the latest version of RStudio Server, so I figured I'd ask.

4

In general you can make use of the R installations you build via Easybuild. Connect only needs to know where your R versions are installed, c.f. https://docs.rstudio.com/connect/admin/r/#r-versions. You won't see an additional environment variables set within the module, though. Ar you using those?

For packages it is more complicated. In general Connect ignores any (non-base) packages installed in the local library of the R installation. You can make exceptions, but that should be used for special cases only. However, the installation of additional packages is a one-off effort that happens only when a particular package version is used the first time for a given R version. The installation time can be reduced using binary packages provided by RStudio Package Manager.

BTW, I would be interested to here more about the performance difference you have seen in the past. With which binaries on which OS did you compare? Which BLAS/LAPACK was used?