Posit Public Package Manager for Amazon Linux 2

[alice]

Hi Everyone,

As many people before me, I am stuck with Docker images that take a very very long time to build thanks to installing R packages. I would like very much to use pre-compiled binaries for my OS, but I am on AWS where we use a mysterious Linux - Amazon Linux 2.

Can we contribute compiled packages to host for AL2? Or are there plans to add AL2?

Are there other solutions I should consider?

Thanks,
Alice

[JoeR]

Hi Alice!

Thanks for reaching out! Due to the age (and pending retirement) of Amazon Linux 2, we do not have plans to build binary packages specifically for AL2. However, as AL2 is highly-compatible with Centos7, many customers in similar situations have reported success in most cases using our Centos7 binary packages. We continue to make those available on Public Package Manager, using the repos url https://p3m.dev/cran/__linux__/centos7/latest

It may not be perfect, but even getting a large subset of the package binaries working and only building a small number of extras could make a significant improvement in your installation time.

Cheers,
Joe

[alice]

Hi Joe,
Thanks for your reply and suggestion! I will try this out.
Alice

[michaelmayer]

If docker build time is an issue, you still can speed things up even when only (or largely) sources are available. This can be achieved by using pak (https://pak.r-lib.org) for your container build. Instead of sequentially downloading, building and installing packages, pak is running all of those steps fully in parallel.

As a consequence the overall build time will be greatly reduced compared to a full source build with install.packages() alone. The more CPUs you "throw" at your docker build system, the faster the build time will become (barring any network and disk I/O bottleneck).

Also, when run with admin privileges (such as when building a docker container), pak will automatically identify system dependencies and install them so you no longer need to worry about them.

(cf singularity-rstudio/data/r-session-complete/scripts/run.R at fe2464a422a4b3db42914140457ef7da885ac436 · sol-eng/singularity-rstudio · GitHub) for an example on how this could look like. In the code, packages_needed is a simple character vector that contains the names of all packages to be installed. For increased reproducibility for package versions I am also using a time-based snapshot (see same code further up).

GitHub - michaelmayer2/pak-singularity-example is another simple example (albeit for singularity/apptainer) where by using pak instead of install.packages() we could reduce the build time of the container from 24 hours to less than 90 mins and install 1500 packages each into two versions of R.