Building packages from source on macOS (Silicon)

General
1

Hi! I'm trying to get to the bottom of this.

When I compile certain packages from source (when using renv::restore() for example and a binary is not available) on macOS (silicon) compilation fails because of missing libraries.

However, the CRAN R page for macOS (R for macOS) says that to compile R and packages one only needs XCode and Fortran. Despite having XCode and the (correct) Fortran, packages like Matrix and mgcv fail to build.

Many posts on the internet suggest to install libraries ad hoc using home brew but I'm not a fan because I don't want to mix apple and brew things (I do use home brew tho).

I just stumbled upon this page recently (Linked in a comment on stackexchange): Binaries of libraries and tools for macOS which claims that extra tools are in fact needed and indeed after for example installing gettext with the function provided on that page matrix and mgcv compile fine.

My main question (maybe more for CRAN than Posit) is, how come this is not more explicitly flagged on the CRAN page? Another (philosophical) question (for all) why isn't this more often mentioned online for example on stackexchange? In other words, why isn't this more common knowledge?

Have I perhaps missed something?

EDIT: I have also found this: macrtools: R Development Toolchain for macOS • macrtools

2

Most people install package binaries on macOS, which almost never need system packages. You only need system packages if you are compiling packages from source.

Nevertheless, if you want to install the CRAN builds of all system packages, you could do this:

curl -L -o sysreqs.tar.xz "https://github.com/r-lib/actions-files/releases/latest/download/r-macos-sysreqs-darwin23-arm64-full.tar.xz"
sudo tar xf sysreqs.tar.xz -C /

C.f. actions/setup-r-sysreqs/action.yaml at main · r-hub/actions · GitHub

You might want to add /opt/R/arm64/bin to your PATH.

3

It is true that most people install package binaries, but with renv that is almost never the case unless it is a very recent project. It is very common to have to install older packages from source when restoring renv projects (sometimes even just a few months old).

Do I understand correctly that your proposed solution does more or less what mcrtools does or what the Binaries of libraries and tools for macOS suggests? I think that still wouldn't work with data.table for example, because it needs OpenMP support and for that you need extra steps (see linker error on mac clang-17: symbol not found in flat namespace '___kmpc_barrier' · Issue #6622 · Rdatatable/data.table · GitHub).

In other words, making a project fully reproducible on macOS is basically a lost cause? (unless one has exactly same hardware same OS version same config and so on?)

4

The configuration I suggested works on CRAN for data.table as is, albeit without OpenMP support. This is an issue specific to data.table (or rather to packages that use OpenMP, but there is very few of them). You can probably set up your system and/or R installation in a way that data.table will get OpenMP support when compiled from source.

Strictly speaking, that's true for any system, not just macOS. But yes macOS is changing relatively fast, macOS releases are only supported for three years by Apple.

I think "full reproducibility" is (or should be) a non-goal in most projects, and we should focus on reasonably easy reproducibility, which is hopefully attainable, even with the current tools we have. E.g. if you need OpenMP support to reproduce some (scientific or other) result, that's probably not a result that is important to reproduce.