PPM/RSPM not building Git packages after upgrade: "R is not configured on this system"

Posit Professional & Hosted Products Package Management
,
1

Hi there,
I've just upgraded to Posit Package Manager v2023.08.0-16. I was on 2022.11.0 before that.

After upgrading, I'm finding that my Git builders for private packages (on GitHub private repos) are not running. The /var/log/rstudio/rstudio-pm/rstudio-pm.log file has this message:

2023/09/29 15:53:52 Warning: Skipping git-builder 'monitor'. R is not configured on this system. Building Git packages is disabled. Please check your Package Manager configuration.

The thing is, this was working fine before the PPM upgrade. Each Git builder is connected to GitHub with a deploy SSH key, and they're tag-based builders.

R is installed, and it's on the system path. It's installed in /opt/R as described in the installation instructions.

$ which -a R
/usr/local/bin/R

$ which -a Rscript
/usr/local/bin/Rscript

$ ls -al /opt/R/
total 0
drwxr-xr-x. 6 root root 58 Sep 29 09:07 .
drwxr-xr-x. 8 root root 87 Feb 17  2023 ..
drwxr-xr-x. 5 root root 41 May 10  2021 4.0.5
drwxr-xr-x. 5 root root 41 Sep 26  2022 4.2.1
drwxr-xr-x. 5 root root 41 Sep 29 09:07 4.2.2
drwxr-xr-x. 5 root root 41 Sep 29 09:07 4.2.3

$ ls -al /usr/local/bin/R
lrwxrwxrwx. 1 root root 18 Sep 29 09:31 /usr/local/bin/R -> /opt/R/4.2.2/bin/R

$ ls -al /usr/local/bin/Rscript
lrwxrwxrwx. 1 root root 24 Sep 29 09:31 /usr/local/bin/Rscript -> /opt/R/4.2.2/bin/Rscript

$ R --version
R version 4.2.2 (2022-10-31) -- "Innocent and Trusting"

I have referenced this version in my configuration file. I was on 4.2.1 before the upgrade, and noticed this issue, so thought perhaps upgrading to 4.2.2 might fix it, but unfortunately not. Here are the relevant lines from the /etc/rstudio-pm/rstudio-pm.gcfg file (I've left out the other keys, but the server section also has Address and DataDir configured):

[Server]
; Git sources require a configured R installation. R is often installed at `/usr/lib/R`
; or `/usr/lib64/R`.
RVersion = /opt/R/4.2.2

What am I missing? A service restart hasn't encouraged PPM to find the installed version of R.

Thanks in advance.

2

Can you check your server log? It's likely that there's an error on startup, probably referencing sandbox tests, and that could be helpful. Also, we've just discovered that SELinux may interfere with the sandbox tests; are you running with SELinux enabled?

3

I thought it would be helpful to report that we are having the same issue upon upgrading to the same version of Package Manager. Our setup is identical to OP's except our R version is still at 4.0.5. We are using Ubuntu 20.04.

The issue does seem to be related to sandboxed namespaces if you look at some of the output of our logs after a restart of the rstudio-pm service:

2023/10/24 17:36:06 Starting Posit Package Manager v2023.08.0-16
2023/10/24 17:36:06 Using cache version 14
2023/10/24 17:36:06 Using metrics cache version 1
time="2023-10-24T17:36:06.913Z" level=info msg="Loading server configuration from /etc/rstudio-pm/rstudio-pm.gcfg"
...
2023/10/24 17:36:07 Service ready to listen for requests
2023/10/24 17:36:07 Starting Posit Package Manager Services
2023/10/24 17:36:07 Starting licensing...
...
2023/10/24 17:36:07 Usage data enabled
2023/10/24 17:36:08 Info: Unable to use user namespace sandboxing: "sandboxed environment contains unexpected variable GOTRACEBACK"
2023/10/24 17:36:08 Operating system:         ubuntu
2023/10/24 17:36:08 Operating system version: 20.04
2023/10/24 17:36:08 Kernel version:           5.15.0-1045-aws
2023/10/24 17:36:08 You are using a supported kernel (5.15.0-1045-aws), but user namespaces are not supported: reason unknown.
2023/10/24 17:36:08 Info: User namespaces are not supported on this system so Git builds will be run without process isolation/sandboxing.
2023/10/24 17:36:08 Attempting to detect R version in a call to rsandbox
2023/10/24 17:36:08 Detected R version 4.0.5 at /opt/R/4.0.5/bin/R
2023/10/24 17:36:09 Checking for persistent storage migration
2023/10/24 17:36:09 No migration required
2023/10/24 17:36:09 Checking for persistent storage migration for CRAN packages
2023/10/24 17:36:09 No CRAN package migration required
2023/10/24 17:36:09 Checking for named metrics migration requirement.
2023/10/24 17:36:09 No named metrics migration required.
2023/10/24 17:36:09 Checking for newly supported operating systems and R versions
2023/10/24 17:36:10 Updated supported operating systems and R versions to latest
2023/10/24 17:36:10 Error: could not decrypt ssh Git credential 'redacted-key': Decryption failed

We did notice that we are able to individual git-builders working again one-by-one if we add Git.AllowUnsandboxedGitBuilds = true to our /etc/rstudio-pm/rstudio-pm.gcfg file. However this does not seem like the ideal solution because we didn't need this config setting before, and in order repair all the git-builders on the instance we will need to manually re-import the SSH key for each (even though this is also unchanged).

I'm putting in a premium support ticket, but I thought I'd share this here to be helpful.

4

@gwausc or anyone who comes across this - Posit Support confirmed this is a known bug introduced and should be rectified in 2023.08.4. The associated NEWS item is:

  • Fixed an issue that could prevent Git builds from working unless sandboxing was disabled via the Git.AllowUnsandboxedGitBuilds = true setting.
    This most commonly occurred when using a Linux Security Module such as SELinux.

Happy upgrading!

5

sigh, the upgrade did not resolve our issue.

6

Hi everyone - thanks for your replies. This issue has been somewhat superseded in my context by the fact that we're migrating to new servers, so will have a chance to reconfigure without SELinux enabled.

Hope the upgrade fixes it for others.