unit tests fail on Ubuntu-22.04 but pass on Ubuntu-20.04

[jkurle]

Hi everyone,

I am puzzled by a problem that I am encountering since GitHub Actions changed their runner image from Ubuntu-20.04 to Ubunto-22.04. There is no problem with setting everything up but my unit tests fail now.

My unit tests pass on Ubuntu-20.04 but fail on Ubuntu-22.04. The unit tests pass locally on my machine and on GitHub Actions with Windows and Mac. I used action-tmate to check the workflow and it seems that MASS::mvrnorm() creates different random numbers under version 22.04 than 20.04, which then causes my unit tests to fail (they are based on random data).

In my function/unit tests, I do use set.seed() to make the results reproducible and in the past, the results have always coincided between OS. I also used rngtools::showRNG() to check the settings of the random number generator and they coincide.

I am unsure what is causing the problem. The mean and var-cov matrix for MASS::mvrnorm() are rounded to four digits, and I was wondering whether this is now causing any issues. Maybe Ubuntu has changed the way these matrices are stored internally or processed.

Any suggestions how to solve this? Should I save data sets to be used in my unit tests rather than generating the data within the unit tests? This should solve the problem (but I still find it worrying that I seem to get different numbers on Ubuntu-22.04 and Windows or Mac).

Any help is greatly appreciated!

[technocrat]

I’m on a phone, so can’t run it down, but this made me think about release notes, maybe 4.1?, that highlighted an algo change. Will try to run down.

[technocrat]

I was thinking of an older change.

[Gabor]

This is weird indeed! Unfortunately I have no idea why it might be happening. It could be a numerical issue, so you could compare the numbers, and round only to two digits if this is the case.

If not, then you may want to reproduce it locally. Docker is a nice way to do that, you can run the ubuntu:22.04 image, and then use GitHub - r-lib/rig: The R Installation Manager to install the very same R builds that we use on GitHub Actions.

Having fixed data in the package (tests) is a good idea to get rid of the randomness, and it is probably the best long term solution.

[jkurle]

Thank you both for your replies!

I will do some further analysis as you suggested and change my unit tests in the long term.