How to manage long-running integration tests in an R package?

Package development
, , ,
1

Hi Posit Community,

We're seeking advice on how to best manage long-running integration tests: specifically, regarding their storage location (within the same repository or a separate one for just long-running tests) and their execution frequency. Our R package includes integration tests (for statistical models) that cause devtools::test() to take approximately 21 minutes, slowing down local development and CI as we add more tests for new features. We currently store all tests under tests/testthat.

We're considering the following approaches and welcome your feedback on their pros/cons or any alternative solutions:

  1. Use naming conventions (e.g., test-unit-*.R, test-integration-*.R) and develop helper functions to allow developers to run specific test subsets locally.
  2. Skip these long-running integration tests during R CMD check.
  3. Create a separate GitHub Actions workflow for long-running integration tests, to be run less frequently.

Thanks!

1 Like
2

R CMD check does more things than just running tests. First, I'd ensure the tests are actually what's taking the bulk of the time. How long does it take to run devtools::test()?

3

Thanks for raising this question. It took 21 minutes to run devtools::test() on my local machine. The 15 minutes I mentioned in the post came from the GitHub Actions log. I’ll update the post to clarify.

4

21 minutes to run devtools::test() is abnormally long, even for a package with thousands of tests. Impossible to say what's the problem without seeing the code. However, it's probably related to either poor test design (this also includes the data used during the tests; they should be as small as possible) or badly optimized functions.

5

Do you have any suggestions or links to documentation on how to implement tests for an R package that implements a statistical model? We found that our writing of compressed .RDS could be improved upon and I also found guidance from Ropensci for "Extended tests". But, any additional guidance/examples of how other packages deal with this problem would be super helpful.

1 Like
6

I'm afraid I don't have any of these and again it's not possible to diagnose the problem without seeing the code. That said, there're many R packages that implement statistical models, so maybe checking the source code (if available) of some of those packages may help.

1 Like