differences in how unit tests are initiated

Package development
,
1

Can someone please tell me if there is a difference between running testthat unit tests in the Rstudio Environment Pane on the build tab using the test button and running testthat::test_local() in the rstudio console?

For some reason running my unit tests via rstudio's environment panel my unit tests work, when I run via test_local in the console the tests fail.

  testthat::test_local(path=".", load_package="source")

This is how I run testthat unit tests in the console.

*ps I know that this is an XY problem that I am asking, I should focus on finding the root cause of my unit testing failures. But I am also interested in why running unit tests in this was i giving me different results.

2

AFAICT RStudio runs devtools::test() in a subprocess. (At least if you opt into using devtools.)

There are subtle difference between devtools::test() and test_local() and then there is the difference of running them in a subprocess, possibly with different PATH, possibly without RStudio's repos setup, etc.

Ideally there would be no difference. If your package is open source and you show us the failures, we might be able to spot why they are different.

3

The repository is here.

edit: changed link to the test_httptest2 branch

4

Well, with

testthat::test_local(stop_on_failure = FALSE)

I get

[ FAIL 73 | WARN 219 | SKIP 0 | PASS 26 ]

and after

Sys.setenv(TESTTHAT_MAX_FAILS = 10000)

with Build -> Test package I get

[ FAIL 73 | WARN 219 | SKIP 0 | PASS 26 ]

Which is the same.

This is a lot of failing test cases. I suggest you isolate a test case that is that is problematic, and try to debug that, and we can probably also help you debug that, should you need help.

5

Running Build->test package I get:

==> devtools::test()

:information_source: Testing RAQSAPI
Use the function
RAQSAPI::aqs_credentials(username, key)
before using other RAQSAPI functions
See ?RAQSAPI::aqs_credentials for more information
:check_mark: | F W S OK | Context
⠏ | 0 | AQS_DATAMART_API-S3-class
Attaching package: 'lubridate'

The following objects are masked from 'package:base':

date, intersect, setdiff, union

:check_mark: | 3 | AQS_DATAMART_API-S3-class
⠏ | 0 | bybox
Attaching package: 'magrittr'

The following objects are masked from 'package:testthat':

equals, is_less_than, not

:check_mark: | 5 | bybox [2.3s]
:check_mark: | 6 | byCBSA [1.1s]
:check_mark: | 14 | bycounty [21.4s]
:check_mark: | 9 | byMA [13.2s]
:check_mark: | 9 | bypqao [4.7s]
:check_mark: | 14 | bysite [41.2s]
:check_mark: | 15 | bystate [16.5s]
:check_mark: | 23 | helperfunctions
:check_mark: | 12 | RAQSAPlistfunctions [1.0s]

══ Results ══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Duration: 102.1 s

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 110 ]

:bullseye: Your tests hit the mark :bullseye:

It could be that you don't have the API key on your system, that is why unit tests are failing for you.

6

I am convinced that this has something to do with the inst/httptest2/redact.R file, because when I remove it, delete the cached mocks then rebuild the package and rerun the tests everything works fine. Even rerunning the tests there is no issue.

7

@Gabor do you mind if I mention you in the issue that I created for this issue on github?

8

No, that's fine. Thanks for asking.

9

mentioned, thank you.

10

Were you able to find anything?

11

Good morning, Just updating that I never got a solution for this. If anyone can help that would be appreachiated.