mockery::stub is creating mocking functions that persist into other tests

[njtierney]

Alrighty, I’ve got a question involving mocking!

I have a function to test for M1 mac laptops, as {greta} currently doesn’t work on M1 and we want to stop the users, to signal that we know about the problem and we are working on it.

The checking for M1 code is here: greta/R/utils.R at master · greta-dev/greta · GitHub and
implemented here: greta/R/checkers.R at master · greta-dev/greta · GitHub .

Notably, this means that this error will appear whenever someone uses the main features of the R package.

We have written tests for it here: https://github.com/greta-dev/greta/blob/master/tests/testthat/test-check_m1.R using mockery.

Alas, we were getting an issue with mockery stubs,

(see this GH action error https://github.com/greta-dev/greta/runs/5194040225?check_suite_focus=true#step:12:366)

This seemed to be related to depth not working? cannot change value of locked binding for '.onLoad' · Issue #30 · r-lib/mockery · GitHub,

So we decided to use the developer version on github.

However, once we changed this over, we started getting even more issues with it leaking out into other parts of testing.

https://github.com/greta-dev/greta/runs/5197084911?check_suite_focus=true#step:12:2563

Ideally, this shouldn't be happening, since my understanding of mocking is that it just mocks that function just that one time in the test. It seems that other users are having similar problems: Mockery not cleaning up after itself properly · Issue #57 · r-lib/mockery · GitHub

Does anyone have any thoughts on how to address this?

[maelle]

Not an actual answer but a workaround. Workaround that does involve having the testing bleed into the code.

is_mac_arm64 <- function() {
 if (nzchar(Sys.getenv("GRETA_TESTING_M1_MESSAGE"))) {
  return(TRUE)
}
  si <- Sys.info()
  is_darwin <- si[["sysname"]] == "Darwin"
  is_arm64 <- si[["machine"]] == "arm64"
  is_darwin && is_arm64
}

Then in test use withr::local_envvar(list("GRETA_TESTING_M1_MESSAGE" = "something")).

What I personally like with this is not having to think about depth.

[Gabor]

In short: don't use depth. It is possible that it never worked properly, and we should remove it from mockery.

[njtierney]

Thanks to @maelle , this is the solution to this:

is_mac_arm64 <- function() {
  if (nzchar(Sys.getenv("GRETA_M1_MESSAGE_TESTING"))) {
    return(TRUE)
  }
  si <- Sys.info()
  is_darwin <- si[["sysname"]] == "Darwin"
  is_arm64 <- si[["machine"]] == "arm64"
  is_darwin && is_arm64
}

And then in tests

   
test_that("check_tf_version fails when M1 mac detected", {
  withr::local_envvar("GRETA_M1_MESSAGE_TESTING" = "on")
  expect_snapshot_error(
    check_tf_version("error")
  )
})

(from these files: greta/utils.R at 4d5a98d7632b72e2b28aae5df23d99b5c76889a7 · maelle/greta · GitHub and greta/test-check_m1.R at 4d5a98d7632b72e2b28aae5df23d99b5c76889a7 · maelle/greta · GitHub)

[njtierney]

Hmmm! I use depth quite a lot, I'll be sure to watch that space.

[njtierney]

Is there a better alternative to depth ?

[Gabor]

Defining a wrapper function that you can mock is a good approach, if you can do that.

Otherwise the mock environment variable or mock option that you did is good as well.

[njtierney]

Hmmm, can you unpack a bit more what you mean about defining a wrapper function that you can mock? Like, wrapping the function you want to mock up in a function, say I had something like so:

is_windows <- function(){
  .Platform$OS.type == "windows"
}

check_stuff <- function(){
   if (is_windows()) stop("doesn't work on windows, sorry")
}

[Gabor]

@njtierney

Like, wrapping the function you want to mock up in a function, say I had something like so:

Yes, exactly like that.