Order difference between 'shinytest' expected and current json files

[saturne]

Hello,

(Question cross-posted on SO)

I have an ordinary shinytest script to test my Shiny app. When I run it with shinytest::testApp, it is successful: both the JSON files and the images are similar between current and expected.

But when the test is ran via testhat:

      expect_pass(
        testApp(
          system.file("app", package = "shinyCircularDichroism"),
          compareImages = TRUE
        )
      )

then the images are the same but the JSON files differ:

    Name         Status      
    001.json  != Files differ
    001.png      No change   
    002.json  != Files differ
    002.png      No change   
    003.json  != Files differ
    003.png      No change   
    004.json  != Files differ
    004.png      No change   
    005.json  != Files differ
    005.png      No change  

I've inspected the differences between the JSON files. Actually they are the same up to the order (that is, one is [a, b, ... and the other is [b, a, ...]).

Is it a known issue? Why does it occur, only via testthat? Is there a way to compare the JSON files up to the order?

Here is the script:

library(shinytest)
app <- ShinyDriver$new("../../", loadTimeout = 1e+05,
                       shinyOptions = list(test.mode = TRUE))
app$snapshotInit("ShinyTest01")

app$uploadFile(
  "upload-data_load" = c(
    "Dichroisme_Circulaire_ProteinD_FarCD00003.csv",
    "Dichroisme_Circulaire_ProteinD_FarCD00004.csv",
    "Dichroisme_Circulaire_ProteinD_FarCD00005.csv",
    "Dichroisme_Circulaire_ProteinD_FarCD00006.csv",
    "Dichroisme_Circulaire_ProteinD_FarCD00007.csv"
  )
)
app$setInputs("upload-data_doLoad" = "click")
Sys.sleep(15)
app$snapshot()

app$setInputs(tabset = "Processing settings")
Sys.sleep(6)
app$setInputs(
  "dataProcess-data_useAbsorbance" = TRUE,
  wait_ = FALSE, values_ = FALSE
)
Plotly <- app$waitForValue(
  "dataProcess-data_plotCD_absorbance",
  ignore = list(NULL), iotype = "output"
)
app$snapshot()

cellValue <- "50 micro-l 18COP32054 Buffer"
cellCol <- 2
for(cellRow in 1:6){
  script <- paste0(
    "HTMLWidgets.find('#dataProcess-data_rhot_plateDesign').hot",
    "  .setDataAtCell(", cellRow - 1, ", ", cellCol - 1, ", '", cellValue, "');"
  )
  app$executeScript(script)
}
cellValue <- "50 micro-l 18COP32001"
cellCol <- 3
for(cellRow in 1:6){
  script <- paste0(
    "HTMLWidgets.find('#dataProcess-data_rhot_plateDesign').hot",
    "  .setDataAtCell(", cellRow - 1, ", ", cellCol - 1, ", '", cellValue, "');"
  )
  app$executeScript(script)
}
Sys.sleep(5)
app$snapshot()

app$setInputs(
  "dataProcess-data_processExperiment" = "click",
  wait_ = FALSE, values_ = FALSE
)
Sys.sleep(15)

app$setInputs(tabset = "Processed Data Table")
app$snapshot()

app$setInputs(tabset = "Processed Data Visualizations")
Plotly <- app$waitForValue(
  "dataPlot-data_plotCD",
  ignore = list(NULL), iotype = "output"
)
app$snapshot()

app$stop()

I can't share the app because it is huge and it is confidential.

[saturne]

Similar issue: https://github.com/rstudio/shinytest/issues/409.

[saturne]

Solved. The problem is caused by the LC_COLLATE variable. Depending on the value of this variable, the results of sort can be different.

library(shinytest)
collation <- Sys.getlocale("LC_COLLATE")
Sys.setenv(LC_COLLATE = "en_US.UTF-8")
Sys.setlocale("LC_COLLATE", "en_US.UTF-8")
shinytest::testApp("../")
Sys.setlocale("LC_COLLATE", collation)

and:

test_that("Shiny test", {
  skip_if_not_installed("shinytest")
  library(shinytest)
  collation <- Sys.getlocale("LC_COLLATE")
  Sys.setenv(LC_COLLATE = "en_US.UTF-8")
  Sys.setlocale("LC_COLLATE", "en_US.UTF-8")
  expect_pass(
    testApp(
      system.file("app", package = "shinyCircularDichroism"),
      quiet = TRUE,
      compareImages = TRUE
    )
  )
  Sys.setlocale("LC_COLLATE", collation)
})