Tests on learnr tutorials' structure (Rmarkdown code)

[Czucca]

I co-author a teaching R package that contains tutorials written in the learnr framework. I am in the process of adding tests to the entire package.

I have no problems writing tests for the code boxes inside the tutorials (both in testthat and tinytest). However, I wondered whether there was a way to write tests on the tutorial structure (Rmarkdown code/ syntax - not the chunks of R code in the boxes).

I browsed the learnr GitHub page and learned about shinytest2. Still, it is not clear to me whether, within a learnr tutorial, it is still used to test chunks of codes in the boxes or whether it is possible to monitor potential flows in the tutorial structure.

Has anyone written tests on the actual Rmarkdown code inside the tutorials? If so,

1. what are the possible solutions to test document integrity? --how has this problem been approached.
2. Is there documentation on how to do it? Where can I find it?

Thanks

[grrrck]

Hi thanks for the feedback! I'm sure many people would be interested to learn from the things you've done in your teaching package. If you could share it, that would be awesome.

I'm not entirely sure what kinds of tests you'd like to write. Can you clarify with a concrete example?

[Czucca]

Hi there! Thanks for the prompt answer! At the moment, our package is going through a major overhaul, but we will be very happy to share it right after.

To clarify my question, a few problems we are facing at the moment.

• Assuming that I load some data in code box one, I want to carry it to code box two with

  ```{r codebox2, exercise=TRUE, echo = TRUE, exercise.setup = "codebox1"}
  some code
  ``` 
  

  We want to write tests to ensure that data loaded in a previous code box is "inherited" from another box or more than one.

• Sometimes, it happens that making changes to the package, some parts can be accidentally deleted.

  For instance

  ```{r codebox3, exercise=TRUE, echo = TRUE}
  2 + 2
  ``
  

  Note that there is a missing closing inverted comma. It is easy to delete one of the ` closing the box accidentally, and the compiler does not always catch it. It would be useful to write a test for that.

These are two examples, but we would like to hear whether someone else has already tested what I call the Rmarkdown structure already.

Many thanks!

[grrrck]

At this time, I think that shinytest2 is your best bet for testing both scenarios. Since shinytest2 is relatively new, I don't have good examples of how to use it in conjunction with learnr tutorials. But the goal of the package is to test the running shiny app and it would cover both of your use cases:

1. To test that setup chunks are chained correctly, you could evaluate test code in the second exercise and ensure that it's evaluated as expected.

2. In the second case, you could check that a known code submission for codebox3 works as expected. Missing a backtick ` would cause breakage of the exercise evaluation, either of codebox3 or one of the tutorials after it.

   Beyond that, you can also use rmarkdown::render(), rather than rmarkdown::run(), to run the pre-render step that produces the .html file required by Shiny. Then you could inspect that HTML file to ensure that the expected pieces are there.

Unfortunately, as I said, I don't have good examples to follow. You can certainly read through learnr's internal usage of shinytest2, but I would caution that my goals in those tests are quite different from yours and so my approach may not be the best to follow.

[Czucca]

Many thanks for your input. Yes, I already checked the shinytest2 tests within learnr and noticed that you are doing something quite different.

Do you think it is possible to combine shinytest2 with tinytest or does it depend on testthat?