Suggestions for testing if students use code to create their variables?

[hgibling]

I'm thinking of a scenario where students have a dataframe:

df <- data.frame(person=c('bob', 'bill', 'jane'), age=c(60, 65, 70))

and they need to determine the mean of the age column. Ideally we'd want students to do

my.mean <- mean(df$age)

And we would use testthat to check that mean(df$age) is the correct answer:

test_that("mean age is correct", {expect_equal(my.mean, 65)})

But if a student is not sure of how to answer the question, they could calculate the mean themselves (by eye or with a calculator, not using code) and submit

my.mean <- 65

Is there a way to check that students are using code or specific functions to determine their answers?

test_that("mean age is correct", { << something that checks if the function mean was used to generate the answer >> })

[technocrat]

See the FAQ: How to do a minimal reproducible example reprex for beginners if that is available within your testing environment.

[hgibling]

Sorry, I thought it was more of a thought question instead of troubleshooting. I've updated my question.

(deleted post: getting used to posting/editing in RStudio community)

[technocrat]

I was unlear, not you. My suggestion is that answers be provided as reprex, which would allow the return value 65 to be evaluated in terms of how it was created. (Still involves parsing.)

As a thought answer: A return value (whether from a function or a primitive, like <-) isn't generally feasible to reverse engineer to its call. Take for example, the mean mpg of mtcars:

answer0 <- 20.09062
answer1 <- mean(mtcars$mpg)
answer2 <- mean(mtcars[,"mpg"])
answer3 <- mean(mtcars[,1])
answer4 <- mtcars$mpg |> mean()

# indistinguisable
answer0
#> [1] 20.09062
answer1
#> [1] 20.09062
answer2
#> [1] 20.09062
answer3
#> [1] 20.09062
answer4
#> [1] 20.09062

# difference only visible in non-printing digits

answer0 == answer1
#> [1] FALSE

Only if the student is somehow required to cut-and-paste an answer, rather than manually entering it, is it here possible to distinguish between manual and programmatic, and then only because of floating point.

[Hong]

It looks like you're trying to check their code, rather than (or in addition to) their results.

I don't think there's any easy automated way to do this. Even if you test that they're using the mean function, they may decide to write their own function, or call it something else. This is really in the realm of code review, for which most people will still resort to the good old-fashioned eyeball.

[foojarek]

What you're looking for is at least partially covered by the gradethis package: https://pkgs.rstudio.com/gradethis/articles/gradethis.html. You can provide a desired solution (or a series of solutions) in the code and then use grade_this_code() function to do the checking. There are examples available with gradethis::gradethis_demo().

This presentation from Mine has a little bit of overview of gradethis (from slide 26): 05 interactivity and immediate feedback

[hgibling]

Thanks for introducing me to this! Unfortunately we aren't able to use learnr/shiny since we are using our institute's JupyterHub. But I will bookmark this for future applications.