Differences in analysis when run on different devices?

General
1

So I have revisited some analysis I did in 2022 as I was rerunning a graph for a publication. However, when I reran my script I got a different result. I have run it on a different laptop and I got a different result AGAIN. I have also run in Google Collab and again, the result was different. Any ideas why?

It's structural topic modelling, on Tweets using date as metadata in the first, and sentiment in the second. Both come back different, but I have only included the one example here.

My data is definitely the same

library(stm)
processed <- textProcessor(clean_data2$text, metadata = clean_data2)
out <- prepDocuments(processed$documents, processed$vocab, processed$meta)
docs <- out$documents
vocab <- out$vocab
meta <- out$meta
set.seed(4381887)
First_stm <- stm(docs, vocab, 13, prevalence = ~ date, data = meta)
prep <- estimateEffect(1:13 ~ s(date), stmobj = First_stm, metadata = meta, uncertainty = "Global",  prior = 1e-5)
plot(x = prep, covariate = "date", method = "continuous", topics = 1:13,  xlab = "Time 2018-2022", printlegend = F, xaxt = 'n')
2

Two possibilities:

  1. The version of R or of packages may have changed.
  2. estimateEffect internally draws random numbers. The seed or the random number generator may have changed.
3

Thanks, would the set.seed not counteract the random number generation issue?

4

I would think so. I missed that you had set the seed just before the function call. The type of random number generator called may change over different R versions, so it might be worth checking whether all the places you are running on at the moment have the same version of R and of the relevant packages. If different versions does turn out to be the issue (which honestly would surprise me), you can use RNGversion() to control which random number generator is being used.

5

This doesn’t seem to have fixed the problem. I have reloaded the old version of each of the packages I’ve used from 2022, and I’m still getting different results.