Our research department has created a Shiny app that allows a user to get an event's probability based on selected criteria. It is based on a neural net created at the beginning of the script using the nnet library. I've set up a basic Shiny Server to host the app for production use using CentOS. The developers use RStudio on Windows workstations for development.
We've discovered that, when the researchers copy their app.R script and supporting CSVs to the CentOS box, and view the app in a browser, the app returns significantly different results than what they see on their dev workstations. The app works properly on both environments, but the probabilities are different!
I've compared the resulting nnet in RStudio on both boxes while debugging. There are significant differences in both the weights and residuals on each OS. The seeds are the same, since the scripts don't change from one OS to the other.
We've reviewed and matched up the versions of R installed on all boxes - R 3.5.1. We've also ensured that all libraries used in the app are at the exact same versions.
We've tried adding encoding args to the read.csv() calls we use, both for Windows-1252 and UTF-8 to ensure it's not an encoding issue.
We've cut down on the precision of the values in the CSV source files, on the off chance that there's a floating point issue since the values natively are 16 digit precision and we didn't know if there was some truncating or rounding that might be involved.
model.data <- read.csv('model.data.csv')
set.seed(110)
cut <- sample(2, nrow(model.data), replace = TRUE, prob = c(.90, .10))
train <- model.data[cut == 1,]
test <- model.data[cut == 2,]
train.x <- train[,-12]
train.y <- train[,12]
test.x <- test[,-12]
test.y <- test[,12]
nn.wp <- nnet(train.x, train.y,
decay = .10,
size = 4,
skip = 1)
Sample data from CSV (~55000 rows):
"TLG","TLG.log","OffLeadBefore","S.Transform","D.x","TG.x","DEZ","OTTO","DeDTTO","eff.time","ADV","OTW"
3600,8.1886891244442,0,0,1,10,75,3,3,0,1.36193576224564,1 3133,8.04974629095219,-3,-0.0535970710596484,1,10,90,2,3,0,-1.36193576224564,0
3109,8.04205641005875,-3,-0.0538035452921865,1,10,63,2,3,0,-1.36193576224564,0
Expected results is that the resulting nnet model should match on both OSes, and thus the app itself would produce correct probabilities.