Getting an error while parallel training an ML model inside a loop. The errors vary and they seem to happen at random during the parallel training phase. Here is one such example:
The full code can be found here
This is the code wherein the error occurs:
credit.task = makeClassifTask(id = "credit",
data = credit, target = "risk")
lrn = makeLearner("classif.svm", predict.type = "prob")
credit.lrn = cpoScale() %>>% cpoDummyEncode() %>>% lrn
param.set = pSS(
cost: numeric[0.01, 1]
)
TUNEITERS = 100L
RESAMPLING = cv5
if (PARALLEL) {
parallelMap::parallelStartSocket(parallel::detectCores(), level = "mlr.tuneParams")
}
ctrl = makeTuneControlRandom(maxit = TUNEITERS * length(param.set$pars))
lrn.tuning = makeTuneWrapper(lrn, RESAMPLING, list(mlr::acc), param.set, ctrl, show.info = FALSE)
res = tuneParams(lrn, credit.task, RESAMPLING, par.set = param.set, control = ctrl,
show.info = FALSE)
performance = resample(lrn.tuning, credit.task, RESAMPLING, list(mlr::acc))$aggr
if (PARALLEL) {
parallelMap::parallelStop()
}
credit.lrn = setHyperPars2(credit.lrn, res$x)
credit.model = mlr::train(credit.lrn, credit.task)
NOTE: The exact same code works perfectly fine when executed outside of a loop.
NOTE: This issue started appearing after I updated my RStudio to the current latest version (Version: 2023.12.0+369)
NOTE: Strangely, if I place all of the code inside of a function and then call said function, it works again.
This makes me think it's a nesting issue, but I have no idea. I am relatively new to R and I am merely reviewing existing code for use in my research.
Any guidance will be greatly appreciated. Thank you.