documentation or support for kubernetes jobs for parallel processing

Machine Learning and Modeling
, ,
1

Are there hands on examples for using tidymodels {tune} parallel features with kubernetes (k8s).

The documentation i have found thus far are discussing solutions like {clustermq} or using {doParallel} when trying to use tune_grid

eg

library(doParallel)
cl <- makePSOCKcluster(parallel::detectCores(logical = FALSE)-1)
registerDoParallel(cl)

or

library(clustermq)
options(clustermq.scheduler = "multiprocess")
clustermq::register_dopar_cmq( n_jobs = parallel::detectCores(logical = FALSE) - 1)

I want to leverage as much as possible my RS workbench environment that has k8s, and not the HPC alternative (eg slurm).

2

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.