doParallel::registerDoParallel causes grid search fail with LightGBM engine

Does anyone have an idea why doParallel::registerDoParallel function messes up grid search (tune::tune_grid) with lightGBM engine? The error message is: All models failed. Run show_notes(.Last.tune.result) for more information.
I am using treesnip and registering threads as per documentation .
What other option do I have to run grid search in parallel?


Thanks for the post!

The tidymodels team and treesnip authors have shifted our development to a new parsnip extension package for tree-based models called bonsai, which also supports the LightGBM engine. Switching to that package should do the trick. :slight_smile:

If parallelization wiith bonsai ends up giving you trouble, please post a reprex demonstrating your problem and I'll look into this further!

1 Like

switching to bonsai package solved the issue. Thanks Simon!
On unrelated note: when using bonsai, the tune::tune_grid stopped outputting progress info, all things being the same tune::control_grid(verbose = TRUE). Is this an expected behavior? An older SO answer by Julia mentioned that this is the result of parallel processing (at least I know I am using it now, lol). Is this still a thing?

Great, I'm glad this worked for you!

This is indeed still a thing—logging is much trickier for us when parallelization is involved. From tune::control_grid docs:

verbose A logical for logging results (other than warnings and errors, which are always shown) as they are generated during training in a single R process. When using most parallel backends, this argument typically will not result in any logging.

(Emphasis mine.)

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.