setting parameters in tune_sim_anneal() when using with workflow_map()

I'm wondering how to adjust the tune_sim_anneal() defaults when using it with workflow_map(), which requires the fn argument to be a character string (i.e. fn = "tune_sim_anneal"? The documentation for workflowsets indincates the fn argument needs to be a character, but doesn't say how to adjust the various parameters avialable here when providing it this way. As a specific example, if i want iter = 20 instead of the default, how would i do that in the code below?

reprex:

library(tidymodels)
library(finetune)
data(parabolic)

# prep 
set.seed(1)
split <- initial_split(parabolic)
train_set <- training(split)
test_set <- testing(split)

set.seed(2)
train_resamples <- bootstraps(train_set, times = 5)

logistic_reg_spec <- 
  logistic_reg(penalty = tune(),
               mixture = tune()) %>% 
  set_engine("glmnet")

mars_spec <- 
  mars(prod_degree = tune()) %>% 
  set_engine("earth") %>% 
  set_mode("classification")

workflow <- 
  workflow_set(
    preproc = list("formula" = class ~ .),
    models = list(lm = logistic_reg_spec,
                  mars = mars_spec)
  )

# tune 
sim_anneal_ctrl <- 
  control_sim_anneal(
    save_pred = TRUE,
    parallel_over = "everything",
    save_workflow = TRUE,
    restart = 5L,
    verbose = FALSE
  ) 

# if i want iter to be 20 instead of 10, how can i adjust that paramater?

# this doesnt work
tune_results <-
  workflow %>%
  workflow_map(
    seed = 3566,
    verbose = FALSE,
    resamples = train_resamples,
    control = sim_anneal_ctrl,
    fn = "tune_sim_anneal(iter = 20)", 
    metrics = yardstick::metric_set(roc_auc, brier_class)
  )
#> Error in `workflow_map()`:
#> ! `fn` must be one of "tune_grid", "tune_bayes", "fit_resamples",
#>   "tune_race_anova", "tune_race_win_loss", "tune_sim_anneal", or
#>   "tune_cluster", not "tune_sim_anneal(iter = 20)".
#> Backtrace:
#>     ▆
#>  1. ├─workflow %>% ...
#>  2. ├─workflowsets::workflow_map(fn = "tune_sim_anneal(iter = 20)")
#>  3. │ └─rlang::arg_match(fn, allowed_fn$func)
#>  4. │   └─rlang::arg_match0(arg, values, error_arg, error_call = error_call)
#>  5. └─rlang:::stop_arg_match(w, values = x, error_arg = y, error_call = z)
#>  6.   └─rlang::abort(msg, call = error_call, arg = error_arg)

# I can only get this to run using the defaults:
tune_results <-
  workflow %>%
  workflow_map(
    seed = 3566,
    verbose = FALSE,
    resamples = train_resamples,
    control = sim_anneal_ctrl,
    fn = "tune_sim_anneal", 
    metrics = yardstick::metric_set(roc_auc, brier_class)
  )
#> Optimizing roc_auc
#> Initial best: 0.77017
#> 1 ♥ new best           roc_auc=0.77312 (+/-0.00873)
#> 2 ♥ new best           roc_auc=0.77794 (+/-0.009105)
#> 3 ♥ new best           roc_auc=0.77957 (+/-0.009913)
#> 4 ♥ new best           roc_auc=0.78012 (+/-0.01008)
#> 5 ♥ new best           roc_auc=0.78028 (+/-0.01004)
#> 6 ◯ accept suboptimal  roc_auc=0.78023 (+/-0.01015)
#> 7 ◯ accept suboptimal  roc_auc=0.78008 (+/-0.01011)
#> 8 ◯ accept suboptimal  roc_auc=0.77982 (+/-0.01013)
#> 9 ◯ accept suboptimal  roc_auc=0.77982 (+/-0.01013)
#> 10 ✖ restart from best  roc_auc=0.77982 (+/-0.01013)
#> → A | warning: glm.fit: algorithm did not converge, glm.fit: fitted probabilities numerically 0 or 1 occurred, the glm algorithm did not converge for response "Class2"
#> There were issues with some computations   A: x1                                                 → B | warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> There were issues with some computations   A: x1There were issues with some computations   A: x3   B: x1                                                         Optimizing roc_auc
#> There were issues with some computations   A: x3   B: x1                                                         Initial best: 0.97306
#> There were issues with some computations   A: x3   B: x1There were issues with some computations   A: x4   B: x2There were issues with some computations   A: x4   B: x3                                                         1 ◯ accept suboptimal  roc_auc=0.96882 (+/-0.003275)
#> There were issues with some computations   A: x4   B: x3There were issues with some computations   A: x5   B: x5There were issues with some computations   A: x7   B: x6                                                         2 + better suboptimal  roc_auc=0.97306 (+/-0.003032)
#> There were issues with some computations   A: x7   B: x6There were issues with some computations   A: x8   B: x7There were issues with some computations   A: x8   B: x8                                                         3 ◯ accept suboptimal  roc_auc=0.96882 (+/-0.003275)
#> There were issues with some computations   A: x8   B: x8There were issues with some computations   A: x9   B: x10There were issues with some computations   A: x11   B: x11                                                           4 + better suboptimal  roc_auc=0.97306 (+/-0.003032)
#> There were issues with some computations   A: x11   B: x11There were issues with some computations   A: x12   B: x12There were issues with some computations   A: x12   B: x13                                                           5 ✖ restart from best  roc_auc=0.96882 (+/-0.003275)
#> There were issues with some computations   A: x12   B: x13There were issues with some computations   A: x12   B: x16There were issues with some computations   A: x12   B: x19                                                           6 ◯ accept suboptimal  roc_auc=0.96882 (+/-0.003275)
#> There were issues with some computations   A: x12   B: x19There were issues with some computations   A: x13   B: x19There were issues with some computations   A: x15   B: x20                                                           7 + better suboptimal  roc_auc=0.97306 (+/-0.003032)
#> There were issues with some computations   A: x15   B: x20There were issues with some computations   A: x16   B: x21There were issues with some computations   A: x16   B: x23                                                           8 ◯ accept suboptimal  roc_auc=0.96882 (+/-0.003275)
#> There were issues with some computations   A: x16   B: x23There were issues with some computations   A: x17   B: x24There were issues with some computations   A: x20   B: x25                                                           9 + better suboptimal  roc_auc=0.97306 (+/-0.003032)
#> There were issues with some computations   A: x20   B: x25There were issues with some computations   A: x20   B: x26There were issues with some computations   A: x20   B: x28                                                           10 ✖ restart from best  roc_auc=0.96882 (+/-0.003275)
#> There were issues with some computations   A: x20   B: x28There were issues with some computations   A: x20   B: x29

tune_results
#> # A workflow set/tibble: 2 × 4
#>   wflow_id     info             option    result   
#>   <chr>        <list>           <list>    <list>   
#> 1 formula_lm   <tibble [1 × 4]> <opts[3]> <tune[+]>
#> 2 formula_mars <tibble [1 × 4]> <opts[3]> <tune[+]>
Created on 2023-08-30 with reprex v2.0.2

You pass the arguments as you would with the original function.

So instead of

tune_results <-
  workflow %>%
  workflow_map(
    seed = 3566,
    verbose = FALSE,
    resamples = train_resamples,
    control = sim_anneal_ctrl,
    fn = "tune_sim_anneal(iter = 20)", 
    metrics = yardstick::metric_set(roc_auc, brier_class)
  )

use

tune_results <-
  workflow %>%
  workflow_map(
    seed = 3566,
    verbose = FALSE,
    resamples = train_resamples,
    control = sim_anneal_ctrl,
    fn = "tune_sim_anneal", 
    metrics = yardstick::metric_set(roc_auc, brier_class),
    # add them here: 
    iter = 20
  )

You may not have heard of "the ellipses" or "the three dots". If a function has these as an argument, you can usually pass through arguments that are not for the outer function (e.g. workflow_map() in this case) to the "inner" function (tune_sim_anneal()).

In the help file for worflow_map() is has:

...: Options to pass to the modeling function. See details below.

and the details say:

When passing options, anything passed in the ... will be combined with any values in the option column. The values in ... will override that column's values and the new options are added to the options column.

It is a bit unusual here but you can also add options to the workflow set before running workflow_map() or just pass them to ... if the argument is used across workflows).

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.