Why shouldn't resampling be automated?

[Rsky]

here is link.

usemodels 0.0.1

One thing that should not be automated is the choice of resampling method. The code templates require the user to choose the rsample function that is appropriate.

Why shouldn't resampling be automated?
I think, Do I need to adjust the ratio depending on the amount of data, because it will have a negative impact on the generalization performance?

Additional question.

Is it possible that tuning automation will appear in the future for functions for sampling as well?
Will there be a package like tune?
Shouldn't that theoretically be automated too?

thank you!

[Emilhvitfeldt]

The sampling you want to do will depend on what kind of data you have and the task at hand you are trying to accomplish. Assuming you want to estimate predictive performance and go with cross-validation, you still need to decide which type of cross-validation you want to do.

You could do a

• standard k-fold cross-validation with vfold_cv()
• if you have a time component you might want to do a Time-based Resampling
• or if you suspect there is a spatial structure in your data it could be beneficial to do Spatial or Cluster Cross-Validation

There are also cases where you don't want to do cross-validation as your resampling scheme and you want to do bootstrapping instead.

There is no one best resampling method that works all the time hence why this field is left for the user to specify.