Rose - K-fold cross validation with Random Forest, pls help me.

Hi everyone,

i am new in R Studio. I want to write a code;

  1. i separate data set train and test
  2. i want to use ROSE sampling with 10 times k-fold cross validation (train data set).
  3. after all, i predict class with Random forest.

This is my R code, please help me? Thank you.

data2<-read.csv("C:/Users/ugur/Desktop/R/ff.csv", header=TRUE)
ind <- sample(2, nrow(data2), replace = TRUE, prob = c(0.7, 0.3))
train <- data2[ind==1,]
test <- data2[ind==2,]
fit<-trainControl(data=ROSE(churn~., data =train , N = 4950, seed=111),method="repeatedcv", number=5, search="random", repeats="3", savePredictions=T)

#modelfitrose<-ROSE(churn~., data = , N = 4950, seed=111)$data


Hi @Ugur,

You have not provided your data, so I have created an example using the mtcars dataset that is included in R. I have also chosen to use the modern {tidymodels} infrastructure to essentially replicate the approach shown above which uses {caret}, which has sort of been superseded by `tidymodels, even though it hasn't been deprecated, per se.



# Prepare some example data
data <-
  mtcars %>% 
  mutate(vs = factor(vs))

# Split the data 70/30 for training/testing
train_test_split <- initial_split(data, 0.7, strata = 'vs')

# Extract training data
train <- training(train_test_split)

# Create 5-fold 3-repeat CV splits
cv_folds <- vfold_cv(train, v = 5, repeats = 3, strata = 'vs')

# Simple recipe with ROSE
rec <- recipe(vs ~ ., data = train) %>% 

# Specify model
model <- rand_forest(
  mode = 'classification',
  engine = 'randomForest',
  mtry = tune(),
  trees = tune(),
  min_n = tune()

# Define workflow
wflow <- workflow(
  preprocessor = rec,
  spec = model

# Perform CV
res <-
    object = wflow,
    resamples = cv_folds,
    control = control_grid(save_pred = TRUE)

# Find the best hyperparameters for the randomForest model
best_hyperparams <- select_best(res)

# Add those hyperparams to the workflow
final_wflow <- finalize_workflow(wflow, best_hyperparams)

# Fit the updated workflow to the whole train data
# Evaluate performance in the held-out test data
final_fit <- last_fit(final_wflow, train_test_split)

# Confusion matrix in the test data
collect_predictions(final_fit) %>% 
  conf_mat(vs, .pred_class)
#>           Truth
#> Prediction 0 1
#>          0 4 2
#>          1 2 3

This should provide a strong starting point to modify for your project.

1 Like

Thank you Matt. I will read your post today and i write you again. This very valuable thank you so much again @mattwarkentin Also I send data

No need to share your data. I suggest trying to adapt the code I shared for your own project.

1 Like

Also, it is really important to subsample at the right time. The caret manual talks a lot about this.

@mattwarkentin is right though, we really can't do much unless we know more about the data.

1 Like

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.