hello. when I classify my data with randomForest function and predict the test data, the accuracy and kappa are 0.96 and 0.95 respectively. for optimization, I chose safs function with rfSA method with 1000 iterations. when predicting with the test data, the accuracy and kappa are less than the classification itself. can someone please say why? because of the optimization process, the performance metrics should show the improvement of classification accuracy. my data is U.S crash records with the class "FATALS".
library(randomForest)
set.seed(1700)
forest <- randomForest(as.factor(FATALS) ~.,data=newtr, importance=TRUE)
predictionrf <-predict(forest, newte,type="class")
trf<-table(predictionrf,newte$FATALS,dnn=c("Predicted", "Actual"))
rfcm<- confusionMatrix(trf)
rfcm
sarfctrl<-safsControl(functions=rfSA,method="cv",number=10)
sarf <- safs(x = newtr[,-12], y = newtr[,12], iters = 1000,differences = TRUE,safsControl = sarfctrl)
sarfp<-predict(sarf,newte,type="class")
tsarfp<-table(sarfp$pred,newte$FATALS,dnn=c("Predicted", "Actual"))
sarfcm<-confusionMatrix(tsarfp)