I've been working with an 'evolutionary tree' package known as 'evtree,' but unfortunately I've ran into some obstacles in terms of accuracy-type measures for the package.

I wondered if anyone would be able to help provide some code in terms of obtaining an AUC for classification-type data, and MSE for regression-type analysis.

The two_class_example is a prediction dataset that has 'the truth' attached to the first column. The other three columns are predictions: i.e. the model predicted likelihood the result is 'class 1'; model predicted likelihood the result is 'class 2'; and the final binary guess 'predicted'.

(It's a bit different from chess22, because chess22 is 'the truth' and two independent, explanatory variables.) If you don't have likelihood predictions from your model forecast, you could estimate the ROC by simply turning your binary prediction into 1 or 0 rather than a likelihood, but a binary guess is less informative since it doesn't include information about how unsure you are.

Since the tree model based on the chess22 simulated data makes a perfect binary model, the AUC from the prediction--sans likelihoods--is also perfect, the auc is 1 and really not all that interesting.