WCCI Performance Prediction Challenge

The challenge is over, but a new challenge is on-going using the same datasets, check it out!

SNB(CMA)

Submitted by Marc Boulle

Naive Bayes assumption

Preprocessing:
Bayes optimal MODL discretization method
Feature Selection:
Evaluation criterion: Bayes regularization of the selection
Search heuristic: Multi-start(Fast Forward Backward)
Model averaging:
Full model averaging, with compression based weighing schema
Post-optimization:
Decision threshold optimized for BER

No parameter tuning, same method for all the datasets
Classifier learnt on the full train datasets

Dataset Balanced Error Test guess Guess error Test score Area Under Curve
Train Valid Test Train Valid Test
ada 0.1589 0.1947 0.1849 0.167 0.0179 0.2027 0.9173 0.8956 0.9026
gina 0.1128 0.073 0.1277 0.135 0.0073 0.1349 0.9554 0.9648 0.9438
hiva 0.2562 0.3606 0.323 0.282 0.041 0.364 0.7834 0.6234 0.7312
nova 0.0459 0.108 0.0776 0.086 0.0084 0.0858 0.991 0.9317 0.9736
sylva 0.005 0.0049 0.0066 0.007 0.0004 0.0069 0.9995 0.9993 0.999
Overall 0.1158 0.1482 0.144 0.1354 0.015 0.1588 0.9293 0.8829 0.91

This entry is a complete valid challenge entry.