WCCI Performance Prediction Challenge

The challenge is over, but a new challenge is on-going using the same datasets, check it out!

Bayesian Neural Networks

Submitted by Radford Neal

Bayesian neural networks, using various architectures, sometimes looking at principle components, with some special stuff for HIVA and NOVA. Models were chosen to be rather general, with selection beyond that based on validation performance and the model's own predicted performance on the test set. For HIVA, I ended up averaging predictions of three models.

Dataset Balanced Error Test guess Guess error Test score Area Under Curve
Train Valid Test Train Valid Test
ada 0.1444 0.1656 0.1753 0.1656 0.0097 0.185 0.9311 0.9234 0.9107
gina 0 0 0.0418 0.0635 0.0216 0.0635 1 1 0.9915
hiva 0.1245 0.2259 0.2824 0.2937 0.0113 0.2916 0.949 0.917 0.7627
nova 0.0008 0.004 0.0528 0.0706 0.0178 0.0706 1 1 0.9878
sylva 0.0033 0.0024 0.0066 0.007 0.0005 0.0069 1 1 0.9991
Overall 0.0546 0.0796 0.1118 0.1201 0.0122 0.1235 0.976 0.9681 0.9304

This entry is a complete valid challenge entry.