The challenge is over, but a new challenge is on-going using the same datasets, check it out!
The results on each dataset should be formatted in ASCII files according to the following table. If you are a Matlab user, you may find some of the sample code routines useful for formatting the data. You can view an example of each format from the filename column. Optionally, you may submit your models in Matlab format.
|[dataname]_train.resu||Optional||Compulsory||Classifier outputs for training examples||+/-1 indicating class prediction.|
|[dataname]_valid.resu||Compulsory||Compulsory||Classifier outputs for validation examples|
|[dataname]_test.resu||Optional||Compulsory||Classifier outputs for test examples|
|[dataname]_train.conf||Optional+||Optional+||Classifier confidence for training examples||Non-negative real numbers indicating the confidence in the classification (large values indicating higher confidence). They do not need to be probabilities, and can be simply absolute values of discriminant values. Optionally they can be normalized between 0 and 1 to be interpreted as abs(P(y=1|x)-P(y=-1|x)).|
|[dataname]_valid.conf||Optional+||Optional+||Classifier confidence for validation examples|
|[dataname]_test.conf||Optional+||Optional+||Classifier confidence for test examples|
|[dataname].guess||Optional*||Compulsory*||Your prediction of the BER (Balanced Error Rate) that you will achieve on test data||A single number between 0 and 1.|
|[dataname]_model.mat||Optional||Optional||The trained CLOP model used to compute the submitted results||A Matlab learning object saved with the command save('[dataname]_model.mat', 'modelname').|
Submitted files must be in either a .zip or .tar.gz archive format. You can download the example zip archive or the example tar.gz archive to help familiarise yourself with the archive structures and contents (the results were generated with the sample code). Submitted files must use exactly the same filenames as in the example archive. If you use tar.gz archives please do not include any leading directory names for the files. Use
zip results.zip *.resu *.conf *.guess *.mator
tar cvf results.tar *.resu *.conf *.guess *.mat; gzip results.tarto create valid archives.
If you wish that your method is ranked on the overall table you should include classification results on ALL the datasets for the five tasks, but this is mandatory only for final submissions.
The method of submission is via the form on the submissions page. Please limit yourself to 5 submissions per day maximum. If you encounter problems with submission, please contact the Challenge Webmaster.
Your last 5 valid submissions will count towards the final ranking. (There are no more "bonus entries"). The deadline for submissions is March 1, 2006.