Clarify that the reference predictions don't work with prepareTestRes…

…ults

README.md

@@ -42,6 +42,15 @@ which predictions to include and which subset (train/val) to use.
 
 Loads flat test set predictions and prepares them for submission.
 
+## Reference predictions
+
+The `preds/reference` directory contains multiple validation set prediction
+files generated by established pose estimation models.
+You can compare against these predictions using `evalMPII.m`.
+
+NOTE: Since the reference predictions are for the validation set, they are
+_not_ compatible with the `prepareTestResults.m` script.
+
 ## File origins
 
 In order to keep evaluation in line with existing work, a lot of files in this

prepareTestResults.m

@@ -39,7 +39,20 @@
 
 % Check that the number of predictions is correct
 n_test_examples = length(test_indices);
-assert(size(test_preds, 3) == n_test_examples);
+
+% Validate the number of predictions
+n_test_preds = size(test_preds, 3);
+if (n_test_preds ~= n_test_examples)
+  switch n_test_preds
+    case 2958
+      fprintf('! It looks like your predictions are from the validation set,\n')
+      fprintf('! but this script works for test set predictions only.\n')
+    case 22246
+      fprintf('! It looks like your predictions are from the training set,\n')
+      fprintf('! but this script works for test set predictions only.\n')
+  end
+  error(sprintf('Expected %d predictions, got %d', n_test_examples, n_test_preds))
+end
 
 pred = RELEASE.annolist;
 n_joints = size(test_preds, 2); % Should be 16
@@ -79,13 +92,13 @@
 
 correlation = corr(cell2mat(centers)', cell2mat(thorax_preds)');
 
-display('Correlation between person centers and thorax predictions:\n');
+fprintf('Correlation between person centers and thorax predictions:\n');
 display(diag(correlation));
 
 if min(diag(correlation)) < 0.5
-  display('! Low correlation between predicted thorax positions and\n');
-  display('! annotated person centers detected. This could indicate a\n');
-  display('! mismatch in example ordering, or very poor predictions.\n');
+  fprintf('! Low correlation between predicted thorax positions and\n');
+  fprintf('! annotated person centers detected. This could indicate a\n');
+  fprintf('! mismatch in example ordering, or very poor predictions.\n');
 end
 
 save(OUT_FILE, 'pred', '-v7');