[scikit-learn] purpose of test: check_classifiers_train

[Michael Capizzi]

I’m wondering if anyone can identify the purpose of this test:
check_classifiers_train(), specifically this line:
https://github.com/scikit-learn/scikit-learn/blob/ef5cb84a/sklearn/utils/estimator_checks.py#L1106

My custom classifier (which I’m hoping to submit to scikit-learn-contrib)
is failing this test:

  File "/Users/mcapizzi/miniconda3/envs/nb_plus_svm/lib/python3.6/site-packages/sklearn/utils/estimator_checks.py",
line 1106, in check_classifiers_train
    assert_greater(accuracy_score(y, y_pred), 0.83)
AssertionError: 0.31333333333333335 not greater than 0.83

And while it’s disturbing that my classifier is getting 31% accuracy when,
clearly, the test writer expects it to be in the upper-80s, I’m not sure I
understand why that would be a test condition.

Thanks for any insight.
​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20171011/ad7090e0/attachment.html>