[scikit-learn] no positive predictions by neural_network.MLPClassifier

[Thomas Evangelidis]

Hello Sebastian,

I did normalization of my training set and used the same mean and stdev
values to normalize my test set, instead of calculating means and stdev
from the test set. I did that because my training set size is finite and
the value of each feature is a descriptor that is characteristic of the 3D
shape of the observation. The test set would definitely have different mean
and stdev values from the training set, and if I had used them to normalize
it then I believe I would have distorted the original descriptor values.
Anyway, after this normalization I don't get 0 positive predictions anymore
by the MLPClassifier.

I still don't understand your second suggestion. I cannot find any
parameter to control the epoch or measure the cost in sklearn
.neural_network.MLPClassifier. Do you suggest to use your own classes from
github instead?
Besides that my goal is not to make one MLPClassifier using a specific
training set, but rather to write a program that can take as input various
training sets each time and and train a neural network that will classify a
given test set. Therefore, unless I didn't understand your points, working
with 3 arbitrary random_state values on my current training set in order to
find one value to yield good predictions, wont solve my problem.

best
Thomas



On 8 December 2016 at 01:19, Sebastian Raschka <se.raschka at gmail.com> wrote:

> Hi, Thomas,
> we had a related thread on the email list some time ago, let me post it
> for reference further below. Regarding your question, I think you may want
> make sure that you standardized the features (which makes the learning
> generally it less sensitive to learning rate and random weight
> initialization). However, even then, I would try at least 1-3 different
> random seeds and look at the cost vs time — what can happen is that you
> land in different minima depending on the weight initialization as
> demonstrated in the example below (in MLPs you have the problem of a
> complex cost surface).
>
> Best,
> Sebastian
>
> The default is set 100 units in the hidden layer, but theoretically, it
> should work with 2 hidden logistic units (I think that’s the typical
> textbook/class example). I think what happens is that it gets stuck in
> local minima depending on the random weight initialization. E.g., the
> following works just fine:
>
> from sklearn.neural_network import MLPClassifier
> X = [[0, 0], [0, 1], [1, 0], [1, 1]]
> y = [0, 1, 1, 0]
> clf = MLPClassifier(solver='lbfgs',
>                     activation='logistic',
>                     alpha=0.0,
>                     hidden_layer_sizes=(2,),
>                     learning_rate_init=0.1,
>                     max_iter=1000,
>                     random_state=20)
> clf.fit(X, y)
> res = clf.predict([[0, 0], [0, 1], [1, 0], [1, 1]])
> print(res)
> print(clf.loss_)
>
>
> but changing the random seed to 1 leads to:
>
> [0 1 1 1]
> 0.34660921283
>
> For comparison, I used a more vanilla MLP (1 hidden layer with 2 units and
> logistic activation as well; https://github.com/
> rasbt/python-machine-learning-book/blob/master/code/ch12/ch12.ipynb),
> essentially resulting in the same problem:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Dec 7, 2016, at 6:45 PM, Thomas Evangelidis <tevang3 at gmail.com> wrote:
>
> I tried the sklearn.neural_network.MLPClassifier with the default
> parameters using the input data I quoted in my previous post about
> Nu-Support Vector Classifier. The predictions are great but the problem is
> that sometimes when I rerun the MLPClassifier it predicts no positive
> observations (class 1). I have noticed that this can be controlled by the
> random_state parameter, e.g. MLPClassifier(random_state=0) gives always no
> positive predictions. My question is how can I chose the right random_state
> value in a real blind test case?
>
> thanks in advance
> Thomas
>
>
> --
> ======================================================================
> Thomas Evangelidis
> Research Specialist
> CEITEC - Central European Institute of Technology
> Masaryk University
> Kamenice 5/A35/1S081,
> 62500 Brno, Czech Republic
>
> email: tevang at pharm.uoa.gr
>           tevang3 at gmail.com
>
> website: https://sites.google.com/site/thomasevangelidishomepage/
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>


-- 

======================================================================

Thomas Evangelidis

Research Specialist
CEITEC - Central European Institute of Technology
Masaryk University
Kamenice 5/A35/1S081,
62500 Brno, Czech Republic

email: tevang at pharm.uoa.gr

          tevang3 at gmail.com


website: https://sites.google.com/site/thomasevangelidishomepage/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161208/ecdc08eb/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Unknown-2.png
Type: image/png
Size: 9601 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161208/ecdc08eb/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Unknown-1.png
Type: image/png
Size: 10222 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161208/ecdc08eb/attachment-0003.png>