[scikit-learn] GridsearchCV

[Joel Nothman]

If you're using something like n_jobs=-1, that will explode memory usage in
proportion to the number of cores, and particularly so if you're passing
the data as a list rather than array and hence can't take advantage of
memmapped data parallelism.

On 16 March 2017 at 15:46, Carlton Banks <noflaco at gmail.com> wrote:

> The ndarray (6,3,3) => (row, col,color channels)
>
> I tried fixing it converting the list of numpy.ndarray to
> numpy.asarray(list)
>
> but this causes a different problem:
>
> is grid use a lot a memory.. I am running on a super computer, and seem to
> have problems with memory.. already used 62 gb ram..
>
> > Den 16. mar. 2017 kl. 05.30 skrev Sebastian Raschka <
> se.raschka at gmail.com>:
> >
> > Sklearn estimators typically assume 2d inputs (as numpy arrays) with
> shape=[n_samples, n_features].
> >
> >> list of Np.ndarrays of shape (6,3,3)
> >
> > I assume you mean a 3D tensor (3D numpy array) with shape=[n_samples,
> n_pixels, n_pixels]? What you could do is to reshape it before you put it
> in, i.e.,
> >
> > data_ary = your_ary.reshape(n_samples, -1).shape
> >
> > then, you need to add a line at the beginning your CNN class that does
> the reverse, i.e., data_ary.reshape(6, n_pixels, n_pixels).shape. Numpy’s
> reshape typically returns view objects, so that these additional steps
> shouldn’t be “too” expensive.
> >
> > Best,
> > Sebastian
> >
> >
> >
> >> On Mar 16, 2017, at 12:00 AM, Carlton Banks <noflaco at gmail.com> wrote:
> >>
> >> Hi…
> >>
> >> I currently trying to optimize my CNN model using gridsearchCV, but
> seem to have some problems feading my input data..
> >>
> >> My training data is stored as a list of Np.ndarrays of shape (6,3,3)
> and my output is stored as a list of np.array with one entry.
> >>
> >> Why am I having problems parsing my data to it?
> >>
> >> best regards
> >> Carl B.
> >> _______________________________________________
> >> scikit-learn mailing list
> >> scikit-learn at python.org
> >> https://mail.python.org/mailman/listinfo/scikit-learn
> >
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170316/c2de649f/attachment.html>