<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Oh… totally forgot about that.. why -1?<br class=""><div><blockquote type="cite" class=""><div class="">Den 16. mar. 2017 kl. 05.58 skrev Joel Nothman <<a href="mailto:joel.nothman@gmail.com" class="">joel.nothman@gmail.com</a>>:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">If you're using something like n_jobs=-1, that will explode memory usage in proportion to the number of cores, and particularly so if you're passing the data as a list rather than array and hence can't take advantage of memmapped data parallelism.</div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 16 March 2017 at 15:46, Carlton Banks <span dir="ltr" class=""><<a href="mailto:noflaco@gmail.com" target="_blank" class="">noflaco@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The ndarray (6,3,3) => (row, col,color channels)<br class="">
<br class="">
I tried fixing it converting the list of numpy.ndarray to numpy.asarray(list)<br class="">
<br class="">
but this causes a different problem:<br class="">
<br class="">
is grid use a lot a memory.. I am running on a super computer, and seem to have problems with memory.. already used 62 gb ram..<br class="">
<div class="HOEnZb"><div class="h5"><br class="">
> Den 16. mar. 2017 kl. 05.30 skrev Sebastian Raschka <<a href="mailto:se.raschka@gmail.com" class="">se.raschka@gmail.com</a>>:<br class="">
><br class="">
> Sklearn estimators typically assume 2d inputs (as numpy arrays) with shape=[n_samples, n_features].<br class="">
><br class="">
>> list of Np.ndarrays of shape (6,3,3)<br class="">
><br class="">
> I assume you mean a 3D tensor (3D numpy array) with shape=[n_samples, n_pixels, n_pixels]? What you could do is to reshape it before you put it in, i.e.,<br class="">
><br class="">
> data_ary = your_ary.reshape(n_samples, -1).shape<br class="">
><br class="">
> then, you need to add a line at the beginning your CNN class that does the reverse, i.e., data_ary.reshape(6, n_pixels, n_pixels).shape. Numpy’s reshape typically returns view objects, so that these additional steps shouldn’t be “too” expensive.<br class="">
><br class="">
> Best,<br class="">
> Sebastian<br class="">
><br class="">
><br class="">
><br class="">
>> On Mar 16, 2017, at 12:00 AM, Carlton Banks <<a href="mailto:noflaco@gmail.com" class="">noflaco@gmail.com</a>> wrote:<br class="">
>><br class="">
>> Hi…<br class="">
>><br class="">
>> I currently trying to optimize my CNN model using gridsearchCV, but seem to have some problems feading my input data..<br class="">
>><br class="">
>> My training data is stored as a list of Np.ndarrays of shape (6,3,3) and my output is stored as a list of np.array with one entry.<br class="">
>><br class="">
>> Why am I having problems parsing my data to it?<br class="">
>><br class="">
>> best regards<br class="">
>> Carl B.<br class="">
>> ______________________________<wbr class="">_________________<br class="">
>> scikit-learn mailing list<br class="">
>> <a href="mailto:scikit-learn@python.org" class="">scikit-learn@python.org</a><br class="">
>> <a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank" class="">https://mail.python.org/<wbr class="">mailman/listinfo/scikit-learn</a><br class="">
><br class="">
> ______________________________<wbr class="">_________________<br class="">
> scikit-learn mailing list<br class="">
> <a href="mailto:scikit-learn@python.org" class="">scikit-learn@python.org</a><br class="">
> <a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank" class="">https://mail.python.org/<wbr class="">mailman/listinfo/scikit-learn</a><br class="">
<br class="">
______________________________<wbr class="">_________________<br class="">
scikit-learn mailing list<br class="">
<a href="mailto:scikit-learn@python.org" class="">scikit-learn@python.org</a><br class="">
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank" class="">https://mail.python.org/<wbr class="">mailman/listinfo/scikit-learn</a><br class="">
</div></div></blockquote></div><br class=""></div>
_______________________________________________<br class="">scikit-learn mailing list<br class=""><a href="mailto:scikit-learn@python.org" class="">scikit-learn@python.org</a><br class="">https://mail.python.org/mailman/listinfo/scikit-learn<br class=""></div></blockquote></div><br class=""></body></html>