[scikit-learn] GridsearchCV

[Sebastian Raschka]

Hm, if you set n_jobs>1, then I think it’s using multiprocessing, which will pass a copy of the input data to each process. That could be one reason for the relatively large memory consumption. 

> On Mar 16, 2017, at 12:46 AM, Carlton Banks <noflaco at gmail.com> wrote:
> 
> The ndarray (6,3,3) => (row, col,color channels)
> 
> I tried fixing it converting the list of numpy.ndarray to numpy.asarray(list)
> 
> but this causes a different problem: 
> 
> is grid use a lot a memory.. I am running on a super computer, and seem to have problems with memory.. already used 62 gb ram.. 
> 
>> Den 16. mar. 2017 kl. 05.30 skrev Sebastian Raschka <se.raschka at gmail.com>:
>> 
>> Sklearn estimators typically assume 2d inputs (as numpy arrays) with shape=[n_samples, n_features]. 
>> 
>>> list of Np.ndarrays of shape (6,3,3)
>> 
>> I assume you mean a 3D tensor (3D numpy array) with shape=[n_samples, n_pixels, n_pixels]? What you could do is to reshape it before you put it in, i.e., 
>> 
>> data_ary = your_ary.reshape(n_samples, -1).shape
>> 
>> then, you need to add a line at the beginning your CNN class that does the reverse, i.e., data_ary.reshape(6, n_pixels, n_pixels).shape. Numpy’s reshape typically returns view objects, so that these additional steps shouldn’t be “too” expensive.
>> 
>> Best,
>> Sebastian
>> 
>> 
>> 
>>> On Mar 16, 2017, at 12:00 AM, Carlton Banks <noflaco at gmail.com> wrote:
>>> 
>>> Hi… 
>>> 
>>> I currently trying to optimize my CNN model using gridsearchCV, but seem to have some problems feading my input data.. 
>>> 
>>> My training data is stored as a list of Np.ndarrays of shape (6,3,3) and my output is stored as a list of np.array with one entry. 
>>> 
>>> Why am I having problems parsing my data to it?
>>> 
>>> best regards 
>>> Carl B. 
>>> _______________________________________________
>>> scikit-learn mailing list
>>> scikit-learn at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-learn
>> 
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
> 
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn