Mailman 3 January 2014 - NumPy-Discussion

Altering/initializing NumPy array in C
by Bart Baker Jan. 2, 2014

Jan. 2, 2014

Hello, I'm having issues with performing operations on an array in C and passing it back to Python. The array values seem to become unitialized upon being passed back to Python. My first attempt involved initializing the array in C as so: double a_fin[max_mth]; where max_mth is an int. I fill in the values of a_fin and then, before returning back to Python, I create a NumPy array in C and fill it in using a pointer to the a_fin array: npy_intp a_dims[2] = {max_mth, 1}; a_fin_array = (PyArrayObject *) PyArray_SimpleNewFromData(2, a_dims, NPY_DOUBLE, a_fin); I update the flags as so: PyArray_UpdateFlags(a_fin_array, NPY_OWNDATA); and return using: PyObject *Result = Py_BuildValue("OO", a_fin_array, b_fin_array); Py_DECREF(a_fin_array); (there is another array, b_bin_array that I create in this way and it suffers from the same issues). Immediately upon returning to Python, all of a_fin_array appears unitilized. This only happens in certain situations and sometime only part of the arrary will be unitilized. I check the values of a_dim and a_fin_array in C using gdb and they appear as expected, but are over-written with unitialized values upon returning to Python. I've tried initializing in Python and then passing the NumPy array in instead of initializing in C, but the effects of the calculations in C are still not kept. My feeling is that, with the Numpy C-API, this should be a simple process, but I'm having a lot of trouble with it. Any help would be much appreciated. I didn't want to give too much information in the post, but please let me know what other information would be useful. -Bart

2 2

Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 88, Issue 1
by Amira Chekir Jan. 1, 2014

Jan. 1, 2014

On 31.12.2013 14:13, Amira Chekir wrote: > > Hello together, > > > > I try to load a (large) NIfTI file (DMRI from Human Connectome Project, > > about 1 GB) with NiBabel. > > > > import nibabel as nib > > img = nib.load("dmri.nii.gz") > > data = img.get_data() > > > > The program crashes during "img.get_data()" with an "MemoryError" > > (having 4 GB of RAM in my machine). > > > > Any suggestions? > > are you using a 64 bit operating system? > which version of numpy? > > assuming nibabel uses np.load under the hood you could try it with numpy > 1.8 which reduces excess memory usage when loading compressed files. Hi, Thanks for your answer. I use ubuntu 12.04 32 bits and python 2.7 I upgrade numpy to 1.8, but the error persists I think that the problem is in gzip.py : max_read_chunk = 10 * 1024 * 1024 # 10Mb What do you think? Best regards, AMIRA 2014/1/1 <numpy-discussion-request(a)scipy.org> > Send NumPy-Discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. Re: proposal: min, max of complex should give warning (Ralf > Gommers) (David Goldsmith) > 2. Re: NumPy-Discussion Digest, Vol 87, Issue 35 (Amira Chekir) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 31 Dec 2013 11:43:49 -0800 > From: David Goldsmith <d.l.goldsmith(a)gmail.com> > Subject: Re: [Numpy-discussion] proposal: min, max of complex should > give warning (Ralf Gommers) > To: numpy-discussion(a)scipy.org > Message-ID: > <CAFtPsZr9FN6wwr85ScDNhqdkWZhG7C+= > rWU6EVuBMG+mY-XJdA(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > > > > As for your proposal, it would be good to know if adding a warning would > > actually catch any bugs. For the truncation warning it caught several in > > scipy and other libs IIRC. > > > > Ralf > > > > In light of this, perhaps the pertinent unit tests should be modified (even > if the warning suggestion isn't adopted, about which I'm neutral...but I'm > a little surprised that there isn't a generic way to globally turn off > specific warnings). > > DG >

2 1

Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 87, Issue 35
by Amira Chekir Jan. 1, 2014

Jan. 1, 2014

Hi, Thanks for your answer. I use ubuntu 12.04 32 bits and python 2.7 I upgrade numpy to 1.8, but the error persists I think that the problem is in gzip.py : max_read_chunk = 10 * 1024 * 1024 # 10Mb What do you think? Best regards, AMIRA 2013/12/31 <numpy-discussion-request(a)scipy.org> > Send NumPy-Discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. Loading large NIfTI file -> MemoryError (Amira Chekir) > 2. Re: Loading large NIfTI file -> MemoryError (Julian Taylor) > 3. Re: proposal: min, max of complex should give warning (Cera, > Tim) > 4. Re: proposal: min, max of complex should give warning > (Neal Becker) > 5. Re: proposal: min, max of complex should give warning > (Ralf Gommers) > 6. Re: proposal: min, max of complex should give warning > (Neal Becker) > 7. ANN: NumPy 1.7.2 release (Julian Taylor) > 8. Re: ANN: NumPy 1.7.2 release (Charles R Harris) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 31 Dec 2013 14:13:57 +0100 > From: Amira Chekir <chekir.amira(a)gmail.com> > Subject: [Numpy-discussion] Loading large NIfTI file -> MemoryError > To: numpy-discussion(a)scipy.org > Message-ID: > <CAB-foYgW4HiL37aPchrbYkgPNJLY1Dt55M8Ki30UaTg= > EQ29Zw(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hello together, > > I try to load a (large) NIfTI file (DMRI from Human Connectome Project, > about 1 GB) with NiBabel. > > import nibabel as nib > img = nib.load("dmri.nii.gz") > data = img.get_data() > > The program crashes during "img.get_data()" with an "MemoryError" (having 4 > GB of RAM in my machine). > > Any suggestions? > > Best regards, > AMIRA >

1 0