Mailman 3 March 2009 - NumPy-Discussion

Proposed schedule for numpy 1.3.0
by David Cournapeau March 3, 2009

March 3, 2009

Hi, A few weeks ago, we had a discussion about the 1.3.0 release schedule, but we did not end up stating a schedule. I stand up to be the release manager for 1.3.0, and suggest the following: Beta: 15th March (only doc + severe regressions accepted after, branch trunk into 1.3.x, trunk opened for 1.4.0) RC: 23th March (nothing but build issues/blockers) Release date: 1st April If you are in the middle of something important with a lot of changes, and you don't think you can make it for 15th March, please notify ASAP, in particular for C code (the exact dates can be changed in that case). What constitutes severe regression and blocker issues is decided by the release manager. I also started to update the release notes: http://projects.scipy.org/scipy/numpy/browser/trunk/doc/release/1.3.0-notes… Please fill it for missing informations (in particular for documentation, and ma, which I have not followed in detail). cheers, David

5 4

Help on subclassing numpy.ma: __array_wrap__
by Kevin Dunn March 3, 2009

March 3, 2009

Hi everyone, I'm subclassing Numpy's MaskedArray to create a data class that handles missing data, but adds some extra info I need to carrry around. However I've been having problems keeping this extra info attached to the subclass instances after performing operations on them. The bare-bones script that I've copied here shows the basic issue: http://pastebin.com/f69b979b8 There are 2 classes: one where I am able to subclass numpy (with help from the great description at http://www.scipy.org/Subclasses), and the other where I subclass numpy.ma, using the same ideas again. When stepping through the code in a debugger, lines 76 to 96, I can see that the numpy subclass, called DT, calls DT.__array_wrap__() after it completes unary and binary operations. But the numpy.ma subclass, called DTMA, does not seem to call DTMA.__array_wrap__(), especially line 111. Just to test this idea, I overrode the __mul__ function in my DTMA subclass to call DTMA.__array_wrap__() and it returns my extra attributes, in the same way that Numpy did. My questions are: (a) Is MA intended to be subclassed? (b) If so, perhaps I'm missing something to make this work. Any pointers will be appreciated. So far it seems the only way for me to sub-class numpy.ma is to override all numpy.ma functions of interest for my class and add a DTMA.__array_wrap() call to the end of them. Hopefully there is an easier way. Related to this question, was there are particular outcome from this archived discussion (I only joined the list recently): http://article.gmane.org/gmane.comp.python.numeric.general/24315 because that dictionary object would be exactly what I'm after here. Thanks, Kevin

3 2

64-bit numpy questions?
by Todd Miller March 3, 2009

March 3, 2009

Hi, I've been looking at a 64-bit numpy problem we were having on Solaris: >>> a=numpy.zeros(0x180000000,dtype='b1') >>> a.data Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: size must be zero or positive A working fix seemed to be this: Index: arrayobject.c =================================================================== --- arrayobject.c (revision 6530) +++ arrayobject.c (working copy) @@ -6774,7 +6774,7 @@ static PyObject * array_data_get(PyArrayObject *self) { - intp nbytes; + Py_ssize_t nbytes; if (!(PyArray_ISONESEGMENT(self))) { PyErr_SetString(PyExc_AttributeError, "cannot get single-"\ "segment buffer for discontiguous array"); @@ -6782,10 +6782,10 @@ } nbytes = PyArray_NBYTES(self); if PyArray_ISWRITEABLE(self) { - return PyBuffer_FromReadWriteObject((PyObject *)self, 0, (int) nbytes); + return PyBuffer_FromReadWriteObject((PyObject *)self, 0, (Py_ssize_t) nbytes); } else { - return PyBuffer_FromObject((PyObject *)self, 0, (int) nbytes); + return PyBuffer_FromObject((PyObject *)self, 0, (Py_ssize_t) nbytes); } } This fix could be simpler but still illustrates the typical problem: use of (or cast to) int rather than something "pointer sized". I can see that a lot of effort has gone into making numpy 64-bit enabled, but I also see a number of uses of int which look like problems on LP64 platforms. Is anyone using numpy in 64-bit environments on a day-to-day basis? Are you using very large arrays, i.e. over 2G in size? Cheers, Todd

5 4

RFR: fix signed/unsigned comparison warnings in numpy
by David Cournapeau March 3, 2009

March 3, 2009

Hi, A small patch to fix some last warnings (numpy almost builds warning free with -W -Wall -Wextra now). I am not sure about those (signed/unsigned casts are potentially dangerous), so I did not apply them directly. It did help me discovering a bug or two in numpy (fixed in the trunk): http://codereview.appspot.com/24043/ http://github.com/cournape/numpy/tree/unsigned_warn cheers, David

2 2

AttributeError: 'str' object has no attribute 'seek'
by Nils Wagner March 3, 2009

March 3, 2009

Hi all, I encountered a problem wrt loadtxt. Traceback (most recent call last): File "mac.py", line 9, in <module> mac = loadtxt('mac_diff.pmat.gz',skiprows=27,comments='!',usecols=(0,2,4),dtype='|S40') File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/io.py", line 384, in loadtxt fh = seek_gzip_factory(fname) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/io.py", line 51, in seek_gzip_factory f.seek = new.instancemethod(seek, f) AttributeError: 'str' object has no attribute 'seek' >>> numpy.__version__ '1.3.0.dev6520' Nils

2 2

problem with assigning to recarrays
by Brian Gerke March 3, 2009

March 3, 2009

Hi- I'm quite new to numpy and to python in general, so I apologize if I'm missing something obvious, but I've come across some seemingly nasty behavior when trying to assign values to the fields of an indexed subarray of a numpy record array. Perhaps an example would explain it best. First, I make a boring record array: In [124]: r = rec.fromarrays([zeros(5), zeros(5)], names='field1,field2') This has five elements with two fields, all values are zero. Now I can change the values for field1 for a few of the array elements: In [125]: r[1].field1=1 In [126]: r[3].field1=1 Let's check and make sure that worked: In [127]: print r.field1 [ 0. 1. 0. 1. 0.] So far, so good. Now I want to change the value of field2 for those same elements: In [128]: r[where(r.field1 == 1.)].field2 = 1 Ok, so now the values of field 2 have been changed, for those elements right? In [129]: r.field2 Out[129]: array([ 0., 0., 0., 0., 0.]) Wait. What? That can't be right. Let's check again: In [130]: print r[where(r.field1 == 1.)].field2 [ 0. 0.] Ok, so it appears that I can *access* fields in this array with an array of indices, but I can't assign new values to fields so accessed. However, I *can* change the values if I use a scalar index. This is different from the behavior of ordinary arrays, for which I can reassign elements' values either way. Moreover, when I try to reassign record array fields by indexing with an array of indices, it would appear that nothing at all happens. This syntax is equivalent to the pass command. So, my question is this: is there some reason for this behavior in record arrays, which is unexpectedly different from the behavior of normal arrays, and rather confusing. If so, why does the attempt to assign values to fields of an indexed subarray not raise some kind of error, rather than doing nothing? I think it's unlikely that I've actually found a bug in numpy, but this behavior does not make sense to me. Thanks for any insights, Brian

3 6

Hosting infrastructure upgrade tomorrow
by Stéfan van der Walt March 2, 2009

March 2, 2009

Hi all, Tomorrow afternoon at 14:00 UTC, the SciPy SVN and Trac services will be migrated to a new machine. Please be advised that, for a period of two hours, access to these and other services hosted on scipy.org may be unavailable. Regards Stéfan

1 0

Floating point question
by Gideon Simpson March 2, 2009

March 2, 2009

I recently discovered that for 8 byte floating point numbers, my fortran compilers (gfortran 4.2 and ifort 11.0) on an OS X core 2 duo machine believe the smallest number 2.220507...E-308. I presume that my C compilers have similar results. I then discovered that the smallest floating point number in python 2.5 is 4.9065...E-324. I have been using numpy to generate data, saving it with savetxt, and then reading it in as ASCII into my fortran code. Recently, it crapped out on something because it didn't like reading it a number that small, though it is apparently perfectly acceptable to python. My two questions are: 1. What is the best way to handle this? Is it just to add a filter of the form u = u * ( np.abs(u) > 2.3 e-308) 2. What gives? What's the origin of this (perceived) inconsistency in floating points across languages within the same platform? I recognize that this isn't specific to Scipy/Numpy, but thought someone here might have the answer. -gideon

4 5

Re: [Numpy-discussion] porting NumPy to Python 3
by James Watson March 2, 2009

March 2, 2009

Hi Ondrej, To get 2to3 to run without warnings, the following files require minor changes: - numpy/distutils/fcompiler/intel.py - numpy/distutils/misc_util.py - numpy/distutils/command/build_src.py - numpy/f2py/crackfortran.py - numpy/lib/function_base.py - numpy/linalg/lapack_lite/make_lite.py There are also other (possibly many, still working on this) files that require syntactic changes to run post 2to3. Is there anywhere specific I should upload these changes? Is this list appropriate? Is there a developer I can send these and future patches to? James. > Date: Tue, 3 Feb 2009 01:03:15 -0800 > From: Ondrej Certik <ondrej(a)certik.cz> > Subject: Re: [Numpy-discussion] porting NumPy to Python 3 > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > <85b5c3130902030103w69180a6bm4143050b4b82bf1f(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Hi James, > > On Thu, Jan 29, 2009 at 2:11 AM, James Watson <watson.jim(a)gmail.com> wrote: >> Hi, >> >> I am interested in contributing to the port of NumPy to Python 3. Who >> I should coordinate effort with? >> >> I have started at the Python end of the problem (as opposed to >> http://www.scipy.org/Python3k), e.g. I have several patches to get >> 2to3 to work on NumPy's Python source code. > > I am sorry that noone has replied to your email. Could you please > upload your patches somewhere? > > Ondrej

5 7

RFR: #1008 Loss of precision in (complex) arcsinh & arctanh
by Pauli Virtanen March 2, 2009

March 2, 2009

http://scipy.org/scipy/numpy/ticket/1008 http://codereview.appspot.com/22054 -- Pauli Virtanen

4 4