Mailman 3 February 2008 - NumPy-Discussion

segfault problem with numpy and pickle
by David Bolme Feb. 8, 2008

Feb. 8, 2008

A am having some trouble when pickling numpy arrays. Basically I use one python script to create the array and then pickle it. When I load the pickled array using a different python script it appears to load fine. When I try to perform a matrix multiply on the array with a vector (using dot) python crashes with a segmentation violation. The array contains 64bit floating point numbers and has a shape of (47, 16384) and the vector has a shape of (16384,1). Other shapes I have tested also have this problem. A workaround is to create a new array that is initialized with the data from the first. For example: a = array(pickle.loads(buffer)) After calling this command the script runs fine. Here is some information on my system setup: OS: Macos X Leopard 64bit Intel numpy.version.version = 1.0.5.dev4624 Below is a simple script that reproduces this error on my system. I would like to get this problem fixed so that I don't have to use this work around every time I load a pickled array. Is this a known bug? I could not find it in the bug tracker. Thanks in advance for the help. Dave import pickle from numpy import array,dot,ones,random # Create the arrays a = random.normal(size=(47,128*128)) b = ones((128*128,2)) c = ones((128*128,1)) # Test dot print "Computing dot product on original arrays..." print dot(a,b).shape # WORKS print dot(a,c).shape # WORKS print "Pickle and unpickel the array with workaround." buffer = pickle.dumps(a) a = array(pickle.loads(buffer)) # Test dot print "Computing dot product with workaround..." print dot(a,b).shape # WORKS print dot(a,c).shape # WORKS print "Pickle and unpickel the array without workaround." a = pickle.loads(buffer) # Test dot print "Computing dot product without workaround..." print dot(a,b).shape # WORKS print dot(a,c).shape # CRASH

3 4

CVXOPT and OpenOffice
by Guilherme Flach Feb. 8, 2008

Feb. 8, 2008

Hi, I'm trying to use the CVXOPT extension for OpenOffice under Windows, but I got this error: CVXOPT might not be installed. On the "CVXOPT plugin for OpenOffice.org Users's Guide" I have seen the warning "The installation of CVXOPT must be in a location known to the OpenOffice.org spreadsheet. On a Linux system this corresponds to a regular "system-wide" installation. For other platforms installation may vary." So what is the right location to install CVXOPT on Windows in order to make OpenOffice found it? Thanks in advance. Guilherme Flach

1 0

bayPIGgies meets Thursday, 2/21: Guido van Rossum on Python 3.0
by jim stockford Feb. 8, 2008

Feb. 8, 2008

* SPECIAL NOTE: because Valentine's Day is on the second * Thursday of February (2/14) bayPIGgies has moved our * meeting to the third Thursday of the month, 2/21. bayPIGgies meeting Thursday 2/21: Guido van Rossum on Python 3.0 by Guido van Rossum Guido previews his keynote about Python 3000 at PyCon next month. Hear all about what Python 3000 means for your code, what tools will be available to help you in the transition, and how to be prepared for the next millennium. Location: Google Campus in Mountain View, CA Building 40, the Kiev room (first floor) bayPIGgies meeting information: http://baypiggies.net/new/plone * Please sign up in advance to have your google access badge ready: http://wiki.python.org/moin/BayPiggiesGoogleMeetings (no later than close of business on Wednesday.) Agenda----------------------------- ..... 7:30 PM ........................... General hubbub, inventory end-of-meeting announcements, any first-minute announcements. ..... 7:35 PM to 8:45 PM ................ The Talk (may extend a bit late) ..... 8:45 PM to 9:00 PM or After The Talk ................ Mapping and Random Access Mapping is a rapid-fire audience announcement of topics the announcers are interested in. Random Access follows immediately to allow follow up individually on the announcements and other topics of interest. ..... The March Meeting ................ TBD

1 0

PySequence_GetItem on numpy array doesn't inc refcount?
by Neal Becker Feb. 8, 2008

Feb. 8, 2008

It seems that calling PySequence_GetItem on a PyArrayObject does not inc refcount on the original object? That is surprising. Then, I suppose my code is supposed to do that itself?

1 1

Problem accessing elements of an array of dtype="O" from C
by Chris Ball Feb. 8, 2008

Feb. 8, 2008

Hi, I'm having some trouble accessing elements in an array of dtype="O" from C code; I hope someone on the list could give me some advice (because I might be doing something stupid). I have an array of simple objects, created as follows: class CF(object): def __init__(self,num=0.0): self.num=num from numpy import array objs = array([[CF(0.0),CF(0.1),CF(0.2)], [CF(1.0),CF(1.1),CF(1.2)]],dtype=object) I'd like to loop through this array and access the 'num' attribute of each CF object - but using C. I have a C function (based on an example in the numpy book - 'Basic Iteration', page 312): double loop(PyObject* a_){ PyArrayIterObject *iter; iter = (PyArrayIterObject *)PyArray_IterNew(a_); while (iter->index < iter->size) { PyObject *cf = (PyObject *)(iter->dataptr); PyObject *num_obj = PyObject_GetAttrString(cf,"num"); PyArray_ITER_NEXT(iter); } return 0.0; } I get a segmentation fault when I try to call loop(objs). (Commenting out the 'PyObject_GetAttrString' line, I do not get a segmentation fault.) Using instead an array of floats and a very similar function, I have no problems: from numpy import array objs = array([[1.0,2.0,3.0],[4.0,5.0,6.0]],dtype=float) ... double loop(PyObject* a_){ PyArrayIterObject *iter; iter = (PyArrayIterObject *)PyArray_IterNew(a_); while (iter->index < iter->size) { double *num = (double *)(iter->dataptr); printf("%f\\n",*num); PyArray_ITER_NEXT(iter); } return 0.0; } I am calling this C code via Instant [1]. I have previously written to the SciPy-Dev mailing list about a strange (to me) indexing problem using arrays of dtype='O' with Weave [2]; I believe that this is the same problem (and that it is either with numpy, or with my understanding of numpy, rather than with Weave or Instant). Please could someone help me out? If you think you could help if I were to give you a runnable example of the problem in your own preferred way of calling C code from Python, please let me know, and I can probably provide one. Thanks, Chris [1] http://www.fenics.org/wiki/Instant [2] http://thread.gmane.org/gmane.comp.python.scientific.devel/7198/focus=7264

3 2

C-api to slicing?
by Neal Becker Feb. 8, 2008

Feb. 8, 2008

Is there a C-api to array slicing?

2 2

isn't it a bug? (matrix multiplication)
by dmitrey Feb. 7, 2008

Feb. 7, 2008

from numpy import array a = array((1.0, 2.0)) b = c = 15 b = b*a#ok c *= a#ok d = array(15) e = array(15) d = d*a#this works ok e *= a#this intended to be same as prev line, but yields error: Traceback (innermost last): File "<stdin>", line 1, in <module> ValueError: invalid return array shape

3 2

Re: [Numpy-discussion] Numpy-discussion Digest, Vol 17, Issue 13
by matthew yeomans Feb. 7, 2008

Feb. 7, 2008

Is it possible to compile numpy with py2exe? Matthew Yeomans On 2/6/08, numpy-discussion-request(a)scipy.org < numpy-discussion-request(a)scipy.org> wrote: > > Send Numpy-discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://projects.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Numpy-discussion digest..." > > > Today's Topics: > > 1. Re: Problem accessing elements of an array of dtype="O" from > C (Travis E. Oliphant) > 2. [Bug] important bug in method sum (humufr(a)yahoo.fr) > 3. Re: [Bug] important bug in method sum (humufr(a)yahoo.fr) > 4. Re: [Bug] important bug in method sum (Keith Goodman) > 5. Re: [Bug] important bug in method sum (Charles R Harris) > 6. Bug in numpy all() function (Dan Goodman) > 7. Re: Numpy and C++ integration... (Sebastian Haase) > 8. [ANN] Blas-LAPACK superpack, 2nd alpha (David Cournapeau) > 9. random enhancement (Neal Becker) > 10. Re: Numpy and C++ integration... (Glen W. Mabey) > 11. Re: Bug in numpy all() function (Ryan May) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 05 Feb 2008 18:32:37 -0600 > From: "Travis E. Oliphant" <oliphant(a)enthought.com> > Subject: Re: [Numpy-discussion] Problem accessing elements of an array > of dtype="O" from C > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: <47A90025.4010508(a)enthought.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Chris Ball wrote: > > Hi, > > > > I'm having some trouble accessing elements in an array of dtype="O" > > from C code; I hope someone on the list could give me some advice > > (because I might be doing something stupid). > > > > I have an array of simple objects, created as follows: > > > > class CF(object): > > def __init__(self,num=0.0): > > self.num=num > > > > from numpy import array > > objs = array([[CF(0.0),CF(0.1),CF(0.2)], > > [CF(1.0),CF(1.1),CF(1.2)]],dtype=object) > > > > > > I'd like to loop through this array and access the 'num' attribute of > > each CF object - but using C. > > > > I have a C function (based on an example in the numpy book - 'Basic > > Iteration', page 312): > > > > double loop(PyObject* a_){ > > > > PyArrayIterObject *iter; > > iter = (PyArrayIterObject *)PyArray_IterNew(a_); > > > > while (iter->index < iter->size) { > > PyObject *cf = (PyObject *)(iter->dataptr); > > PyObject *num_obj = PyObject_GetAttrString(cf,"num"); > > PyArray_ITER_NEXT(iter); > > } > > return 0.0; > > } > > > The problem here is that iter->dataptr should be re-cast to a PyObject > ** because what is contained at the memory location is a *pointer* to > the PyObject. Thus, you have to de-reference iter->dataptr to get the > PyObject * that you want: > > PyObject **cf = (PyObject **)PyArray_ITER_DATA(iter); > PyObject *num_obj = PyObject_GetAttrString(*cf, "num"); > PyArray_ITER_NEXT(iter); > > should do what you want. > > -Travis O. > > > > ------------------------------ > > Message: 2 > Date: Tue, 5 Feb 2008 14:58:40 -0500 > From: humufr(a)yahoo.fr > Subject: [Numpy-discussion] [Bug] important bug in method sum > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: <200802051458.41290.humufr(a)yahoo.fr> > Content-Type: text/plain; charset="us-ascii" > > Hello, > > when doing some test I saw a very important bug in numpy (at least on the > svn > version and 1.0.3 (ubuntu package)). > > I'm using a svn version of numpy: > > In [31]: numpy.__version__ > Out[31]: '1.0.5.dev4767' > > The problem is for an array larger than 256*256 the sum is going crazy. > > In [45]: numpy.arange(256*256) > Out[45]: array([ 0, 1, 2, ..., 65533, 65534, 65535]) > > In [46]: numpy.arange(256*256).sum() > Out[46]: 2147450880 > > In [47]: numpy.arange(257*257) > Out[47]: array([ 0, 1, 2, ..., 66046, 66047, 66048]) > > In [48]: numpy.arange(257*257).sum() > Out[48]: -2113765120 > > >>> import numpy > >>> numpy.arange(256*256).sum() > 2147450880 > >>> numpy.arange(257*257).sum() > -2113765120 > >>> numpy.__version__ > '1.0.3' > > > Sorry for this bad news. > > N. > > > ps: my system is an ubuntu linux 32 > > > > ------------------------------ > > Message: 3 > Date: Tue, 5 Feb 2008 15:20:09 -0500 > From: humufr(a)yahoo.fr > Subject: Re: [Numpy-discussion] [Bug] important bug in method sum > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: <200802051520.09863.humufr(a)yahoo.fr> > Content-Type: text/plain; charset="us-ascii" > > Sorry its not really a bug. I understood why . It's an integer and I'm > doing > an overflow. Perhaps an error message can be printed or an automatic > change > (with a warning) can be done. I think that I prefer to loose the type but > keep the value correct. > > N. > > > > ------------------------------ > > Message: 4 > Date: Tue, 5 Feb 2008 20:27:46 -0800 > From: "Keith Goodman" <kwgoodman(a)gmail.com> > Subject: Re: [Numpy-discussion] [Bug] important bug in method sum > To: "Discussion of Numerical Python" <numpy-discussion(a)scipy.org> > Message-ID: > <f4f93d420802052027v676c678bu6242d13191c41623(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Feb 5, 2008 11:58 AM, <humufr(a)yahoo.fr> wrote: > > The problem is for an array larger than 256*256 the sum is going crazy. > > > > In [45]: numpy.arange(256*256) > > Out[45]: array([ 0, 1, 2, ..., 65533, 65534, 65535]) > > > > In [46]: numpy.arange(256*256).sum() > > Out[46]: 2147450880 > > > > In [47]: numpy.arange(257*257) > > Out[47]: array([ 0, 1, 2, ..., 66046, 66047, 66048]) > > > > In [48]: numpy.arange(257*257).sum() > > Out[48]: -2113765120 > > You hit the limit on how big an integer can be. You'll have to switch > to floats to do the sum: > > >> numpy.arange(257*257, dtype=numpy.float64).sum() > 2181202176.0 > > > ------------------------------ > > Message: 5 > Date: Tue, 5 Feb 2008 22:03:58 -0700 > From: "Charles R Harris" <charlesr.harris(a)gmail.com> > Subject: Re: [Numpy-discussion] [Bug] important bug in method sum > To: "Discussion of Numerical Python" <numpy-discussion(a)scipy.org> > Message-ID: > <e06186140802052103u5547d528j6f8df5a52e44d98a(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > On Feb 5, 2008 9:27 PM, Keith Goodman <kwgoodman(a)gmail.com> wrote: > > > On Feb 5, 2008 11:58 AM, <humufr(a)yahoo.fr> wrote: > > > The problem is for an array larger than 256*256 the sum is going > crazy. > > > > > > In [45]: numpy.arange(256*256) > > > Out[45]: array([ 0, 1, 2, ..., 65533, 65534, 65535]) > > > > > > In [46]: numpy.arange(256*256).sum() > > > Out[46]: 2147450880 > > > > > > In [47]: numpy.arange(257*257) > > > Out[47]: array([ 0, 1, 2, ..., 66046, 66047, 66048]) > > > > > > In [48]: numpy.arange(257*257).sum() > > > Out[48]: -2113765120 > > > > You hit the limit on how big an integer can be. You'll have to switch > > to floats to do the sum: > > > > >> numpy.arange(257*257, dtype=numpy.float64).sum() > > 2181202176.0 > > > > Or tell numpy to use float64 for the sum: > > In [6]: a = arange(257*257) > > In [7]: a.sum(dtype=float64) > Out[7]: 2181202176.0 > > Chuck >

2 1

Bug in numpy all() function
by Dan Goodman Feb. 6, 2008

Feb. 6, 2008

Hi all, I think this is a bug (I'm running Numpy 1.0.3.1): >>> from numpy import * >>> def f(x): return False >>> all(f(x) for x in range(10)) True I guess the all function doesn't know about generators? Dan

4 4

random enhancement
by Neal Becker Feb. 6, 2008

Feb. 6, 2008

One thing missing from random is a mechanism to share a single underlying rng with other code that is not part of numpy.random. For example, I have code that generates distributions that expect a mersenne twister (the shared, underlying rng) to be passed in as a constructor argument. numpy.random shares a single rng amongst it's own distributions, but I don't see any way to share with others. Ideally, I believe this is the preferred design: rng1 = mersenne_twister (seed = 0) poisson (rng1, lambda=4) uniform (rng1, min=0, max=4) It would be best if numpy.random adopted this approach. Since I understand that's not likely, the alternative is for numpy.random to add some API that would allow direct access to the shared rng object.

2 1