Mailman 3 April 2002 - NumPy-Discussion

[ANN] Pyfort 7.1
by Paul F Dubois 19 Apr '02

19 Apr '02

Pyfort 7.1 is available at sf.net/projects/pyfortran. Support for single Fortran characters was added. (Michiel de Hoon) Corrected behavior of scalars with C routines. (Michiel de Hoon) Pyfort is a tool for connecting Python to Fortran. Just to let you know, I'm working on a little tool to make it easier to set up simple projects so that you can build and install them with less effort. I hope to have that available soon.

1 0

The right way to use results of argmax and argmin
by victor＠idaccr.org 17 Apr '02

17 Apr '02

# I'm running python 2.0 on Solaris and Numeric 21.0 #I have an m by n array -- called a and have # j an n long list of integers in range(m), such as j = argmax(a,0) # If I set z = zip(j,range(len(j))) # and try the statement res = take(a,z) # python appears to hang, but if I do res = array(map(lambda x,a=a: a[x[0],x[1]]],z) # It works. # Is there a simpler way of doing what I want, and why does take hang? # is it, perhaps, allocating some n by n work array (this would # probably make things thrash like crazy)? -- Victor S. Miller | " ... Meanwhile, those of us who can compute can hardly victor(a)idaccr.org | be expected to keep writing papers saying 'I can do the CCR, Princeton, NJ | following useless calculation in 2 seconds', and indeed 08540 USA | what editor would publish them?" -- Oliver Atkin

1 0

Extension under windows
by mekkaoui 17 Apr '02

17 Apr '02

Dear Numerical Python Users, I have writen an extension using GSL (Gnu Scientific Library) and Numerical Python. This extension work fine under Linux and I would to do the same under Windows. For that I use Cygwin. When I would create the module $ gcc -shared Example.o -o Example.pyd I receive this message : Example.o<.text+0x58>:Example.c: undefined reference to 'PyArg_ParseTuple' Example.o<.text+0x15e>:Example.c: undefined reference to 'Py_BuildValue' Example.o<.text+0x1b1>:Example.c: undefined reference to 'Py_InitModule4' Example.o<.text+0x1c1>:Example.c: undefined reference to 'PyImport_ImportModule' Example.o<.text+0x1db>:Example.c: undefined reference to 'PyModule_GetDict' Example.o<.text+0x1f4>:Example.c: undefined reference to 'PyDict_GetItemString' Example.o<.text+0x206>:Example.c: undefined reference to 'PyCObject_Type' Example.o<.text+0x214>:Example.c: undefined reference to 'PyCObject_AsVoidPtr' Perhaps this command is wrong. Perhaps, anyone could explain or show me a document which explain the procedure clearly ? Thanks in advance for your help Omar

2 1

RE: [Numpy-discussion] Introduction
by Scott Gilbert 16 Apr '02

16 Apr '02

--- Perry Greenfield <perry(a)stsci.edu> wrote: > > If one had an NDArray that happened to contain a type that numarray > supported, yes it is possible (in fact RecArray does that sort of thing). > > If your point is that in doing so one must use the private attributes > such as _strides, yes that is true. > My point was simply: = One *can* convert from (NDArray + typecode) to a full NumArray = You *do* already convert lists, tuples, ... to NumArrays in ufuncs = So you *could* convert *(NDArrays + typecode) to NumArrays in ufuncs in the same place that checks to see if it is a list, tuple, ... Therefore: = You possibly *could* standardize the attributes in an NDArray (buffer, typecode, shape, stride, offset, ...) = If you *did* standardize the attributes, then others *could* build UserDefinedNDArrays however they see fit and they would work with NumArrays However I get the sense that the numarray module is your baby, and you don't want to change him too much. That's very understandable, you're a proud parent. Truth be told, he's a good looking kid, and I look forward to hanging out with him when he's all grown up. We just have a little different view on parenting, and I was hoping my kid would have an easier time playing with yours. Now that I've beaten that silly metaphor to death... :-) Cheers, -Scott ps: It occurs to me, with the strong sense of encapsulation you desire, that I could have presented this better as requesting that you specify a set of standard *methods* instead of attributes. Something like: def __array_getbuffer__(self): def __array_getoffset__(self): def __array_getshape__(self): def __array_getstrides__(self): def __array_getitemsize__(self): def __array_gettypecode__(self): def __array_getendian__(self): # Who knows what the real list would consist of... # We never got to discuss what a really general # purpose description of an NDArray would require... Then anything which implemented those standard *methods* would be a viable NDArray. From my point of view it amounts to about the same thing, but I think it's a better design and that you might like this idea more. However I'm getting out of breath on this topic, and I have other things I need to do (I'm sure this is true for you too), so if you don't see any merit in this idea, I won't push for it any further. Cheers again. __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/

2 1

Introduction
by Scott Gilbert 16 Apr '02

16 Apr '02

Hello All. I'm interested in this project, and am curious to what level you are willing to accept outside contribution. I just tried to subscribe to the developers list, but I didn't realize that required admin approval. Hopefully it doesn't look like I was shaking the door without knocking first. Is this list active? Is this the correct place to talk about Numarray? A little about me: My name is Scott Gilbert, and I work as a software developer for a company called Rincon Research in Tucson Arizona. We do a lot digital signal processing/analysis among other things. In the last year or so, we've started to use Python in various capacities, and we're hoping to use it for more things. We need a good array module for various things. Some are similar to what it looks like Numarray is targeted at (fft, convolutions, etc...), and others are pretty different (providing buffers for reading data from specialized hardware etc...) About a week ago, I noticed that Guido over in Python developer land was willing to accept patches to the standard array module. As such, I thought I would take that opportunity to try and wedge some desirements and requirements I have into that baseline. Bummer for me, but they weren't exactly exited about bloating out arraymodule.c to meet my needs, and in retrospect that does make good sense. A number of people suggested that this might be a better place to try and get what I need. So here I am, poking around and wondering if I can play in your sandbox. If you're willing to let me contribute, my specific itches that I need to scratch are below. Otherwise - bummer, and I hope you all catch crabs... :-) ----------------------------------- It's taken me a couple of days to understand what's going on in the source. I've read through the design docs, and the PEP, but it wasn't until I tried to re-implement it that it really clicked. My re-implementation of the array portion of what you're doing is attached. There are still some holes to fill in, but it's fairly complete and supports a whole bunch of things which yours does not (Some of which you might even find useful: Pickling, Bit type). I'm pretty proud of it for only 400 lines of Python (Most of which is the bazillion type declarations). It's probably riddled with bugs as it's less than a day old... After initially thinking that you guys were getting too clever, I've come to realize it's a pretty good design overall. Still I have some changes I would like to make if you'll let me. (Both to the design and the implementation) ------------------------- Following your design for the Array stuff, I've been able to implement a pretty usable array class that supports the bazillion array types I need (Bit, Complex Integer, etc...). This gets me past my core requirements without polluting your world, but unfortunately my new XArray type doesn't play so well with your UFuncs. I think my users will definitely want to use your UFuncs when the time comes, so I want to remedy this situation. The first change I would like to make is to rework your code that verifies that an object is a "usable" array. I think NumArray should only check for the interface required, not the actual type hierarchy. By this I mean that the minimum required to be a supported array type is that it support the correct attributes, not that it actually inherit from NDArray: (quoting from your paper) something like: _data _shape _strides _byteoffset _aligned _contiguous _type _byteswap Most of these are just integer fields, or tuples of integers. Ignoring _type for the moment, it appears that the interface required to be a NumArray is much less strict than actually requiring it to derive from NumArray. If you allow me to change a few functions (inputarray() in numarray.py is one small example), I could use my independant XArray class almost as is, and moreover I can implement new array objects (possibly as extension types) for crazy things like working with page aligned memory, memory mapping etc... Well, that's almost enough. The _type field poses a small problem of sorts. It looks like you don't require a _type to be derived from NumericType, and this is a good thing since it allows me (and others) to implement NumArray compatible arrays without actually requiring NumArray to be present. However, it would be nice if you declared a more comprehensive list of typenames - even if they aren't all implemented in NumArray proper. Who knows, maybe the SciPy guys have a use for complex integers or bit arrays. If you make a reasonable canonical list, our data could be passed back and forth even if NumArray doesn't know what to do with it. See my attached module for the types of things I'm thinking of. I'm not so concerned about the "Native Types" that are in there, but I think committing a list of named standard types. (I suspect there are others that are interested in standard C types even if the size changes between machines...) If you were to specify a minimal interface like this in the short term, I could begin propagating my array module to my users. I could get my work done now, knowing that I'll be compatible with NumArray proper once it matures. I'd be willing to participate in making these changes if necessary. Looking at the big picture, I think it's desirable that there really only be one official standard for ND arrays in the Python world. That way, the various independent groups can all share their independent work. You guys are the heir-apparent, so to speak, from the Python guys point of view. I don't know if you're trying to get all of NumArray into the Python distribution or not, but I suspect a good interim step would be to have a PEP that specifies what it means to be a NumArray or NDArray in minimal terms. Perhaps supplying an Array only module in Python that implements this interface. Again, I'd be willing to help with all of this. ------------------------- Ok, other suggestions... Here is the list of things that your design document indicates are required to be a NumArray: _data _shape _strides _byteoffset _aligned _contiguous _type _byteswap I believe that one could calculate the values for _aligned and _contiguous from the other fields. So they shouldn't really be part of the interface required. I suspect it is useful for the C implementation of UFuncs to have this information in the NDINfo struct though, so while I would drop them from attribute interface, I would delegate the task of calculating these values to getNDInfo() and/or getNumInfo(). I also notice that you chose _byteswap to indicate byteswapping is needed. I think a better choice would be to specify the endian-ness of the data (with an _endian attr), and have getNDInfo() and getNumInfo() calculte the _byteswap value for the NDInfo struct. In my implementation, I came up with a slightly different list: self._endian self._offset self._shape self._stride self._itemtype self._itemsize self._itemformat self._buffer The only minimal differences are that _itemsize allows me to work with arrays of bytes without having any clue what the underlying type is (in some cases, _itemtype is "Unknown".) Secondly, I implemented a "Struct" _itemtype, and _itemformat is useful for for this case. (It's the same format string that the struct module in Python uses.) Also, I specified 0 for _itemsize when the actual items aren't byte addressable. In my module, this only occurred with the Bit type. I figured specifying 0 like this could keep a UFunc that isn't Bit aware from stepping on memory that it isn't allowed to. ------------------------- Next thought: Memory Mapping I really like the idea of having Python objects that map huge files a piece at time without using all of available memory. I've seen this in NumArray's charter as part of the reason for breaking away from Numeric, and I'm curious how you intend to address it. Right now, the only requirement for _data seems to be that it implement the PyBufferProcs. For memory mapping something else is needed... I haven't implemented this, so take it as just my rambling thoughts: With the addition of 3 new, optional, attributes to the NumArray object interface, I think this could be efficiently accomplished: _mapproc _mapmin _mapmax If _mapproc is present and not None, then it points to a function who's responsibility it is to set _mapmin and _mapmax appropriately. _mapproc takes one argument which is the desired byte offset into the virtual array. This is probably easier to describe with code: def _mapproc(self, offset): unmap_the_old_range() mmap_a_new_range_that_includes_byteoffset() self._mapmin = minimum_of_new_range() self._mapmax = maximum_of_new_range() In this way, when the delta between _mapmin and _mapmax is large enough, the UFuncs could act over a large contiguous portion of the _data array at a time before another remapping is necessary. If the byteoffset that a UFunc needs to work with is outside of _mapmin and _mapmax, it must call _mapproc to remedy the situation. This puts a lot of work into UFuncs that choose to support this. I suppose that is tough to avoid though. Also, there are threading issues to think about here. I don't know if UFuncs are going to release the Global Interpreter Lock, but if they do it's possible that multiple threads could have the same PyObject and try to _mapproc different offsets at different times. It is possible to implement a mutex for the NumArray without requiring anything special from the PyObject that implements it... ----------------------------- Ok. That's probably way too much content for an Introductory email. I do have more thoughts on this stuff though. They'll just have to wait for another time. Nice to meet you all, -Scott Gilbert __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ import array as _array import struct as _struct class ItemType: def __repr__(self): return self._repr class UnsignedType: pass class SignedType: pass class FloatType: pass class SimpleType(ItemType): def getitem(self, array, offset): s = array._buffer[offset:offset+array._itemsize].tostring() return _struct.unpack(array._endian + self._format, s)[0] def setitem(self, array, offset, value): s = _struct.pack(array._endian + self._format, value) a = _array.array('b') a.fromstring(s) array._buffer[offset:offset+array._itemsize] = a class ComplexType(ItemType): def getitem(self, array, offset): s = array._buffer[offset:offset+array._itemsize].tostring() t = _struct.unpack(array._endian + self._format, s) return complex(t[0], t[1]) def setitem(self, array, offset, value): value = complex(value) re = value.real im = value.imag s = _struct.pack(array._endian + self._format, re, im) a = _array.array('b') a.fromstring(s) array._buffer[offset:offset+array._itemsize] = a class Bit(ItemType): _repr = 'Bit' _mask = (0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80) def getitem(self, array, offset): bi, mi = divmod(offset, 8) if array._buffer[bi] & self._mask[mi]: return 1 else : return 0 def setitem(self, array, offset, value): bi, mi = divmod(offset, 8) b = array._buffer[bi] if value: array._buffer[bi] = b | self._mask[mi] else: array._buffer[bi] = b & ~self._mask[mi] & 0xFF # Native Types class NativeChar(SimpleType, UnsignedType): _repr = 'NativeChar' _format = 'c' _size = _struct.calcsize(_format) class NativeByte(SimpleType, SignedType): _repr = 'NativeByte' _format = 'b' _size = _struct.calcsize(_format) class NativeUByte(SimpleType, UnsignedType): _repr = 'NativeUByte' _format = 'B' _size = _struct.calcsize(_format) class NativeShort(SimpleType, SignedType): _repr = 'NativeShort' _format = 'h' _size = _struct.calcsize(_format) class NativeUShort(SimpleType, UnsignedType): _repr = 'NativeUShort' _format = 'H' _size = _struct.calcsize(_format) class NativeInt(SimpleType, SignedType): _repr = 'NativeInt' _format = 'i' _size = _struct.calcsize(_format) class NativeUInt(SimpleType, UnsignedType): _repr = 'NativeUInt' _format = 'I' _size = _struct.calcsize(_format) class NativeLong(SimpleType, SignedType): _repr = 'NativeLong' _format = 'l' _size = _struct.calcsize(_format) class NativeULong(SimpleType, UnsignedType): _repr = 'NativeULong' _format = 'L' _size = _struct.calcsize(_format) class NativeLongLong(SimpleType, SignedType): _repr = 'NativeLongLong' _format = 'q' _size = _struct.calcsize(_format) class NativeULongLong(SimpleType, UnsignedType): _repr = 'NativeULongLong' _format = 'Q' _size = _struct.calcsize(_format) def _get_portable_uint_format(size): if _struct.calcsize('B') == size: return 'B' if _struct.calcsize('H') == size: return 'H' if _struct.calcsize('I') == size: return 'I' if _struct.calcsize('L') == size: return 'L' if _struct.calcsize('Q') == size: return 'Q' # we could raise an exception, but they might not need this type return NotImplemented def _get_portable_int_format(size): if _struct.calcsize('b') == size: return 'b' if _struct.calcsize('h') == size: return 'h' if _struct.calcsize('i') == size: return 'i' if _struct.calcsize('l') == size: return 'l' if _struct.calcsize('q') == size: return 'q' return NotImplemented # Portable Types class UInt8(SimpleType, UnsignedType): _repr = 'UInt8' _size = 1 _format = _get_portable_uint_format(_size) class UInt16(SimpleType, UnsignedType): _repr = 'UInt16' _size = 2 _format = _get_portable_uint_format(_size) class UInt32(SimpleType, UnsignedType): _repr = 'UInt32' _size = 4 _format = _get_portable_uint_format(_size) class UInt64(SimpleType, UnsignedType): _repr = 'UInt64' _size = 8 _format = _get_portable_int_format(_size) class Int8(SimpleType, SignedType): _repr = 'Int8' _size = 1 _format = _get_portable_int_format(_size) class Int16(SimpleType, SignedType): _repr = 'Int16' _size = 2 _format = _get_portable_int_format(_size) class Int32(SimpleType, SignedType): _repr = 'Int32' _size = 4 _format = _get_portable_int_format(_size) class Int64(SimpleType, SignedType): _repr = 'Int64' _size = 8 _format = _get_portable_int_format(_size) class Float32(SimpleType, FloatType): _repr = 'Float32' _size = 4 _format = 'f' class Float64(SimpleType, FloatType): _repr = 'Float64' _size = 8 _format = 'd' class ComplexInt8(ComplexType): _repr = 'ComplexInt8' _size = 2 _format = _get_portable_int_format(_size/2)*2 class ComplexInt16(ComplexType): _repr = 'ComplexInt16' _size = 4 _format = _get_portable_int_format(_size/2)*2 class ComplexInt32(ComplexType): _repr = 'ComplexInt32' _size = 8 _format = _get_portable_int_format(_size/2)*2 class ComplexInt64(ComplexType): _repr = 'ComplexInt64' _size = 16 _format = _get_portable_int_format(_size/2)*2 class ComplexFloat32(ComplexType, FloatType): _repr = 'ComplexFloat32' _size = 8 _format = 'ff' class ComplexFloat64(ComplexType, FloatType): _repr = 'ComplexFloat64' _size = 16 _format = 'dd' class Struct(ItemType): _repr = 'Struct' def getitem(self, array, offset): s = array._buffer[offset:offset+array._itemsize].tostring() return _struct.unpack(array._endian + array._itemformat, s) def setitem(self, array, offset, value): s = _struct.pack(array._endian + array._itemformat, *value) a = _array.array('b') a.fromstring(s) array._buffer[offset:offset+array._itemsize] = a class Unknown(ItemType): _repr = 'Unknown' def getitem(self, array, offset): return array._buffer[offset:offset+array._itemsize] def setitem(self, array, offset, value): array._buffer[offset:offset+array._itemsize] = value def _normalize_shape(shape): if isinstance(shape, (int, long)): shape = (shape,) return tuple(shape) __list4096 = None def _alloc_bytes(size): if (size > 4096): # This is a speed hack. The array module calls memcpy repeatedly # based on the sequence * operator. Dividing this by a decent # sized number really saves a lot of time. global __list4096 if __list4096 is None: __list4096 = range(4096) for i in xrange(4096): __list4096[i] = 0 d, m = divmod(size, 4096) if m: d += 1 return _array.array('b', __list4096)*d else: return _array.array('b', [0])*size def _get_item_count(shape): count = 1 for x in shape: count *= x return count def _get_std_stride(shape, itemsize): stride = list(shape) n = len(stride) for i in xrange(n): stride[i] = itemsize for j in xrange(i+1, n): stride[i] *= stride[j] return tuple(stride) class XArray: def __init__(self, shape, itemtype=None, itemsize=None, itemformat=None): if _struct.pack('@h', 1) == '\x01\x00': self._endian = '<' elif _struct.pack('@h', 1) == '\x00\x01': self._endian = '>' else: raise NotImplemented("what kind of endian is this machine?") self._shape = _normalize_shape(shape) self._offset = 0 self._itemformat = itemformat # should specify itemtype or itemsize, but not both if itemtype and itemsize: if not issubclass(itemtype, Unknown): raise ValueError("Can't specify both itemtype and itemsize") # default itemtype is Float64 if itemtype is None and itemsize is None: itemtype = Float64 # This indicates they specified an itemsize, but not an itemtype if itemtype is None: itemtype = Unknown self._itemtype = itemtype() # Bit is a special case if issubclass(itemtype, Bit): self.__initbool() return # Other special cases if issubclass(itemtype, Struct): self._itemsize = _struct.calcsize(self._endian + itemformat) elif issubclass(itemtype, Unknown): self._itemsize = itemsize else: self._itemsize = self._itemtype._size self._buffer = _alloc_bytes(self._itemsize*_get_item_count(self._shape)) self._stride = _get_std_stride(self._shape, self._itemsize) def __initbool(self): self._itemsize = 0 itemcount = _get_item_count(self._shape) if itemcount%8: self._buffer = _alloc_bytes(itemcount/8 + 1) else: self._buffer = _alloc_bytes(itemcount/8) self._stride = _get_std_stride(self._shape, 1) def __getstate__(self): return ( self._endian, self._offset, self._shape, self._stride, self._itemtype, self._itemsize, self._itemformat, self._buffer.tostring() ) def __setstate__(self, tup): ( self._endian, self._offset, self._shape, self._stride, self._itemtype, self._itemsize, self._itemformat, bufstr ) = tup self._buffer = _array.array('b') self._buffer.fromstring(bufstr) # TODO: Need to check that the _itemtype size matches the _itemsize # for this platform since the object may have been unpickled from # somewhere else. If it doesn't match, either raise an exception or # fix it if we can? def __repr__(self): # TODO: This should be made to mimic what Numarray/Numeric do return "XArray(shape=%s, itemtype=%s, itemsize=%s)" % ( self._shape, self._itemtype, self._itemsize) def __getitem__(self, subscripts): offset = self.__get_offset(subscripts) return self._itemtype.getitem(self, offset) def __setitem__(self, subscripts, value): offset = self.__get_offset(subscripts) self._itemtype.setitem(self, offset, value) def __get_offset(self, subscripts): offset = self._offset if isinstance(subscripts, (int, long)): subscripts = (subscripts,) n = len(self._stride) if len(subscripts) != n: raise IndexError("Subscript must be same dimension as array") for i in xrange(n): sub = subscripts[i] dim = self._shape[i] if sub < 0: sub += dim if sub < 0 or sub >= dim: raise IndexError("Subscript out of range") offset += self._stride[i]*sub return offset class XArray_SliceByReference(XArray): # TODO: overload __getitem__, __setitem__ to share the _buffer with # a new stride and shape. pass class XArray_SliceByCopyOnWrite(XArray): # TODO: overload __getitem__, same as XArraySliceByReference, but # need to overload __setitem__ to make a copy and detach. pass if __name__ == '__main__': x1 = XArray(20) print x1 print x1[2] x1[2] = 100 x1[3] = 200 print x1[0], x1[2], x1[3] a2 = XArray((20, 30), itemtype=ComplexFloat64) a2[4, 5] = 999 print a2[4, 5] x3 = XArray((100, 100, 20)) print x3 x4 = XArray((1000, 1000), itemtype=Bit) print x4 print x4[999, 999] x4[999, 100] = 1 print x4[999, 100] x5 = XArray((10, 10, 10, 10), itemtype=Struct, itemformat='cHLd') print x5, x5[1, 2, 3, 4] x5[1, 2, 3, 4] = ('x', 32000, 100, 1.0) print x5, x5[1, 2, 3, 4]

3 13

ANN: Numarray-0.3.1 and 0.3.2
by Todd Miller 15 Apr '02

15 Apr '02

Numarray 0.3.1 and 0.3.2 --------------------------------- Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modelled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, and improved type promotions. Numarray-0.3.1 incorporates a number of bug fixes and enhancements to the C-API, including a minimal Numeric emulation layer which makes it easy to port simple Numeric C-extensions to numarray. The emulation layer is incomplete, so not all Numeric extensions will work, but simple ones *do* with a minimal amount of effort. See Doc/numpy_compat for an example of convolution done using the emulation layer. New for Numarray-0.3.1 is the Numarray manual in PDF and HTML formats; other formats are available for users if the source distribution. Numarray-0.3.2 is a source only release to support Alpha/Tru64. It is essentially Numarray-0.3.1 + one portability bug fix. WHERE ----------- Numarray-0.3.1 windows executable installers and source code tar ball is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://stsdas.stsci.edu/numarray/index.html Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS -------------------------- numarray-0.3.1 requires Python 2.0 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. Thanks go to Jochen Kupper of the University of North Carolina for his work on Numarray and for porting the Numarray manual to TeX format. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller(a)stsci.edu

1 0

segfault in Numpy esxtension
by kragen＠pobox.com 13 Apr '02

13 Apr '02

(All of the below is with regard to Numeric 20.2.0.) For a consulting client, I wrote a extension module that does the equivalent of sum(take(a, b)), but without the temporary result in between. I was surprised that when I tried to .resize() the result of this routine, I got a segmentation fault and a core dump. It was crashing at this line in arrayobject.c: if (memcmp(self->descr->zero, all_zero, elsize) == 0) { self->descr, in this case, was the type description for arrays of type "double". It seems that self->descr->zero was 0, as in a null pointer, not a pointer to a location containing (double)0, and this was causing it to crash. It looks like the .zero fields of the type descriptions (which live in arraytypes.c and _numpy.so) are initialized to be null pointers, and only when the initmultiarray() function in multiarraymodule.c is run are these pointers set to point to actual zeroes somewhere in allocated memory. I guess Numeric.py imports multiarray.so, which calls initmultiarray(), so the solution for me was to make sure I import Numeric before importing my module (or at least before resizing arrays produced by my module). But, to my mind, this segfault is a bug --- importing a module that follows all the rules shouldn't put Python in a state that's so dangerously inconsistent that innocent things like .resize() can crash it. Maybe the same .so file that includes the actual data items should be responsible for initializing them --- especially since import_array() imports _numpy without importing multiarray. (I assume there's a reason it wasn't done this way in the first place.) What do other people think? -- /* By Kragen Sitaker, http://pobox.com/~kragen/puzzle4.html */ char b[2][10000],*s,*t=b,*d,*e=b+1,**p;main(int c,char**v){int n=atoi(v[1]); strcpy(b,v[2]);while(n--){for(s=t,d=e;*s;s++){for(p=v+3;*p;p++)if(**p==*s){ strcpy(d,*p+2);d+=strlen(d);goto x;}*d++=*s;x:}s=t;t=e;e=s;*d++=0;}puts(t);}

1 0

slice question and bug
by clee＠spiralis.merseine.nu 11 Apr '02

11 Apr '02

Hello, I'm trying to track down a segv when I do the B[:] operation on an array, "B", a that I've built in as a view on external data. During the process I ran into the following code (Numeric-21.0): /* {%c++%} */ extern int PyArray_Free(PyObject *op, char *ptr) { PyArrayObject *ap = (PyArrayObject *)op; int i, n; if (ap->nd > 2) return -1; if (ap->nd == 3) { n = ap->dimensions[0]; for (i=0; i<n; i++) { free(((char **)ptr)[i]); } } if (ap->nd >= 2) { free(ptr); } Py_DECREF(ap); return 0; } /* {%c++%} */ The multiple, incompatible tests of ap->nd are the problem. -chris

5 5

Puzzling numpy results?
by Andrew McNamara 08 Apr '02

08 Apr '02

The behavior I'm seeing with zero length Numeric arrays is not what I would have expected: >>> from Numeric import * >>> array([5]) != array([]) zeros((0,), 'l') >>> array([]) == array([]) zeros((0,), 'l') >>> allclose(array([5]), array([])) 1 This is with Numeric-20.3 (and Numeric-20.2.1) - is this behavior correct, or have I stumbled across a bug? If both sides of the comparison are arrays with a length greater than zero, the comparisons work as expected: >>> array([5]) != array([6]) array([1]) >>> array([5, 5]) != array([6]) array([1, 1]) >>> array([5]) != array([5]) array([0]) The problem came up when I was writing unittests for some Numpy code: under some circumstances, the code under test is expected to return a zero length array: I was somewhat surprised when I couldn't make the test fail! 8-) -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

2 1

numerical integration
by Jochen Küpper 06 Apr '02

06 Apr '02

The following message is a courtesy copy of an article that has been posted to comp.lang.python.announce as well. I have made a numerical intergation package available at ,---- | http://python.jochen-kuepper.de/integrate `---- This is a copy of the integrate module of scipy by Travis Oliphant plus some small changes and rearrangements to make it work standalone (well, it need Numeric). All credits go to the scipy folks, esp. Travis, all errors should be blamed on me. Greetings, Jochen PS: In the long run this module will be phased out in favor of scipy, but for now it might be useful for someone... -- Einigkeit und Recht und Freiheit http://www.Jochen-Kuepper.de Liberté, Égalité, Fraternité GnuPG key: 44BCCD8E Sex, drugs and rock-n-roll

1 0