Mailman 3 February 2002 - NumPy-Discussion

Interface for numpy C-API <-> simple C++ matrix class
by Stefan Heinrichs March 4, 2002

March 4, 2002

Hello, we would like to write routines processing numerical python arrays in C++, so that at least boundary checking can be enabled at runtime. While there are a lot of matrix libraries available for C++, I could not find the glue that interfaces such a library to the C-API of numerical python. Seamless access to a minimal C++ library would make the C++ part of programming much easier. Has anyone already written some wrapper/glue code? Thanks and best regards, Stefan -- ------------------------------------------------------------------- Email: Stefan.Heinrichs(a)uni-konstanz.de Address: Fakulaet fuer Physik, Universitaet Konstanz, Universitaetsstr.10, 78457 Konstanz, Germany Phone: +49 7531 88 3814

4 4

numarray interface and performance issues (for dot product and transpose)
by a.schmolck＠gmx.net March 1, 2002

March 1, 2002

Hi, Numeric is an impressively powerful and in many respects easy and comfortable to use package (e.g. it's sophisticated slicing operations, not to mention the power and elegance of the underlying python language) and one would hope that it can one day replace Matlab (which is both expensive and a nightmare as a programming language) as a standard platform for numerical calculations. Two important drawbacks of Numeric (python) compared to Matlab currently seem to be the plotting functionality and the availability of specialist libraries (say for pattern recognition problems etc.). However python seems to be quickly catching up in this area (e.g. scipy and other projects are bustling with activity and, as far as I know, there is currently a promising undergraduate project on the way at my university to provide powerful plotting facilities for python). There is however a problem that, for the use to which I want to put Numeric, runs deeper and provides me with quite a headache: Two essential matrix operations (matrix-multiplication and transposition (which is what I am mainly using) are both considerably a) less efficient and b) less notationally elegant under Numeric than under Matlab. Before I expound on that, I'd better mention two things: Firstly, I, have to confess largely ignorant about the gritty details of implementing efficient numerical computation (and indeed, I can't even claim to possess a good grasp of the underlying Linear Algebra). Secondly, and partly as a consequence, I realize that I'm likely to view this from a rather narrow perspective. Nonetheless, I need to solve these problems at least for myself, even if such a solution might be inappropriate for a more general audience, so I'd be very keen to hear what suggestions or experiences from other Numeric users are. I also want to share what my phd supervisor, Richard Everson, and I have come up with so far in terms of a), as I think it might be quite useful for other people, too (see [2], more about this later). Ok, so now let me proceed to state clearly what I mean by a) and b): The following Matlab fragment M * (C' * C) * V' * u currently has to be written like this in python: dot(dot(dot(M, dot(transpose(C), C)), transpose(v)), u) Or, even worse if one doesn't want to pollute the namespace: Numeric.dot(Numeric.dot(Numeric.dot(Numeric.M, Numeric.dot(Numeric.transpose(C), C)), Numeric.transpose(v)), u) Part a) (efficiency) -------------------- Apart from the syntactic inconveniences, the Matlab above will execute __considerably__ faster if C is a rather larger matrix (say 1000x1000). There are AFAIK two chief reasons for this: 1. Matlab uses a highly optimized ATLAS[3] BLAS routine to compute the dot-product, whereas Numeric uses a straight-forward home-brewn solution. 2. Numeric performs unnecessary transpose operations (prior to 20.3, I think, more about this later). The transpose operation is really damaging with big matrices, because it creates a complete copy, rather than trying to do something lazy (if your memory is already almost half filled up with (matrix) C, then creating a (in principle superfluous) transposed copy is not going to do you any good). The above C' * C actually creates, AFAIK, _3_ versions of C, 2 of them transposed (prior to 20.3; dot(a,b) translates into innerproduct(a, swapaxes(b, -1, -2)) In newer versions of Numeric, this is replaced by multiarray.matrixproduct(a, b) which has the considerable advantage that it doesn't create an unnecessary copy and the considerable disadvantage that it seems to be factor 3 or so slower than the old (already not blazingly fast) version for large Matrix x Matrix multiplication, (see timing results [1])). Now fortunately, we made some significant progress on the performance issues. My supervisor valiantly patched Numeric's multiarrayobject.c to use ATLAS for {scalar,vector,matrix} * {scalar,vector,matrix} multiplications. As a consequence the multiplication of pair of 1000x1000 matrices is computed more than _40_ times faster on my athlon machine (see [1], it seems to work fine but both the timing results as well as the code should be treated with a bit of caution). The BLAS-enabled dot takes almost exactly the same time as Matlab doing the same thing. In my case, this makes the difference between waiting several minutes and a day, and between deciding to use python or something faster but more painful. Although to benefit from this patch, one has to install ATLAS this is straight-forward (see [2] and [3] for pointers). In addition it is easy to install Lapack at the same time, as ATLAS provides significantly optimised versions of some of the Lapack routines too. Using Lapack rather than the lapack_lite that comes with Numeric also has the advantage that it is extensively tested and works very reliably (e.g. the lapack_lite routine that is called by Heigenvalues is pretty broken and searching the net suggests that this has been the case for some time now). This still leaves point 2), transposes, as a potential source for improvement. As far as I understand, the Lapack routines operate on flattened arrays, i.e. the only difference on calling some routine on a matrix and its transpose is that one has to pass a different additional parameters in the second case that specifies the structure of the matrix. Therefore if one were to program C' * C in fortran, only C itself needed to exist in memory. Would it therefore be possible that operations like transpose, slicing etc. be carried out lazily (i.e. so that only _modifying_ the result of the transpose or slice or passing it to some function for some reason needs a real copy resulted in an actual copy operation being performed)? As far as I understand the slicing in numarray is no longer simple aliasing (which is a good thing, since it make arrays more compatible with lists), are there any plans for implementing such an on-demand scheme? Part b) (syntax) ________________ As I said, dot(dot(dot(M, dot(transpose(C), C)), transpose(v)), u) is pretty obscure compared to M * (C' * C) * V' * u) Although linear algebra is not the be all and end all of numerical calculations, many numerical algorithms end up involving a good deal of linear algebra. The awkwardness of python/Numeric's notation for common linear algebra operation severely impairs, in my opinion, the utility of writing, using and debugging algorithms in python/Numeric. This is a particular shame given python's generally elegant and expresssive syntax. So far, I've thought of the following possible solutions: 0. Do nothing: Just live with the awkward syntax. 1. Comment: add a comment like the Matlab line above above complex expressions. 2. Wait for fix. Hope that either: 2.1. numarray will overload '*' as matrix multiplication, rather than elementwise multiplication (highly unlikely, I suspect). 2.2. the core python team accepts one of the proposed operator additions (or even, fond phantasy, allows users to add new operators) 3. Wrap: create a DotMatrix class that overloads '*' to be dot and maybe self.t to return the transpose -- this also means that all the numerical libraries I frequently use need to be wrapped. 4. Preprocess: Write some preprocessor that translates e.g. A ~* B in dot(A,B). 5. Hack: create own customized version of array that overloads e.g. '<<' to mean dot-product. The possible downsides: 0. Do nothing: Seems unacceptable (unless as a temporary solution), because the code becomes much less readable (and consequently harder to understand and more error-prone), thus forgoing one of the main reasons to choose python in the first place. 1. Comment: Unsatisfactory, because there is no way to gurantee that comment and code are in synch, which very likely will lead to difficult to find bugs. 2. Wait for a fix: Either would be nice (I could live with having to write multiply(a,b) -- I suppose however some other people can't, because I can't think of another reason why Numeric didn't overload * for matrixproduct in the first place, moreover it would mean a significant interface change (and code breakage), so I guess it is rather unlikely). From what I gather from searching the net and looking at peps there is also not much of a chance to get a new operator anytime soon. 3. Wrap: Promises to be a maintenance nightmare (BTW: will the new array class be subclassable?), but otherwise looks feasible. Has anyone done this? 4. Preprocess: Would have the advantage that I could always get back to "standard" python/Numeric code, but would involve quite a bit of work and might also break tools that parse python. 5. Hack array: Seems evil, but in principle feasible, because AFAIK '<<' isn't used in Numeric itself and hopefully it wouldn't involve too much work. However, given a free choice '<<' is hardly the operator I would choose to represent the dot product. I am not completely sure what the policy and rationale behind the current division of labor between functions, methods and attributes in numarray is but as far as the lack of a missing transposition postfix operator is concerned, one reasonable approach to make the transpose operation more readable (without the need to change anything in python itself) would seem to me to provide arrays with an attribute .t or .T so that: a.t == transpose(a) While this would admittedly be a bit of a syntactic hack, the operation is so commonplace and the gain in readability (in my eyes) is so significant that it would seem to me to merit serious consideration (maybe a.cc or so might also be an idea for complex conjugate notation, but I'm less convinced about that?). If the addition of a single operator for the exclusive benefit of Numeric users is rejected by the core python team, maybe it's worthwhile lobbying for some mechanism that allows users to define new operators (like e.g. in Haskell)... OK, that's all -- thanks for bearing with me through this long email. Suggestions and comments about the patch [2] and possible solutions to issues raised are greatly appreciated, regards, alexander schmolck Footnotes: [1] timing results for the patched version of Numeric[2] comparing new and old 'dot' performance: http://www.dcs.ex.ac.uk/~aschmolc/Numeric/TimingsForAtlasDot.txt [2] A patch for Numeric 21.1b that speeds up Numeric's dot function by several factors for large matrices can be found here: http://www.dcs.ex.ac.uk/~aschmolc/Numeric/ [3] ATLAS (http://math-atlas.sourceforge.net/) is a project to provide a highly optimized version of BLAS (Basic Linear Algebra Subroutines, a standard and thouroughly tested implementation of basic linear algebra operations) and some LAPACK routines. The charm of ATLAS is that it is platform-independent and yet highly optimized, which it achieves by essentially fine tuning a number of parameters until optimum performance for the _particular_ machine on which it is built is reached. As a consequence complete builds can take some time, but binary versions for ATLAS for common processors are available from http://www.netlib.org/atlas/archives (moreover, even if one decides to build ATLAS oneself, the search space can be considerably cut down if one accepts the suggested "experience" values offered during the make process). -- Alexander Schmolck Postgraduate Research Student Department of Computer Science University of Exeter A.Schmolck(a)gmx.net http://www.dcs.ex.ac.uk/people/aschmolc/

7 6

Numarray-0.2 issues with GCC and SPARC
by Todd Miller Feb. 28, 2002

Feb. 28, 2002

Hi, I'm Todd Miller and I work at STSCI on Numarray. Two people have reported problems with compiling Numarray-0.11 or 0.2 with GCC on SPARC. There are two problems: 1. Compiling the _ufuncmodule.c using gcc-2.95 on a SPARC (with the default switches) uses tons of virtual memory and typically fails. a. This can be avoided by adding the compilation flags: EXTRA_COMPILE_ARGS=["-O0", "-Wno-return-type"] to your setup.py when compiling *numarray*. b. Alternately, you can wait for numarray-0.3 which will partition the ufuncmodule into smaller compilation units. We suspect these will avoid the problem naturally and permit the use of optimization. c. Lastly, if you have Sun cc, you might want to try using it instead of gcc. This is what we do at STSCI. You need to recompile Python itself if you want to do this and your python was already compiled with gcc. 2. Python compiled with gcc generates misaligned storage within buffer objects. Numarray-0.2 is dependent on the problematic variant of the buffer object so if you want to use Float64 or Complex128 on a SPARC you may experience core dumps. a. I have a non-portable patch which worked for me with gcc-2.95 on SPARC. I can e-mail this to anyone interested. Apply this patch and recompile *python*. b. You might be able to fix this with gcc compilation switches for Python: try -munaligned-doubles and recompile *python*. c. Numarray-0.3 will address this issue by providing its own minimal memory object which features correctly aligned storage. This solution will not require recompiling python, but won't be available until numarray-0.3. d. Python compiled with Sun cc using the default switches doesn't manifest this bug. If you have Sun cc, you may want to recompile *python* using that. In general, I think the "better part of valor" is probably to wait 3 weeks for numarray-0.3 when both issues should be addressed. If you want to try numarray-0.2 now with GCC on a SPARC, I hope some of these ideas work for you. Todd -- Todd Miller jmiller(a)stsci.edu STSCI / SSG (410) 338 4576

1 0

Re: [Numpy-discussion] numarray interface and performance issues (for dot product and transpose)
by Krishnaswami, Neel Feb. 28, 2002

Feb. 28, 2002

a.schmolck(a)gmx.net [mailto:a.schmolck@gmx.net] wrote: > > Numeric is an impressively powerful and in many respects easy and > comfortable to use package (e.g. it's sophisticated slicing > operations, not to mention the power and elegance of the underlying > python language) and one would hope that it can one day replace Matlab > (which is both expensive and a nightmare as a programming language) as > a standard platform for numerical calculations. I'm in much the same boat, only with Gauss as the language I want to replace. > There is however a problem that, for the use to which I want > to put Numeric, runs deeper and provides me with quite a headache: > > Two essential matrix operations (matrix-multiplication and > transposition (which is what I am mainly using) are both considerably > > a) less efficient and > b) less notationally elegant > > under Numeric than under Matlab. These are my two problems as well. I can live with the clumsy function call interface to the matrix ops, but the loss of efficiency is a real killer for me. In my code, Gauss is about 8-10x faster than Numpy, which is a killer speed loss. (And Gauss is modestly slower than C, though I don't care about this because the Gauss is fast enough.) Right now, I have a data-mining program that I prototyped in Numpy and am now rewriting in C. Because Numpy isn't fast enough, I have wasted close to a week on this rewrite. This sounds bitter, but it's not meant to. I have to deploy on VMS, and after we had gotten Numpy working on OpenVMS I really hoped that the Alpha would fast enough that I could just use the Python prototype. -- Neel Krishnaswami neelk(a)cswcasa.com

1 0

SPECIAL INVITATION !
by paul Feb. 28, 2002

Feb. 28, 2002

<html> <head> <title></title> </head> <body bgcolor="#FFFFFF" text="#000000" leftmargin="0" topmargin="0" marginwidth="0" marginheight="0"> <table cellspacing="0" cellpadding="0" border="0" align="center"> <tr> <td width="16" height="9" valign="top"></td> <td width="77" height="9" valign="top"></td> <td width="157" height="9" valign="top"></td> <td width="158" height="9" valign="top"></td> <td width="72" height="9" valign="top"></td> <td width="20" height="9" valign="top"></td> </tr> <tr> <td width="16" height="32" valign="top"></td> <td width="77" height="32" valign="top"></td> <td width="315" height="32" colspan="2" valign="top"><img src="http://www.mdtrail.com/advertising/images/a-new.gif" width="315" height="30"></td> <td width="72" height="32" valign="top"></td> <td width="20" height="32" valign="top"></td> </tr> <tr> <td width="16" height="40" valign="top"></td> <td width="77" height="40" valign="top"></td> <td width="315" height="40" colspan="2" valign="top"><img src="http://www.mdtrail.com/advertising/images/prelaunch.gif" width="315" height="30"></td> <td width="72" height="40" valign="top"></td> <td width="20" height="40" valign="top"></td> </tr> <tr> <td width="250" height="91" colspan="3" valign="top"><img src="http://www.mdtrail.com/advertising/images/do-1.gif" width="250" height="91"></td> <td width="250" height="91" colspan="3" valign="top"><img src="http://www.mdtrail.com/advertising/images/do-2.gif" width="250" height="91"></td> </tr> <tr> <td width="250" height="90" colspan="3" valign="top"><img src="http://www.mdtrail.com/advertising/images/do-3.gif" width="250" height="90"></td> <td width="250" height="90" colspan="3" valign="top"><img src="http://www.mdtrail.com/advertising/images/do-4.gif" width="250" height="90"></td> </tr> <tr> <td width="16" height="34" valign="top"></td> <td width="464" height="34" colspan="4" valign="top"><img src="http://www.mdtrail.com/advertising/images/top.gif" width="464" height="28"></td> <td width="20" height="34" valign="top"></td> </tr> <tr> <td width="16" height="13" valign="top"></td> <td width="464" height="13" colspan="4" valign="top"><img src="http://www.mdtrail.com/advertising/images/table-up.gif" width="464" height="13"></td> <td width="20" height="13" valign="top"></td> </tr> <tr> <td width="16" height="115" valign="top"></td> <td width="464" height="115" colspan="4" valign="top" background="http://www.mdtrail.com/advertising/images/table-bg.gif"> <table width="95%" border="0" cellspacing="0" cellpadding="0" align="center"> <tr> <td> mdtrail.com Compensation Plan is calculated to create high returns with a steady flow of residual income for the rest of your life on commission payouts! Our PowerTrail referral program is designed to be easy for new participants to make back their investment very quickly and from then on...BIG PROFITS! mdtrail.com Compensation Plan is intended to be equitable to all from the start!! This will ENSURE that members stick with the program... This means real IMPACT to YOUR SHORT-TERM and LONG-TERM EARNINGS. mdtrail.com payout ratio is more than 80%. This is very critical to your success. This program is LAUNCHING SOON and offers state-of-the-art healthcare delivery systems, and a dedicated back end support team. It has taken more than 3 year to develop. AND a big investment was made in the preparation of this interactive health club, ALL for the success of YOU and YOUR PEOPLE !  Wouldn't you like to get a BIG piece of the action this time around? <img src="http://www.mdtrail.com/advertising/images/please1.gif" width="136" height="38" border="0"><img src="http://www.mdtrail.com/advertising/images/please2.gif" width="28" height="38" border="0"><a href="http://www.mdtrail.com"><img src="http://www.mdtrail.com/advertising/images/please3.gif" width="245" height="38" border="0"></a> <img src="http://www.mdtrail.com/advertising/images/needed.gif" width="378" height="54"> </td> </tr> </table> </td> <td width="20" height="115" valign="top"></td> </tr> <tr> <td width="16" height="9" valign="top"></td> <td width="464" height="9" colspan="4" valign="top"><img src="http://www.mdtrail.com/advertising/images/table-nottom.gif" width="464" height="9"></td> <td width="20" height="9" valign="top"></td> </tr> <tr> <td width="16" height="1" valign="top"><img width="16" height="1" src="transparent.gif"></td> <td width="77" height="1" valign="top"><img width="77" height="1" src="transparent.gif"></td> <td width="157" height="1" valign="top"><img width="157" height="1" src="transparent.gif"></td> <td width="158" height="1" valign="top"><img width="158" height="1" src="transparent.gif"></td> <td width="72" height="1" valign="top"><img width="72" height="1" src="transparent.gif"></td> <td width="20" height="1" valign="top"><img width="20" height="1" src="transparent.gif"></td> </tr> </table> </body> </html>

1 0

draft numarray manual available
by Perry Greenfield Feb. 27, 2002

Feb. 27, 2002

We have begun adapting the Numeric manual for numarray. An early draft of such a manual (in PDF format) is available from the numarray home page: http://stsdas.stsci.edu/numarray Comments and corrections are welcome. Over the next few months we will be adding chapters on how to interface C code to numarray as well as how to use non-numeric arrays. Perry Greenfield

3 5

last 21 beta
by Paul Dubois Feb. 27, 2002

Feb. 27, 2002

I've just put up Numeric-21.0b3.tar.gz; I intend this to be the final beta. Thanks to fellow developers for some recent bug fixes. Please give it a workout. Also, I need someone to try it with Python 2.1, as I didn't have one laying around and one of my fixes is for that. P.S. for Travis: Please put your name in the changes.txt if you fixed a bug, not the name of the person who complained unless they also supplied the fix. Thanks for knocking down some hard ones.)

1 0

PyFITS 0.6.2 available
by Perry Greenfield Feb. 27, 2002

Feb. 27, 2002

We are announcing the availability of PyFITS, a Python module to provide a means of reading and writing FITS format files. The FITS (Flexible Image Transport System) format is standardized data format widely used for astronomical data (http://fits.gsfc.nasa.gov/). This module is based on the PyFITS module initially developed by Paul Barrett while he was at NASA/Goddard but has since been modified and adapted for use by the Space Telescope Science Institute. It has been developed primarily by Paul Barrett, Jin-Chung Hsu, and Todd Miller, with assistance from Warren Hack, Phil Hodge, and Michele De La Pena. It is being released under an Open Source License. A web page for PyFITS is available: http://stsdas.stsci.edu/pyfits There is a preliminary draft for a PyFITS manual available from that page. It is still in the early stages and is incomplete (particularly with regard to the details of how to manipulate table objects). However, there should be enough information to indicate how to perform basic operations with FITS files. PyFITS requires that numarray v0.2 be installed (http://stsdas.stsci.edu/numarray). Numarray is a relatively new replacement for the Numeric module (and largely backward- compatible at the Python level). It also currently lacks all of the 3rd-party libraries that Numeric has (this will begin to change within a couple of months). However, it is not compatible at the C-API level nor are numarray and Numeric arrays interchangeable. It is possible to have both modules loaded simultaneously and to convert between numarray and Numeric arrays using the tostring/fromstring mechanism each provides (at the expense of extra memory usage, of course) The PyFITS module is simply a single Python file (pyfits.py). Installation consists of placing that file in a directory in the Python search path. Besides the use of numarray, there are significant differences with the interface provided by the original PyFITS. We do not expect many future backward-incompatible changes to the interface of PyFITS. (Though more methods and functions will almost certainly be added.) This is an early version and undoubtably bugs will be discovered when used with a greater variety of FITS files. Currently PyFITS take a fairly strict interpretation of FITS files. There are likely to be problems with FITS data that do not strictly conform to the standard. We intend to accommodate such variances, particularly if they involve widely used or available data (so please let us know when such problems occur). Things not yet supported but are part of future development: 1) Verification and/or correction of FITS objects being written to disk so that they are legal FITS. This is being added now and should be available in about a month. Currently, one may construct FITS headers that are inconsistent with the data and write such FITS objects to disk. Future versions will provide options to either a) correct discrepancies and warn, b) correct discrepancies silently, c) throw a Python exception, or d) write illegal FITS (for test purposes!). 2) Support for ascii tables or random groups format. Support for ASCII tables will be done soon (~1 month). When random group support is added is uncertain. 3) Support for memory mapping FITS data (to reduce memory demands). We expect to provide this capability in about 3 months. 4) Support for columns in binary tables having scaled values (e.g. BSCALE or BZERO) or boolean values. Currently booleans are stored as Int8 arrays and users must explicitly convert them into a boolean array. Likewise, scaled columns must be copied with scaling and offset by testing for those attributes explicitly. Future versions will produce such copies automatically. 5) Support for tables with TNULL values. This awaits an enhancement to numarray to support mask arrays (planned). (At least a couple months off) Please contact help(a)stsci.edu for with questions about it usage, bug reports, or requests for enhancements. Perry Greenfield Science Software Group

1 0

ANN: numarray-0.2
by Todd Miller Feb. 27, 2002

Feb. 27, 2002

Numarray 0.2 ------------ Numarray is a Numeric replacement which features c-code generated from python template scripts, the capacity to operate directly on arrays in files, and improved type promotion semantics. Numarray-0.2 incorporates bug fixes and one very significant change to the user interface, namely in how shape, real, imag and flat attributes are handled. Also, when numarray objects compared to None will (arr == None) no longer produce an execption and generate a false value (rather than a Boolean array). The next version will incorporate safety checks to prevent possible crashing of Python if a user misuses or otherwise changes private variables in numarray. There will also be a new memory object type to fix a problem discovered with buffer objects (which are currently used to used to allocate memory). We expect to have this version ready in 3 weeks. WHERE ----------- Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://stsdas.stsci.edu/numarray/index.html REQUIREMENTS -------------------------- numarray-0.2 requires Python 2.0 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, and Phil Hodge at the Space Telescope Science Institute. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. CHANGES-0.2: --------------------- 1. Added support for python-2.2 properties, specifically: .shape, getshape(), setshape() .flat, getflat() .real, getreal(), setreal() .imag, getimag(), setimag() .imaginary, getimaginary(), setimaginary() # aliases for imag numarray-0.2 is not 100% compatible with numarray-0.11. To port numarray-0.11 code to numarray-0.2: a. Instances of array.reshape(x) must be changed to array.setshape(x) or array.shape = x. Users with python versions prior to 2.2 must use array.setshape(x). b. Instances of array.real() must be replaced with array.real or array.getreal(). Users with python versions prior to 2.2 must use array.getreal(). c. Instances of array.imag() must be replaced with array.imag or array.getimag(). Users with python versions prior to 2.2 must use array.getimag(). 2. Fixed bugs in some of the numarray functions related to: a. Making copies of the input arrays when required to do so. b. Supporting nested sequences of python numbers in lieu of arrays. affected functions: reshape, transpose, resize, diagonal, trace, sort, argsort, ravel, ... 3. Fixed a bug in Complex arrays related to handling of type Complex64. This bug manifested as incorrect results for many operations with Complex64 arrays. "ones" in particular, returned obviously incorrect results for type=Complex64. 4. Fixed a bug in type conversion in "where" with y = where (equal (x, 0.), 1., x) on single precision array 'x' resulting in a double precision 'y'. This fix also affects "choose". 5. Added getrank() method and associated property "rank" for python2.2 and on. 6. Fixed a bug in nonzero where the "input screening" code was truncating small floating point values to 0. 7. Fixed a bug in all unary/binary ufuncs where output buffer offset was hard-coded to 0 for "fast" mode. This caused the following failure: >>> a=arange(10) >>> a[5:8] += 3 >>> a array([3, 4, 5, ... ]) 8. Fixed bug / added support for array([], type=ZZZ). 9. Added pi to the numarray namespace by importing it from math. 10. Added arrayrange alias for arange. 11. Improved numarray (in)equality testing by adding handling for None and re-implementing __nonzero__ as the bitwise-or of the element-wise comparison of the array with 0. 12. Fixed bug in iscontiguous() which assumed that _bytestride == _itemsize for contiguous arrays. This is not true when slicing occurs, but perhaps should be. 13. Added typecodes which are compatible with NumPy typecodes: "d":Float64,"f":Float32,"l":Int32,"D":Complex128,"F":Complex64, "b":UInt8, "c":Int8 Modified NumericType.__cmp__ to support comparisons against aliases. Modified NumericType.__repr__ to return the type name. 14. Modified the doctest for recarray so that it works correctly on win32. -- Todd Miller jmiller(a)stsci.edu STSCI / SSG (410) 338 4576

1 0

memory not being freed
by Mathew Yeates Feb. 26, 2002

Feb. 26, 2002

Hi I'm having problems with garbage collection I wrote an extension which creates an array and returns it foo() { arr = (PyArrayObject *) PyArray_FromDims( ..... ret = Py_BuildValue("O", arr); return ret; } but now if I do while 1: a=foo() memory is never free'd. I've even tried explicitly calling gc.collect and adding del(a) after a=foo. Is the problem that Py_BuildValue increases the reference count? Mathew

4 4