Mailman 3 November 2005 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon 17 May '11

17 May '11

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

scipy cygwin compilation error
by Jordan Dawe 04 Dec '05

04 Dec '05

I just tried to install scipy under cygwin using both 0.61 and the current svn. I get this: gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o -L/usr/lib/python2.4/config -lpython2.4 -o build/lib.cygwin-1.5.18-i686-2.4/scipy/base/umath.dll build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `UBYTE_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2215: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `USHORT_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2233: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `UINT_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2251: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `ULONG_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2269: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `BYTE_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2305: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o:/usr/local/core/build/src/scipy/base/src/umathmodule.c:2325: more undefined references to `_feraiseexcept' follow build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `PyUFunc_checkfperr': /usr/local/core/scipy/base/src/ufuncobject.c:475: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:475: undefined reference to `_feclearexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `PyUFunc_clearfperr': /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_feclearexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `PyUFunc_GenericFunction': /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_feclearexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `construct_reduce': /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_feclearexcept' collect2: ld returned 1 exit status build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `UBYTE_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2215: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `USHORT_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2233: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `UINT_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2251: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `ULONG_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2269: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `BYTE_multiply': /usr/local/core/build/src/scipy/base/src/umathmodule.c:2305: undefined reference to `_feraiseexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o:/usr/local/core/build/src/scipy/base/src/umathmodule.c:2325: more undefined references to `_feraiseexcept' follow build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `PyUFunc_checkfperr': /usr/local/core/scipy/base/src/ufuncobject.c:475: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:475: undefined reference to `_feclearexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `PyUFunc_clearfperr': /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_feclearexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `PyUFunc_GenericFunction': /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_feclearexcept' build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o: In function `construct_reduce': /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_fetestexcept' /usr/local/core/scipy/base/src/ufuncobject.c:506: undefined reference to `_feclearexcept' collect2: ld returned 1 exit status error: Command "gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.5.18-i686-2.4/build/src/scipy/base/src/umathmodule.o -L/usr/lib/python2.4/config -lpython2.4 -o build/lib.cygwin-1.5.18-i686-2.4/scipy/base/umath.dll" failed with exit status 1 Any suggestions? Jordan

2 1

Numeric3.0 - Currently available under scipy/base as version 0.4.1 (Windows)
by Colin J. Williams 04 Dec '05

04 Dec '05

I have a package based on subclassing numarray, which is working satisfactorily, and am looking at the possibility of transferring the package to work under the revised Numeric. My feeling is that the transfer is probably feasible but that it would be premature to work on it at this time. One of the problems is the cluttered namespace, through the use of "from X import *". This is a style which is deprecated, see page 401 of Alex Martelli's /Python in a Nutshell/. Another problem, at this stage, is that many doc strings are missing and that some which exist are a little cryptic. I would like to take another look when the next win32 binaries are available. Some further thoughts on the present state of Numeric3.0 are available here <http://www3.sympatico.ca/cjw/scipy1/>. Colin W,

2 3

Your PayPal Account Could Be Suspended
by PayPal 29 Nov '05

29 Nov '05

1 0

Re: help in improving data analysis code
by gf 25 Nov '05

25 Nov '05

From: Francesc Altet <faltet@ca...> Re: help in improving data analysis code 2005-11-25 07:32 >>A Divendres 25 Novembre 2005 16:27, Francesc Altet va escriure: > > print nn[argsort(abs(nn_c-nn_c.mean()),0)][:-int(sz*0.10),0] >> >> Ups. I have had a confusion. This should work better ;-) >> >> print nn[argsort(abs(nn-nn.mean()),0)][:-int(sz*0.10),0] Hi Francesc, Thank you for the suggestions. Your code is performing a different task than mine was. In particular, I believe it does not 're-mean' the data after removing each point. However, based on the great ideas from your code, I now have the function below that looks to be more efficient (although I haven't measured it). Any additional suggestions are appreciated. -g ==== from numarray import argsort, floor, absolute def eliminate_outliers(data,frac): num_to_eliminate = int(floor(data.size())*frac) for i in range(num_to_eliminate): data = data[argsort(absolute(data-data.mean()),0)][:-1,0] return data if __name__ == "__main__": from numarray.mlab import rand sz = 100 nn = rand(sz,1) nn[:10] = 20*rand(10,1) nn[sz-10:] = -20*rand(10,1) print eliminate_outliers(nn,0.10)

1 0

help in improving data analysis code
by gf 25 Nov '05

25 Nov '05

Hi, I am a newbie to Numeric/numarray programming and would appreciate your help in improving the code below (which I'm sure is quite ugly to an experienced numarray programmer). An analysis we are carrying out requires the following: 1. evaluate the mean of a set of data 2. eliminate the data point farthest from the mean 3. repeat steps 1 and 2 until a certain specified fraction of points has been eliminated. Since this analysis will have to be performed (probably repeatedly) on approximately ten thousand data sets, each of which contains 100-500 points, I would like the code to be as fast as possible. Thanks for your help. -g ==== from numarray import add, array, asarray, absolute, argsort, floor, take, size def mean(m,axis=0): m = asarray(m) return add.reduce(m,axis)/float(m.shape[axis]) def eliminate_outliers(dat,frac): num_to_eliminate = int(floor(size(dat,0)*frac)) for i in range(num_to_eliminate): ind = argsort(absolute(dat-mean(dat)),0) sdat = take(dat,ind,0)[:,0] dat = sdat[:-1] return dat #-------------------------------------------------------------------- if __name__ == "__main__": from MLab import rand sz = 100 nn = rand(sz,1) nn[:10] = 20*rand(10,1) nn[sz-10:] = -20*rand(10,1) print eliminate_outliers(nn,0.10)

2 2

Problem compiling scipy-core
by Anoop 23 Nov '05

23 Nov '05

Hi. I have problems installing scipy-core from source code on a Rocks Cluster distribution of Linux running on x86-64 hardware. The error messages tell me: /usr/bin/g77 -shared build/temp.linux-x86_64-2.3/scipy/corelib/blasdot/_dotblas.o -L/usr/lib -lblas -lg2c -o build/lib.linux-x86_64-2.3/scipy/lib/_dotblas.so /usr/bin/ld: skipping incompatible /usr/lib/libblas.so when searching for -lblas /usr/bin/ld: skipping incompatible /usr/lib/libblas.a when searching for -lblas /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../libblas.so when searching for -lblas /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../libblas.a when searching for -lblas /usr/bin/ld: skipping incompatible /usr/lib/libblas.so when searching for -lblas /usr/bin/ld: skipping incompatible /usr/lib/libblas.a when searching for -lblas /usr/bin/ld: cannot find -lblas collect2: ld returned 1 exit status I really need to get this running. Help? Thanks, Anoop

2 1

ANN: PyTables 1.2
by Francesc Altet 23 Nov '05

23 Nov '05

PyTables is a library to deal with very large datasets. It leverages the excellent HDF5 and numarray libraries to allow doing that in a very efficient way using the Python language. More info in: http://pytables.sourceforge.net/ ========================= Announcing PyTables 1.2 ========================= The PyTables development team is happy to announce the availability of a new major version of PyTables package. This version sports a completely new in-memory tree implementation based around a *node cache system*. This system loads nodes only when needed and unloads them when they are rarely used. The new feature allows the opening and creation of HDF5 files with large hierarchies very quickly and with a low memory consumption (the object tree is no longer completely loaded in-memory), while retaining all the powerful browsing capabilities of the previous implementation of the object tree. You can read more about the dings and bells of the new cache system in: http://www.carabos.com/downloads/pytables/NewObjectTreeCache.pdf Also, Jeff Whitaker has kindly contributed a new module called tables.NetCDF. It is designed to be used as a drop-in replacement for Scientific.IO.NetCDF, with only minor actions to existing code. Also, if you have the Scientific.IO.NetCDF module installed, it allows to do conversions between HDF5 <--> NetCDF3 formats. Go to the PyTables web site for downloading the beast: http://pytables.sourceforge.net/ If you want more info about this release, please check out the more comprehensive announcement message available in: http://www.carabos.com/downloads/pytables/ANNOUNCE-1.2.html Acknowledgments =============== Thanks to the users who provided feature improvements, patches, bug reports, support and suggestions. See THANKS file in distribution package for a (incomplete) list of contributors. Many thanks also to SourceForge who have helped to make and distribute this package! And last but not least, a big thank you to THG (http://www.hdfgroup.org/) for sponsoring many of the new features recently introduced in PyTables. --- **Enjoy data!** -- The PyTables Team -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

3 5

Problems with linalg.cholesky?
by Andrew Jaffe 22 Nov '05

22 Nov '05

[Originally posted to g.c.p.user...] Hi all, In the newest incarnation of scipy_core, I am having trouble with the cholesky(a) routine: /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/scipy/linalg/basic_lite.py in cholesky_decomposition(a) 115 else: 116 lapack_routine = lapack_lite.dpotrf --> 117 results = lapack_routine('L', n, a, m, 0) 118 if results['info'] > 0: 119 raise LinAlgError, 'Matrix is not positive definite - Cholesky decomposition cannot be computed' LapackError: Parameter a is not contiguous in lapack_lite.dpotrf But this isn't true; I get this error even when I pass trivial and contiguous matrices such as the output of identity(). Other linalg routines (included complicated ones like singular_value_decomp) seem to work fine. Any ideas? Andrew

3 2

Memory mapped files in scipy core
by Travis Oliphant 21 Nov '05

21 Nov '05

I would appreciate understanding typically use cases for memory-mapped files. I am not sure I understand why certain choices were made for numarray's memmap and memmap slice classes. They seem like a lot of "extra" stuff and I'm not sure what all that stuff is for. Rather than just copy these over, I would like to understand what people typically want to do with memory-mapped files to see if scipy core doesn't already provide that. For example, write now I can open a file, use mmap to obtain a memory map object and then pass that object into frombuffer in scipy_core to get an ndarray whose memory maps a file on disk. Now, this ndarray can be sliced and indexed and manipulated all the while referring to the file on disk (well technically, I suppose, the memory-mapped object would need to be flushed to synchronize). Now, I could see wanting to make the process of opening the file, getting the mmap object and setting it's buffer to the array object a little easier. Thus, a simple memmap class would be a useful construct -- I could even see it inheriting from the ndarray directly and adding a few methods. I guess I just don't see why one would care about a memory-mapped slice object, when the mmaparray sub-class would be perfectly useful. On a related, but orthogonal note: My understanding is that using memory-mapped files for *very* large files will require modification to the mmap module in Python --- something I think we should push. One part of that process would be to add the C-struct array interface to the mmap module and the buffer object -- perhaps this is how we get the array interface into Python quickly. Then, if we could make a base-type mmap that did not use the buffer interface or the sequence interface (similar to the bigndarray in scipy_core) and therefore by-passed the problems with Python in those areas, then the current mmap object could inherit from the base class and provide current functionality while still exposing the array interface for access to >2GB files on 64-bit systems. Who would like to take up the ball for modifying mmap in Python in this fashion? -Travis

3 2

The file DEVELOPERS.TXT
by Colin J. Williams 21 Nov '05

21 Nov '05

scipy._import_tools.py has a comment referring to DEVELOPERS.TXT but I don't find it in the Windows download.

2 1

OT: Enthought IT Admin Job Posting
by Travis N. Vaught 21 Nov '05

21 Nov '05

All, I'm posting this here in the hopes that someone that is passionate about python/numeric/scipy will "answer the call": `Enthought, Inc. <http://www.enthought.com>`__ (Austin, TX, USA) ==================================================================== **Job Description**: Position: IT Administrator --------------------------- Enthought is looking for an exceptional IT Administrator to manage the IT infrastructure for it's Austin, TX offices. This person will have a passion for supporting software development tools and environments. The target server platforms include Linux (RedHat and Fedora), Windows XP, and Solaris. Workstations are a mix of Windows and Linux. We're looking for an Admin that focuses on supporting software developers. Desired Skills and Capabilities: * B.S. in Computer Science of other related field (preferably not MIS) * 5+ Years Experience in Enterprise IT Administration Role * Ability to program or script in a programming language or shell - (Python, Perl and small C programs) * Ability to solve problems quickly and completely * Strong inter-personal and communication skills as well as a team player in a group of highly talented software developers * Willingness to pitch in wherever needed * Interested in making developer's lives easier Duties: * Perform installation, patching and other server maintenance to ensure security and stability of server infrastructure. * Maintain core infrastructure technologies such as Firewall, VPN, apache web server, mail server, NIS, NFS, DNS, DHCP, SSH, network-wide backups, RAID and Samba * Identify routine tasks and automate through shell/Python scripting * Perform on-call duties and out of hours maintenance as necessary * Configure Nortel phone system * Build RPMs for RH/FC Linux * Using development tools on Linux such as gcc/g++, make, autoconf, etc. * Working with Python and building & installing Python packages using distutils * Support developer tools such as SVN repositories, bug trackers, Software Project Management utilities Company: -------- Enthought is a scientific computing company located in downtown Austin, Texas. Founded in 2001, it has grown nearly 100% per year both in staff and revenue to become a stable and profitable technology company. This growth has been possible because of Enthought’s talented team and because of our commitment to developing quality software. We strive to combine advanced algorithm development with modern software practices, such as component based architectures, application scripting, and intuitive user interface design. We take a holistic approach to software development, in which architects and developers team with technical writers, human factors specialists, and project managers (always highly technical individuals) to develop a complete solution for our customers. Much of our work is based on the Python programming language, and we are actively engaged in open source development (www.scipy.org <http://www.scipy.org>). We’re lucky enough to work on interesting problems and are looking for talented people to join us. Some of our current efforts are in the areas of geophysics, electromagnetics, fluid dynamics, micro-rheology, CAD, 2-D and 3-D visualization, and others. All of these tools are developed as plug-ins into our Envisage "Scientific IDE" framework. * **Contact**: Travis N. Vaught, CEO * **E-mail contact**: jobs(a)enthought.com * **Web**: http://www.enthought.com/careers.htm -- ........................ Travis N. Vaught CEO Enthought, Inc. http://www.enthought.com ........................

1 0

Real life examples
by Eduardo Stark 20 Nov '05

20 Nov '05

Dear naumarray and numpy community: Could you tell me where can I get real life examples to apply numarray. I am a Python and numarray begineer. Thanks Eduardo Stark --------------------------------- Correo Yahoo! Comprueba qué es nuevo, aquí http://correo.yahoo.es

1 0

Real life examples
by Eduardo Stark 20 Nov '05

20 Nov '05

Dear naumarray and numpy community: Could you tell me where can I get real life examples to apply numarray. I am a Python and numarray begineer. Thanks Eduardo Stark --------------------------------- Correo Yahoo! Comprueba qué es nuevo, aquí http://correo.yahoo.es

1 0

compilation error on OSX 10.4
by Curzio Basso 19 Nov '05

19 Nov '05

Hi all, I get the following error compiling an extension module using numarray 1.4.1 on Mac OSX 10.4: ----- building 'fmlio' extension gcc-3.3 -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -DLINUX -I/home/basso/usr/include/python -I/sw/include/python2.4 -c src/fmlio.cpp -o build/temp.darwin-8.3.0-Power_Macintosh-2.4/src/fmlio.o In file included from /sw/include/python2.4/numarray/libnumarray.h:17, from /sw/include/python2.4/numarray/numarray.h:6, from src/fmlio.h:10, from src/fmlio.cpp:2: /sw/include/python2.4/numarray/nummacro.h:27: error: parse error before `;' token error: command 'gcc-3.3' failed with exit status 1 ----- the same happens with gcc-4. On Gentoo I can compile without problems. Did anyone else get a similar problem? Ah, and numarray.h is included right after Python.h. cheers curzio

2 3

numarray CVS seems to have an __array_struct__ bug
by Andrew Straw 17 Nov '05

17 Nov '05

Hi, I think there's a bug in the new __array_struct__ code in numarray CVS. Consider an object a which implements the __array_struct__ interface. If one constructs a numarray "view" of this array using aview = numarray.asarray(a), the numarray.ndarray instance does not keep a reference to a.__array_struct__. Thus, a segfault may occur because a.data can be freed although aview may attempt to access it. In particular code like this will crash: buf = numarray.asarray( array_struct_factory() ) # buffer is freed buf[1] = 2 # attempt is made to write to the non-existant buffer, causing a segfault I've "fixed" this issue with the following, which I'm sure is not the right way, but it's all I can do at this point. diff -u -r1.130 numarraycore.py --- Lib/numarraycore.py 4 Nov 2005 20:27:11 -0000 1.130 +++ Lib/numarraycore.py 17 Nov 2005 11:38:29 -0000 @@ -357,6 +357,7 @@ a = None if hasattr(sequence, "__array_struct__"): a = _numarray._array_from_array_struct(sequence.__array_struct__) + a._hack_to_hold_reference_ = sequence.__array_struct__ elif (hasattr(sequence, "__array_shape__") and hasattr(sequence, "__array_typestr__")): typestr = sequence.__array_typestr__ And while I'm at it, can the "from distutils.command.config import log" be turned off in setup.py? I like to see those errors, warnings, and status strings whiz past...

3 2

compiling 24.2 on solaris 8
by John Hunter 16 Nov '05

16 Nov '05

To compile numpy 24.2 on solaris 8, I needed to make the following change to Packages/RNG/Src/ranf.c #if !defined(__sgi) //int gettimeofday(struct timeval *, struct timezone *); //old int gettimeofday(); //new #endif Not sure why __sgi is defined... JDH

1 0

Call for a Scientific/Engineering Tutorial or Two at PyCon 2006
by Jeff Rush 15 Nov '05

15 Nov '05

Greetings. I'm working with the Python Conference planning group, for a conference to be held in Dallas in 2006. As you probably know from postings to python-announce, this year, for the first time, we're having a full day (Feb 23, 2006) of trainer-compensated tutorials, structured as either half-day or full-day sessions. From the feedback we received at last year's PyCon, there was a strong interest in scientific/engineering uses of Python, which has a rich set of tools and frameworks. We hope this year to dedicate one room on Tutorial Day to such topics. While I was looking over the submissions for the tutorials tonight, I noticed there were none related to science/engineering. It would be cool to have a half-day class on one of the underlying packages used across disciplines. Note that these are to be professionally-run half or full day classes, suggested with handouts or notebooks. And the trainer will receive monetary compensation, depending upon the number of attendees that register for specific tutorials. If you or someone on your staff is interested, there is more information at: http://us.pycon.org/TX2006/CallForTutorials While the submission deadline expires tonight, we have some flexibility. Jeff Rush

1 0

FIFO/circular buffer class in Numeric?
by RayS 14 Nov '05

14 Nov '05

Has anyone implemented a fast, generic circular array buffer in Numpy? I didn't see one in the usual searches, but I don't want to re-invent one either. I did see a number of C implementations like http://www.codeproject.com/csharp/circularbuffer.asp (http://www.gois.ws/files/attach/200209/200209201850040.zip) which could be weave'd, I suppose. I have some previous implementations I've used, but what I had in mind was making a new FIFO class with attributes of address, state<None,reading,writing>, pointer, size etc., and methods like flush (if I implement it as a memmap) and other things, like queue has. The important part of the idea is that multiple threads and processes could fully access it. Currently, I will have an ADC process fill it, and other processes use it. I'll post code as I go, in any case. Ray

1 0

Numeric 24.2 build fails
by N. Volbers 12 Nov '05

12 Nov '05

Sorry to once again report a build error ;-) Numeric 24.1 builds just fine, while Numeric 24.2 produces the attaced messages (out.txt) and error messages (err.txt). HTH, Niklas. /usr/lib/gcc-lib/i486-slackware-linux/3.3.4/../../../../i486-slackware-linux/bin/ld: cannot find -llapack collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 running build running build_py package init file 'Include/__init__.py' not found (or not a regular file) package init file 'Include/__init__.py' not found (or not a regular file) running build_ext building 'lapack_lite' extension gcc -pthread -shared build/temp.linux-i686-2.4/Src/lapack_litemodule.o -L/usr/lib/atlas -llapack -lcblas -lf77blas -latlas -lg2c -o build/lib.linux-i686-2.4/lapack_lite.so

1 0

A small glitch remains in CVS numarray
by Francesc Altet 12 Nov '05

12 Nov '05

Hi, I've checked CVS numarray today and yes, the problem with strided arrays seems to be gone. It does remain some small (I think) issue: >>> import Numeric;Numeric.__version__ '24.1' >>> import numarray;numarray.__version__ '1.4.2' >>> num=Numeric.array([0.,1.]) >>> na=numarray.zeros(shape=2) >>> na[...] = num Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: NA_DescrFromType: unknown type: -1 and also: >>> numarray.array(num) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line 359, in array a = _numarray._array_from_array_struct(sequence.__array_struct__) TypeError: NA_DescrFromType: unknown type: -1 Regards, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

2 1

SciPy SVN altered so that newscipy and newcore are now the trunk.
by Travis Oliphant 10 Nov '05

10 Nov '05

I've finished altering the subversion repository of SciPy so that the new development is taking place on the trunk of both scipy and scipy_core. The old versions are under branches named oldcore and oldscipy. Get the new repositor(y,ies) using: *Core*: svn co http://svn.scipy.org/svn/scipy_core/trunk core *Full SciPy*: svn co http://svn.scipy.org/svn/scipy/trunk scipy Doing both will place two directories named core and scipy in your current directory containing the current state of both repositories. python setup.py install should work in each directory. The Freeze is now over. I want to track down the bug that Christopher Hanley noted and another f2py-related bug before making a release, which I expect to happen by the weekend. -Travis

2 1

Re: [Numpy-discussion] Writing/reading a numeric array to a file
by Niklas Volbers 10 Nov '05

10 Nov '05

First of all, thanks for your answers. Travis Oliphant <oliphant(a)ee.byu.edu> schrieb am 09.11.05 20:23:16: > What unneeded functionality is there. SciPy Core is just a Numeric > replacement. It does not NEED Atlas, it just uses it if you have it --- > exactly the same as Numeric and numarray. > It is a misconception to say scipy core needs any thing else but Python > installed. So, let's not let that rumor kill convergence to a single > package. I am sorry for this faulty assumption of mine which you have now clarified. I had assumed that ATLAS was needed, just because scipy_core would not build on my system w/o a prior install of these libraries. With the knowledge, that it _should_ work, I have just retried to build scipy_core 0.4.2 on my rather fresh installation of Slackware Linux 10.2 (python 2.4.1) and still couldn't get it to build. I have attached the log files out.txt and err.txt to this e-mail, which I got by typing $ python setup.py build 1>out.txt 2>err.txt in the scipy_core main directory. The two files are compressed into a single tar.gz archive (5k), because I do not know if 60k attachments are o.k. on this list. Maybe you can help me solve my problem? > > (4) What solutions do you use to save/load data to files? > > SciPy core arrays have tofile methods and a fromfile function. They are > raw reading and writing --- nothing fancy. You need to use Pickling if > you want to recognize endian-ness among platforms. What is your > opposition to Pickling? >From how I understood pickling, it converts the python objects to a binary representation. What I want is a data representation that is independent of python. Anyway, I now have a few possibilities to explore besides netcdf (with the alternative file formats FITS and with pytables, i.e. HDF5, proposed here). > I think you should take a look at PyTables for more elegant solutions. I wasn't sure if it will stay compatible to scipy_core, because right now it uses numarray. But from what you said I am sure that eventually pytables might also switch to scipy_core. Best regards, and thanks for your efforts, Niklas. ______________________________________________________________________ XXL-Speicher, PC-Virenschutz, Spartarife & mehr: Nur im WEB.DE Club! Jetzt gratis testen! http://freemail.web.de/home/landingpad/?mc=021130

4 3

BUG!
by Sune Rastad Bahn 10 Nov '05

10 Nov '05

Dear devs, Thanks a lot for a very nice and extremely useful python module. I recommend all my colleagues to try you package whenever they ask for a matlab like environment. This week one of these friends came by me and showed me a bug in RandomArray. try the following python -c "from RandomArray import *; seed(1874764637, 1152239787);print randint(0,1000)" and notice how the result is the upper limit (1000) even though the documentation clearly states that the number should be strictly below 1000. My guess is that this is an issue with the underlying ranlib library, but perhaps someone can clarify? best, Sune -- Software Application Manager Sune Rastad Bahn Phone: +45 35 320 638 Mobile: +45 23 455 997 Email: bahn(a)atomistix.com

3 2

Re: [Numpy-discussion] Yet another problem in CVS numarray
by Todd Miller 10 Nov '05

10 Nov '05

Francesc Altet wrote: >El dv 04 de 11 del 2005 a les 15:35 -0500, en/na Todd Miller va >escriure: > > >>This turned out to be a problem with the way numarray handles Numeric's >>multi-segment buffer protocol. I worked around this by implementing >>David Cooke's __array_struct__ array interface for numarray. >> >> > >Did you commit the changes in CVS? I'm getting the same problems with >the current CVS version: > > > >>>>Numeric.__version__ >>>> >>>> >'24.1' > > >>>>import numarray >>>>numarray.__version__ >>>> >>>> >'1.4.2' > > >>>>num=Numeric.array([1,2,3,4]) >>>>numarray.array(num) >>>> >>>> >array([1, 2, 3, 4]) > > >>>>num2=num[::2] >>>>numarray.array(num2) >>>> >>>> >Traceback (most recent call last): > File "<stdin>", line 1, in ? > File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line >380, in array > a = a.astype(type) > File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line >867, in astype > return self.copy() > File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line >927, in copy > c = _gen.NDArray.copy(self) > File "/usr/lib/python2.4/site-packages/numarray/generic.py", line 724, >in copy > arr._itemsize) >numarray.libnumarray.error: copy4bytes: access beyond buffer. offset=11 >buffersize=8 > > This turned out to be a bug in numarray buffer size determination... striding wasn't accounted for so the buffer appeared to be too small. It's fixed now in CVS. There's still a backward compatibility problem for old Numerics which don't implement __array_struct__ and get multi-segment buffers through __array_data__. Regards, Todd

2 3

Writing/reading a numeric array to a file
by N. Volbers 10 Nov '05

10 Nov '05

Hello everyone, I have been using Numeric and/or numarray for a while now and I now I have a question concerning the reading and writing data to a file. Until now I had used pycdf, which is an interface to the netcdf library, to save and load my numeric arrays. However, since my application already has too many dependencies, I would like to cut down on these two and replace the saving and loading by a method intrinisc to Numeric or numarray. Pickling is not an alternative to me, because I need to have a representation that can be read by other programs (and yet is faster than ASCII). NetCDF was a good choice for that, but I would really like to not depend on it. So my questions are: (1) Are the numarray functions 'fromfile' and 'tofile' 100% portable among platforms, i.e. do they automatically recognize endian-ness and such? (2) Does it make sense to still use numarray? I know Travis would say "use scipy_core". However, for me this would provide much unneeded functionality and I have not yet found an easy way to install scipy_core (it seems to require ATLAS and such, which are not so easy to install if you don't have it prepackaged). And after all, my goal is to cut down dependencies... (3) Let me restate question (2): Will numarray still be maintained? Or is it also deprecated? What would you advice someone who just needs the array interface? And of course, (4) What solutions do you use to save/load data to files? I would deeply appreciate any help on this subject, Niklas Volbers.

5 4

Bug or feature?
by Sune Rastad Bahn 10 Nov '05

10 Nov '05

Dear devs, Thanks a lot for a very nice and extremely useful python module. I recommend all my colleagues to try you package whenever they ask for a matlab like environment. This week one of these friends came by me with an oddity relating to the RandomArray module. try the following on your linux command prompt: while true; do python -c "from RandomArray import *;seed();print randint(100000)"; done and check the output. I get a steadily but slowly decreasing series of numbers. e.g: 4013 4013 4012 4012 4012 4012 4012 4012 4012 4012 4010 4010 4010 According to the documentation seed() should initiatilize from the system clock. If one checks the seed (using get_seed) it turns out to be the case indeed, but that it is only very slowly changing (once a second), and only on the second seed. e.g python -c "from RandomArray import *;seed();print get_seed()" (113161, 3079) (113161, 3079) (113161, 3080) (113161, 3080) (113161, 3080) (113161, 3080) My best guess is that this is the cause for the behavior seen above. This could be filed as a feature, but I believe it is not the behavior people would expect. At least the documentation should emphasize that two runs started within a short time ( upto 1s) will give the exact same result (experts on pseudo random numbers will know this, but the length of the time 1sec is surprisingly long). that the first random number is slowly varying (strongly correlated) with time (this is more surprising). One fix is to take one randomnumber and throw away, since the second number is not correlated with time (except from noted above). Best, Sune -- Software Application Manager Sune Rastad Bahn Phone: +45 35 320 638 Mobile: +45 23 455 997 Email: bahn(a)atomistix.com

1 0

Python egg support in Numeric 24.2 in CVS
by David M. Cooke 09 Nov '05

09 Nov '05

I've added support to Numeric to allow a Python egg [1] to be made easily. Basically, run $ python setup.py bdist_egg and an .egg file should be laid in dist/, which you can install with easy_install. This will only work if you have setuptools installed. I've taken steps so that usual 'setup.py build; setup.py install' will still install Numeric the old way (with a .pth file). The header files for Numeric are now also installed in a package Numeric_headers, which contains a function get_numeric_include(). This returns the directory in which the headers are installed in, so you can use it in your setup.py. The docstring for the function explains how to use it. It's modelled after get_scipy_include() in scipy.base. This way, the header files are included in the egg, and can be used. I'd appreciate it if someone with eggperience (Robert?) could take a look at it before Travis collects Numeric for 24.2. [1] http://peak.telecommunity.com/DevCenter/PythonEggs -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm(a)physics.mcmaster.ca

1 0

Numeric CVS __array_struct__ interface is broken on 64 bit platforms
by Andrew Straw 09 Nov '05

09 Nov '05

Hi, A couple bugs have been reported (including mine last night) which indicate a problem with the following bit of code in Numerical/Src/arrayobject.c (near line 2200) on 64 bit platforms. I think it'll be a lot faster for someone else to fix it, so I'll leave it at this for now. if (strcmp(name, "__array_struct__") == 0) { PyArrayInterface *inter; inter = (PyArrayInterface *)malloc(sizeof(PyArrayInterface)); inter->version = 2; inter->nd = self->nd; if ((inter->nd == 0) || (sizeof(int) == sizeof(Py_intptr_t))) { inter->shape = (Py_intptr_t *)self->dimensions; inter->strides = (Py_intptr_t *)self->strides; } else { int i; for (i=0; i<self->nd; i++) { inter->shape[i] = self->dimensions[i]; inter->strides[i] = self->strides[i]; } }

1 0

initializing new ufuncs
by Stefan Kuzminski 08 Nov '05

08 Nov '05

Hi, I'm writing some new ufuncs, and I'm wondering where I can insert some initialization and finalization code for each ufunc invocation. I can see where it would go in the generated code, but I don't know how to do it through the code generation API. thanks, Stefan Kuzminski __________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs

1 0

Final release of Numeric is actually 24.1
by Travis Oliphant 08 Nov '05

08 Nov '05

I made a final release of Numeric (24.1). This is "really" the last release of Numeric ;-) I did it because of David Cooke's excellent array_protocol enhancement of __array_struct__. I wanted to make sure the final Numeric fully supported the array protocol including the "faster version." I also tracked down a bug today in scipy core that was inherited from Numeric. A part of me wanted to not fix the bug in Numeric so that people who need a stable platform will move to scipy core :-) But the better part of me won, and I fixed the problem and made a new Numeric release. I do hope people are encouraged to move toward scipy core, however. It is stabilizing quite rapidly. All of scipy now builds with the new scipy core, and all of (full) scipy's 1300 tests pass for both 32-bit and 64-bit systems. I will be making a release of scipy_core (probably called version 0.6 this weekend). It is still missing finished records.py, ma.py, and chararray.py modules (being worked on). When these are available I will make release of scipy core version 1.0 Best regards, -Travis

3 4

Numeric 24.1 tar ball and CVS were not consistent
by Travis Oliphant 07 Nov '05

07 Nov '05

I just updated the CVS of Numeric with changes that went into the Numeric 24.1 release but were (unfortunately) not committed to CVS. These changes are likely responsible for the test failures that people have experienced when testing scipy with CVS versions of Numeric. Sorry about --- too many things to think about apparently... -Travis

2 1

bus error in check_kurtosis
by Rob Managan 07 Nov '05

07 Nov '05

With todays svn versions (newcore 1440, newscipy 1423) scipy.test(level=1,verbosity=2) dies with a bus error on check_kurtosis check_basic (scipy.stats.stats.test_stats.test_mean) ... ok check_ravel (scipy.stats.stats.test_stats.test_mean) ... ok check_basic (scipy.stats.stats.test_stats.test_median) ... ok check_basic (scipy.stats.stats.test_stats.test_mode) ... ok check_kurtosis (scipy.stats.stats.test_stats.test_moments)Bus error This is on Mac OSX 10.3.9, python 2.4.1, gcc 3.3. Is this a known problem? -- *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- Rob Managan email managan at llnl.gov LLNL phone: 925-423-0903 P.O. Box 808, L-095 FAX: 925-422-3389 Livermore, CA 94551-0808

1 0

numpy 24.x on 64-bit Athlon
by Darren Dale 07 Nov '05

07 Nov '05

I just built Numeric 24.0 and 24.1 against ATLAS on a 64 bit athlon system. I get the following error for both packages. python test.py ........E.................................... ====================================================================== ERROR: Test the diagonal function. ---------------------------------------------------------------------- Traceback (most recent call last): File "test.py", line 584, in testDiagonal assert_eq(Numeric.diagonal(c,1), [[2,7,4], [2,7,4]]) File "test.py", line 28, in assert_eq assert eq(a,b) File "test.py", line 23, in eq raise ValueError("sequences have different shapes:\na%s=%r\nb%s=%r" % ValueError: sequences have different shapes: a(4, 2)=array([[5, 1], [6, 2], [7, 3], [8, 4]]) b(2, 3)=[[2, 7, 4], [2, 7, 4]] I also get the following error when I run newscipy's test(5,10): bench_random (scipy.fftpack.basic.test_basic.test_fft) Fast Fourier Transform ================================================= | real input | complex input ------------------------------------------------- size | scipy | Numeric | scipy | Numeric ------------------------------------------------- 100 | 0.10Segmentation fault Has anyone else had a problem building Numeric-24.x on a 64-bit AMD system? Thanks, Darren

2 3

question about dotblas and ACML
by Darren Dale 05 Nov '05

05 Nov '05

I just got a new Athlon 64 bit system. I'm in the process of building Numeric-24.1 against the AMD Core Math Library. Does anyone know if it is possible to enable the use_dotblas option in customize.py with ACML? Thanks, Darren

1 0

Yet another problem in CVS numarray
by Francesc Altet 04 Nov '05

04 Nov '05

Hi, I've detected another problem in CVS numarray when converting non-contiguous Numeric objects. With numarray 1.3.3 the next works: >>> Numeric.__version__ '23.8' >>> numarray.__version__ '1.3.3' >>> num=Numeric.array([1,2,3,4],'b') >>> num2=num[::2] >>> num2.iscontiguous() 0 >>> na=numarray.array(num2) >>> na array([1, 3]) but, with CVS version of numarray: >>> numarray.__version__ '1.4.2' >>> Numeric.__version__ '24.1' >>> na=numarray.array([1,2,3,4],'b') >>> num=Numeric.array([1,2,3,4],'b') >>> num2=num[::2] >>> num2.iscontiguous() 0 >>> na=numarray.array(num2) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line 376, in array a = a.astype(type) File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line 863, in astype return self.copy() File "/usr/lib/python2.4/site-packages/numarray/numarraycore.py", line 923, in copy c = _gen.NDArray.copy(self) File "/usr/lib/python2.4/site-packages/numarray/generic.py", line 724, in copy arr._itemsize) numarray.libnumarray.error: copy1bytes: access beyond buffer. offset=2 buffersize=2 Cheers, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

2 1

How to install SciPy Core
by Shu Li 04 Nov '05

04 Nov '05

Hi, I am a little confused about how to install SciPy Core. I am new to SciPy so I think starting with the new core is a good idea. When I installed the core package first and then scipy, in the process of installing scipy, it showed it also installed a lot of stuff to the core directory which made me worry that part of the new core is overwritten by old core. And when I installed scipy first, then some words on the web saying some "__init__.py" file will "break the scipy" certainly worried me because I don't know what the break here means and why people are not fixing it if it is avoidable. Also even when new core(0.4.2 beta) is installed the scipy still complained not seeing "Numeric", which added to the confusion. Could anybody offer some explanation? Thanks a lot! -- best, Sam (AKA Shu Li)

1 0

Options for wrapping C and C++ code for use with Numeric
by Chris Barker 04 Nov '05

04 Nov '05

Hi all, I suddenly seem to have the need for working with a bunch of different existing C and C++ code, so I'm looking for a way to make it easier. I love NumPy, so I really want to use NumPy arrays My needs fall into three categories: 1) Writing custom extension code from scratch: In the past, I've done this by just using the NumPy API, but it seems that I shouldn't have to do all that book keeping myself. 2) Wrapping existing libraries: At the moment, I'd like to wrap Richard Shewchuk's triangle: http://www.cs.cmu.edu/~quake/triangle.html Which is straight C, as well as the Proj4 map projections library, also C, and there may be others. 3) Wrapping in house code, C & C++, that is under development, but I want to be able to use it from Python and C++, and also perhaps to write tests for it in Python. The options I'm looking at are: Pyrex: This seems like perhaps the easiest way to write extension code, but it doesn't do any automatic wrapping. Boost::Python: This looks like a very nice way to write extension modules, and it even appears to already include support for NumPy arrays (which ones? is that support being maintained, and will it support SciPy.base?) SWIG: The has the major advantage of automatically wrapping code. I see this as a particular strength for wrapping our in house code that is under development. In theory, once I've written a bunch of type maps, i can just re-run it whenever the code base changes. For each of these, I'd love to hear what people's experiences have been, and it would be great if anyone can point me to samples that are small but non-trivial. Other options I should consider would be great too. One more question: Should I use NumPy arrays to pass data back and forth between Python and C, or should I use the new array object/protocol instead? If so, has anyone developed examples, SWIG type maps, etc for this? Thanks for any feedback, -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

6 6

Re: Adams to you
by Chrissy Lanctot 04 Nov '05

04 Nov '05

Go overpay or your Meddicat isit our Pharm ss Sh od days, Quit ing f ions - v aExpre op. Get additional information - http://starterxlq.presetey.com

1 0

Options for wrapping C and C++ code for use with Numeric
by RayS 03 Nov '05

03 Nov '05

Have you looked at ctypes code generator? http://starship.python.net/crew/theller/ctypes/codegen.html "The generator converts declarations in C header files into executable Python code: enums, structs, unions, function declarations, com interfaces, and preprocessor definitions." It has progressed well in the last several months, in my experience. I did some tests a while back that found it compared well with weave etc. http://sourceforge.net/mailarchive/forum.php?thread_id=7636431&forum_id=246… Ray Schumacher

1 0

More on array protocol
by Francesc Altet 03 Nov '05

03 Nov '05

Hi, I'm having problems in using the array protocol in numarray-->Numeric in some situations. I'm using the PyBuffer_FromReadWriteMemory C call in order to create buffers that will be later used to create numarray objects (both NumArray and CharArray) in Python space. The problem is that the array protocol implementation is doing badly this conversion. I was able to create a small example in pure python that reproduces the problem: >>> import numarray >>> from numarray import memory >>> import Numeric >>> na=numarray.array([1,2]) >>> Numeric.array(na) array([1, 2]) but ... >>> wbuf = memory.writeable_buffer(na) >>> nawb = numarray.array(wbuf,type="Int32",shape=(2,)) >>> nawb array([1, 2]) >>> Numeric.array(nawb) array([ 2, 135510112]) I'm afraid that the problem should be related with the kind of buffer that is attached to the numarray object: >>> na._data <memory at 0x082577f8 with size:0x00000008 held by object 0x401f10a0 aliasing object 0x00000000> >>> nawb._data <read-write buffer for 0x401f10a0, size -1, offset 0 at 0x40572020> Travis, Todd, are you going to support read-write buffers or should I try to move to using the memory module in order to cope with this? For a series of reasons I'd like to keep using regular read-write buffers, but anyway... I think I'll be able to do the change if absolutely necessary. Thanks, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

2 1

byteswap questions
by Russell E. Owen 03 Nov '05

03 Nov '05

I've got a module that can send array data to ds9. It screws up on byteswapped data and I'm trying to fix it. The code sends the data to ds9 via a pipe using arr.tofile(). To send byteswapped data I think I have two choices (given that I don't want to modify the input array): 1) Figure out the byte order and specify a suitable flag to ds9 (it accepts "bigendian" or "littleendian" flags). This avoids copying the data, so it sounds attractive, but can I do it without relying on "internals"? Numarrays have an "isbyteswapped" method, but all this says is "not native machine order"; it doesn't actually tell me the order. I need to know if the order is bigendian or littleendian, and I can't find a documented way to get that, just an undocumented attribute _byteorder. 2) If the array is not in native byte order, make a native byte ordered copy before sending it. The following seems to work: if arr.isbyteswapped(): arr = arr.copy() but is this the recommended way to get native-order copy? - Is "copy" guaranteed to return a copy in native byte order? The documentation doesn't say. - I was tempted to use num.array(arr) to make the copy, but the documentation doesn't say what "array" does if the input is a already a numarray array. As an aside, I tried to use the byteswapped method, but it has a totally different effect: >>> d array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.], ...type=Float32) >>> d.isbyteswapped() 1 >>> dcopy = d.copy() >>> d array([[-9012., -9012., -9012., ..., -9012., -9012., -9012.], ...type=Float32) >>> dcopy.isbyteswapped() 0 >>> dswap = d.byteswapped() >>> dswap array([[ 1.91063654e-38, 1.91063654e-38, 1.91063654e-38, ..., ...type=Float32) which I guess means I'd have to byteswap the byteswapped data. Aargh. -- Russell

4 5

Problem with array assignment
by Francesc Altet 03 Nov '05

03 Nov '05

Hi, I've recently upgraded to Numeric 24.0 and numarray 1.4.1 and I'm having a problem that did not exist before: >>> import Numeric >>> import numarray >>> Numeric.__version__ '24.0' >>> numarray.__version__ '1.4.1' >>> na=numarray.zeros((3,),'f') >>> num=Numeric.zeros((3,),'f') >>> na[...] = num Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: array sizes must be consistent. but... >>> na=numarray.zeros((3,),'i') >>> num=Numeric.zeros((3,),'i') >>> na[...] = num >>> na array([0, 0, 0]) ????? However, using Numeric 23.8 and numarray 1.3.3: >>> import Numeric >>> import numarray >>> Numeric.__version__ '23.8' >>> numarray.__version__ '1.3.3' >>> na=numarray.zeros((3,),'f') >>> num=Numeric.zeros((3,),'f') >>> na[...] = num >>> na array([ 0., 0., 0.], type=Float32) I don't know if the problem is coming from Numeric or numarray, but it only happens when one mix both: >>> Numeric.__version__ '24.0' >>> numarray.__version__ '1.4.1' >>> na=numarray.zeros((3,),'f') >>> na2=numarray.zeros((3,),'f') >>> na2[...] = na >>> na2 array([ 0., 0., 0.], type=Float32) >>> num=Numeric.zeros((3,),'f') >>> num2=Numeric.zeros((3,),'f') >>> num2[...] = num >>> num array([ 0., 0., 0.],'f') Perhaps the new array protocol is involved here? -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

2 3

1.4.1 build failure under OSX
by Chris Fonnesbeck 03 Nov '05

03 Nov '05

Hi, Compiling 1.4.1 under OSX with either gcc 3.3 or 4.0 fails (all previous versions of numarray compiled without error). Here is the full output: Using EXTRA_COMPILE_ARGS = ['-Ddarwin'] Using external BLAS and LAPACK running config Wrote config.h In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Src/_convmodule.c:13: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Src/_sortmodule.c:13: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Src/_bytesmodule.c:13: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, ... several dozen of the above, then ... Packages/LinearAlgebra2/Src/lapack_litemodule.c:12: warning: function declaration isn't a prototype Packages/LinearAlgebra2/Src/lapack_litemodule.c:17: warning: function declaration isn't a prototype Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dgeev': Packages/LinearAlgebra2/Src/lapack_litemodule.c:107: warning: implicit declaration of function 'dgeev_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dsyevd': Packages/LinearAlgebra2/Src/lapack_litemodule.c:199: warning: implicit declaration of function 'dsyevd_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zheevd': Packages/LinearAlgebra2/Src/lapack_litemodule.c:296: warning: implicit declaration of function 'zheevd_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dgelsd': Packages/LinearAlgebra2/Src/lapack_litemodule.c:340: warning: implicit declaration of function 'dgelsd_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dgesv': Packages/LinearAlgebra2/Src/lapack_litemodule.c:378: warning: implicit declaration of function 'dgesv_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dgesdd': Packages/LinearAlgebra2/Src/lapack_litemodule.c:418: warning: implicit declaration of function 'dgesdd_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dgetrf': Packages/LinearAlgebra2/Src/lapack_litemodule.c:449: warning: implicit declaration of function 'dgetrf_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zungqr': Packages/LinearAlgebra2/Src/lapack_litemodule.c:477: warning: implicit declaration of function 'zungqr_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dorgqr': Packages/LinearAlgebra2/Src/lapack_litemodule.c:508: warning: implicit declaration of function 'dorgqr_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zgeqrf': Packages/LinearAlgebra2/Src/lapack_litemodule.c:539: warning: implicit declaration of function 'zgeqrf_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dgeqrf': Packages/LinearAlgebra2/Src/lapack_litemodule.c:570: warning: implicit declaration of function 'dgeqrf_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_dpotrf': Packages/LinearAlgebra2/Src/lapack_litemodule.c:594: warning: implicit declaration of function 'dpotrf_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zgeev': Packages/LinearAlgebra2/Src/lapack_litemodule.c:632: warning: implicit declaration of function 'zgeev_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zgelsd': Packages/LinearAlgebra2/Src/lapack_litemodule.c:679: warning: implicit declaration of function 'zgelsd_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zgesv': Packages/LinearAlgebra2/Src/lapack_litemodule.c:714: warning: implicit declaration of function 'zgesv_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zgesdd': Packages/LinearAlgebra2/Src/lapack_litemodule.c:759: warning: implicit declaration of function 'zgesdd_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zgetrf': Packages/LinearAlgebra2/Src/lapack_litemodule.c:794: warning: implicit declaration of function 'zgetrf_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: In function 'lapack_lite_zpotrf': Packages/LinearAlgebra2/Src/lapack_litemodule.c:818: warning: implicit declaration of function 'zpotrf_' Packages/LinearAlgebra2/Src/lapack_litemodule.c: At top level: Packages/LinearAlgebra2/Src/lapack_litemodule.c:851: warning: function declaration isn't a prototype Packages/LinearAlgebra2/Src/lapack_litemodule.c:848: warning: 'lapack_liteError' defined but not used powerpc-apple-darwin8-gcc-4.0.0: -framework: linker input file unused because linking not done powerpc-apple-darwin8-gcc-4.0.0: vecLib: linker input file unused because linking not done In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Packages/RandomArray2/Src/ranlib.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Packages/RandomArray2/Src/ranlibmodule.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Packages/RandomArray2/Src/com.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Src/_dotblas.c:12: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from Src/_dotblas.c:15: /System/Library/Frameworks/vecLib.framework/Headers/cblas.h:665: warning: function declaration isn't a prototype In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Examples/convolve/high_levelmodule.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Examples/convolve/element_wisemodule.c:3: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Examples/convolve/one_dimensionalmodule.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Examples/convolve/numpy_compatmodule.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Examples/convolve/numpy_compat2.c:1: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list In file included from /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/Python.h:55, from Examples/convolve/testlite.c:26: /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Library/Frameworks/Python.framework/Versions/2.4/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list Any ideas for a solution to this problem are welcome. -- Chris Fonnesbeck Atlanta, GA

2 1

enumerate indices and values of a multidimensionnal numarray.array
by Philou 02 Nov '05

02 Nov '05

Hi list, I'm a new subsciber to this numpy list. I have a multidimensionnal array (>2). For example: x = numarray.array([[[1,2,3],[4,5,6]],[[1,2,3],[4,5,6]]]) print x [[[1 2 3] [4 5 6]] [[1 2 3] [4 5 6]]] I need to enumerate both indices and final values (1D) of the x array. So i need: (0,0,0) 1 (0,0,1) 2 (0,0,2) 2 (0,1,0) 4 (0,1,1) 5 (0,1,2) 6 (1,0,0) 1 (1,0,1) 2 How can i do without using the x.flat() method? Thanks a lot for the answer. regards, Philippe Collet

2 1

Performance of the array protocol
by Francesc Altet 01 Nov '05

01 Nov '05

Hi, I'm trying to start using the array protocol for conversion between Numeric <--> numarray (and newcore in the future), but I'm a bit disappointed because of its performance. For numarray --> Numeric we have: >>> t1=timeit.Timer("num=Numeric.array(na)", "import numarray; import Numeric; na=numarray.arange(10)") >>> t1.repeat(3,10000) [0.59375977516174316, 0.57908082008361816, 0.56574010848999023] >>>t2=timeit.Timer("num=Numeric.fromstring(na._data,typecode=na.typecode())", "import numarray; import Numeric; na=numarray.arange(10)") >>> t2.repeat(3,10000) [0.11653494834899902, 0.1140749454498291, 0.1141819953918457] i.e. the array protocol seems 5x slower than the fromstring() method. Conversely, for Numeric --> numarray: >>> t3=timeit.Timer("na=numarray.array(num)", "import numarray; import Numeric;num=Numeric.arange(10)") >>> t3.repeat(3,10000) [1.3475611209869385, 1.3277668952941895, 1.3417830467224121] >>>t4=timeit.Timer("na=numarray.array(buffer(num),type=num.typecode(),shape=num.shape)", "import numarray; import Numeric; num=Numeric.arange(10)") >>> t4.repeat(3,10000) [0.42027187347412109, 0.41690587997436523, 0.41626906394958496] in this case, the array protocol is 3x slower than using the buffer interface. I'm wondering whether this relatively poor performance in present implementation of the array protocol is surmountable or is an intrinsic limitation of it. Thanks, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

5 15