Mailman 3 September 2004 - NumPy-Discussion

A bug in creating array of long int.
by Shin 20 Sep '04

20 Sep '04

I got the following strange value when converting a long integer into numarray. I spent some time in tracking it down in my program. Is it a bug or an expected one? Thanks. >>> array([True]) array([1], type=Bool) >>> array([1]) array([1]) >>> array([1L]) # How about preserving long int type? array([1]) >>> array([10000000000000000L]) array([1874919424]) # Oops. The value is changed without any notice. -- Daehyok Shin (Peter)

3 3

RE: [Numpy-discussion] Assignment from a list is slow in Numarray
by Nadav Horesh 20 Sep '04

20 Sep '04

Yo may try the aging module TableIO (http://php.iupui.edu/~mmiller3/python/). Just replace "import Numeric" by "import numarray as Numeric". It may give up to two-fold speed improvement. Since you have C skills, you might update the small C source to interact directly with numarray. Nadav -----Original Message----- From: Timo Korvola [mailto:tkorvola@e.math.helsinki.fi] Sent: Sun 19-Sep-04 20:35 To: numpy-discussion(a)lists.sourceforge.net Cc: Subject: [Numpy-discussion] Assignment from a list is slow in Numarray Hello, I am new to the list, sorry if you've been through this before. I am trying to do some FEM computations using Petsc, to which I have written Python bindings using Swig. That involves passing arrays around, which I found delightfully simple with NA_{Input,Output,Io}Array. Numeric seems more difficult for output and bidirectional arrays. My code for reading a triangulation from a file went roughly like this: coord = zeros( (n_vertices, 2), Float) for v in n_vertices: coord[ v, :] = [float( s) for s in file.readline().split()] This was taking quite a bit of time with ~50000 vertices and ~100000 elements, for which three integers per element are read in a similar manner. I found it was faster to loop explicitly: coord = zeros( (n_vertices, 2), Float) for v in n_vertices: for j, c in enumerate( [float( s) for s in file.readline().split()]): coord[ v, j] = c Morally this uglier code with an explicit loop should not be faster but it is with Numarray. With Numeric assignment from a list has reasonable performance. How can it be improved for Numarray? -- Timo Korvola <URL:http://www.iki.fi/tkorvola> ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Numpy-discussion mailing list Numpy-discussion(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

PyArray_FromDimsAndData: Numeric interface to PyCObject data
by Berthold Höllmann 16 Sep '04

16 Sep '04

Hello, For integrating some codes I try to use PyCObjects with Numeric interfaces. I have PyCObjects that contain several arrays. I want to provide these arrays to Python as Numeric arrays using PyArray_FromDimsAndData, the arrays could grow quite large, so I want to handle arround references, not copies. Of course this is dangerous. If I destroy the PyCObject and try to access the Numeric array referencing its data later, who knows what happens. It would be nice to have some callback function slot to allow to increase the refcount of the PyCObject when a Numeric array with a reference is created and to decrease it when the array is deleted. Is there a way to achive this. Further after skimming over the numarray documentation it seems PyArray_FromDimsAndData and similar functions for numarray are copiing the data instead of using a reference. Is this true? If yes, is it planned to provide a method for generating numarray arrays using references only? Kind regards Berthold Höllmann -- Germanischer Lloyd AG CAE Development Vorsetzen 35 20459 Hamburg Phone: +49(0)40 36149-7374 Fax: +49(0)40 36149-7320 e-mail: hoel(a)gl-group.com Internet: http://www.gl-group.com This e-mail contains confidential information for the exclusive attention of the intended addressee. Any access of third parties to this e-mail is unauthorised. Any use of this e-mail by unintended recipients such as copying, distribution, disclosure etc. is prohibited and may be unlawful. When addressed to our clients the content of this e-mail is subject to the General Terms and Conditions of GL's Group of Companies applicable at the date of this e-mail. GL's Group of Companies does not warrant and/or guarantee that this message at the moment of receipt is authentic, correct and its communication free of errors, interruption etc.

1 0

doc bug in numarray 1.1
by Chris Barker 15 Sep '04

15 Sep '04

HI all, I just installed numarray 1.1. All went well with setup.py. Then I decided to try to build it against an atlas lapack. I found, under the Linear Algebra section: """ the setup procedure needs to be modified to force the lapack_lite module to be linked against those rather than the builtin replacement functions. Edit Packages/LinearAlgebra2/setup.py and edit the variables sourcelist, lapack_dirs, and lapack_libs. In sourcelist you should remove all sourcefiles besides .... """ but there is no: Packages/LinearAlgebra2/setup.py However, I did find in: addons.py """ if os.environ.has_key('USE_LAPACK'): BUILTIN_BLAS_LAPACK = 0 else: BUILTIN_BLAS_LAPACK = 1 """ so I tried: export USE_LAPACK=true python setup.py build --gencode Now it appears to be compiling linear_algebra differently, as I now get a linking error, can't find: f90math, fio, or f77math However, if I remove those from lapack_libs in addons.py, I can get it to compile, install, and as far as I can tell, run fine. Why are they in that list? By the way, this is all under Gentoo Linux. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

3 6

character arrays supported by C API?
by Faheem Mitha 14 Sep '04

14 Sep '04

Dear People, Are character arrays supported by the Numarray C API? My impression from the documentation is no, but I would appreciate a confirmation. Thanks. Faheem.

1 0

PEP for overiding and/or/not
by Tim Hochberg 14 Sep '04

14 Sep '04

In case anyone missed it, Greg Ewing posted a PEP that describes a way to allow the overriding of the and/or/not. Since numeric applications are listed as one of the motivating factors, people here might want to look it over and weight in. There's been some discussion both on python-list and python-dev. -tim

1 0

PyArray_FromDimsAndData and data copying
by esatel 13 Sep '04

13 Sep '04

In April of this year (around April 22), there was some talk about altering PyArray_FromDimsAndData work so that it would work without copying data (for compatibility with Numerical). Does anyone know if the change was made? The documentation still suggests the data is copied. Thanks, Eli

1 0

Conference presentations
by Travis E. Oliphant 13 Sep '04

13 Sep '04

Hi all, The SciPy 2004 conference was a great success. I personally enjoyed seeing all attendees and finding out about the activity that has been occurring with Python and Science. As promised, all of the presentations that were submitted to abstracts(a)scipy.org are now available on-line under the conference-schedule page. The link is http://www.scipy.org/wikis/scipy04/ConferenceSchedule If anyone who hasn't submitted their presentation would like to, you still can. As I was only able to attend the first day, I cannot comment on the entire conference. However, what I saw was very encouraging. There continues to be a great amount of work being done in using Python for Scientific Computing and the remaining problems seems to be how to get the word out and increase the user base. Many thanks are due to the presenters and the conference sponsors: *The National Biomedical Computation Resource* <http://nbcr.sdsc.edu> (NBCR, UCSD, San Diego, CA) The mission of the National Biomedical Computation Resource at the University of California San Diego and partners at The Scripps Research Institute and Washington University is to conduct, catalyze, and advance biomedical research by harnessing, developing and deploying forefront computational, information, and grid technologies. NBCR is supported by _National Institutes of Health (NIH) _ through a _National Center for Research Resources_ centers grant (P 41 RR08605). *The Center for Advanced Computing Research* <http://www.cacr.caltech.edu/> (CACR, CalTech <http://www.scipy.org/wikis/scipy04/CalTech>, Pasadena, CA) CACR is dedicated to the pursuit of excellence in the field of high-performance computing, communication, and data engineering. Major activities include carrying out large-scale scientific and engineering applications on parallel supercomputers and coordinating collaborative research projects on high-speed network technologies, distributed computing and database methodologies, and related topics. Our goal is to help further the state of the art in scientific computing. *Enthought, Inc.* <http://www.enthought.com> (Austin, TX) Enthought, Inc. provides business and scientific computing solutions through software development, consulting and training. Best regards to all, -Travis Oliphant Brigham Young University 459 CB Provo, UT 84602 oliphant.travis(a)ieee.org

1 0

Reshaing continued
by Karthikesh Raju 13 Sep '04

13 Sep '04

Hi All, yes, Reshaping can be done row wise using transpose, for example a = transpose(reshape(arange(0,9),(3,3))) but what about higher dimension arrays. example a = reshape(arange(0,18),(2,3,3)) a[0,:,:], a[1,:,:] should be rows wise extracts like a[0,:,:] = 0 3 6 1 4 7 2 5 8 etc Why cant we define the axis along which reshape should work .. warm regards karthik ----------------------------------------------------------------------- Karthikesh Raju, email: karthik(a)james.hut.fi karthikesh.raju(a)gmail.com Researcher, http://www.cis.hut.fi/karthik Helsinki University of Technology, Tel: +358-9-451 5389 Laboratory of Comp. & Info. Sc., Fax: +358-9-451 3277 Department of Computer Sc., P.O Box 5400, FIN 02015 HUT, Espoo, FINLAND -----------------------------------------------------------------------

4 5

ANN: numarray-1.1
by Todd Miller 13 Sep '04

13 Sep '04

Release Notes for numarray-1.1 Numarray is an array processing package designed to efficiently manipulate large multi-dimensional arrays. Numarray is modelled after Numeric and features c-code generated from python template scripts, the capacity to operate directly on arrays in files, and improved type promotions. Although numarray-1.1 is predominantly a bugfix release, if you use numarray, I strongly recommend upgrading. I. ENHANCEMENTS 986194 Add SMP threading Build/install with --smp to enable ufuncs to release the GIL during their compute loops. You have to supply your own threads and partition your array computations among them to realize any SMP benefit. This adds overhead so don't do it unless you have multiple CPUs and know how to manage multiple compute threads. 1016142 CharArray eval() too slow CharArray.fasteval() was modified to use strtod() rather than Python's eval(). This makes it ~70x faster for converting CharArrays to NumArrays. fasteval() no longer works for complex types. eval() still works for everything. 989618 Document memmap.py (memory mapping) 996177 Unsigned int type support limited 1008968 Add kroenecker product II. BUGS FIXED / CLOSED 984286 max.reduce of byteswapped array Sebastian Haase reported that the reduction of large (>100KB) byteswapped arrays did not work correctly. This bug affected reductions and accumulations of byteswapped and misaligned arrays causing them to produce incorrect answers. Thanks Sebastian! 1011456 numeric compatibility byteoffset numarray's Numeric compatibility C-API didn't correctly account for the byte offsets produced by sub-arrays and array slices. This was fixed by re-defining the meaning of the ->data pointer in the PyArrayObject struct to include byteoffset. NA_OFFSETDATA() was likewise redefined to return ->data rather than ->data + ->byteoffset. Correctly written code is still source compatible. Incorrectly written code will generally be transparently fixed. Code which accounted for byteoffset without using NA_OFFSETDATA() will break. This bug affected functions in numarray.numeric as well as add-on packages like numarray.linear_algebra and numarray.fft. 1009462 matrixmultiply (a,b) leaves b transposed Many people reported this side effect. Thanks to all. 919297 Windows build fails VC++ 7.0 964356 random_array.randint exceeds boundaries 985710 buffer not aligned on 8 byte boundary (Windows-98 broken) 990328 Object Array repr for >1000 elements 997898 Invalid sequences errors 1004600 Segfault in array element deletion 1005537 Incorrect handling of overlapping assignments in Numarray 1008375 Weirdness with 'new' method 1008462 searchsorted bug and fix 1009309 randint bug fix patch 1015896 a.is_c_array() mixed int/bool results 1016140 argsort of string arrays See http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse for more details. III. CAUTIONS 1. This release is binary incompatible with numarray-1.0. Writers of C-extensions which directly reference the byteoffset field of the PyArrayObject should be aware that the data pointer is now the sum of byteoffset and the buffer base pointer. All C extensions which use the numarray C-API must be recompiled. This incompatibility was an unfortunate consequence of the fix for "numeric compatibility byteoffset". WHERE ----------- Numarray-1.1 windows executable installers, source code, and manual is here: http://sourceforge.net/project/showfiles.php?group_id=1369 Numarray is hosted by Source Forge in the same project which hosts Numeric: http://sourceforge.net/projects/numpy/ The web page for Numarray information is at: http://stsdas.stsci.edu/numarray/index.html Trackers for Numarray Bugs, Feature Requests, Support, and Patches are at the Source Forge project for NumPy at: http://sourceforge.net/tracker/?group_id=1369 REQUIREMENTS ------------------------------ numarray-1.1 requires Python 2.2.2 or greater. AUTHORS, LICENSE ------------------------------ Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science Institute. We'd like to acknowledge the assitance of Francesc Alted, Paul Dubois, Sebastian Haase, Tim Hochberg, Nadav Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant, Pearu Peterson, Peter Verveer, Colin Williams, and everyone else who has contributed with comments and feedback. Numarray is made available under a BSD-style License. See LICENSE.txt in the source distribution for details. -- Todd Miller jmiller(a)stsci.edu

1 0