Mailman 3 March 2005 - NumPy-Discussion

Please chime in on proposed methods for arrays
by Travis Oliphant 17 Mar '05

17 Mar '05

One item I have not received a lot of feedback on is the new proposal for a greatly increased number of methods on the ndarray. The current PEP has a listing of all the proposed methods and attributes (some more were added after consulting current numarray in more detail and looking at all the functions in current Numeric.py) If a function call essentially involved an arrayobject with some other parameters then it was turned into a method. If it involved two "equal" arrays then it was left as a function. This is a somewhat arbitrary convention, and so I am asking for suggestions as to what should be methods. Should all the ufuncs be methods as well? I think Konrad suggested this. What is the opinion of others? The move from functions to methods will mean that some of the function calls currently in Numeric.py will be redundant, but I think they should stay there for backwards compatibility, (perhaps with a deprecation warning...) A final question: I think we need to think carefully about multidimensional indexing so that it replaces current usage of take, put, putmask. For example, how, in numarray would you replace take(a,[1,5,10],axis=-2) if a is a 10x20x30 array? Note that in this case take returns a 10x3x30 array (call it g) with g[:,0,:] = a[:,1,:] g[:,1,:] = a[:,5,:] g[:,2,:] = a[:,10,:] I submit that a[...,[1,5,10],:] would be an appropriate syntax. This would mean changing the current PEP a bit. -Travis

8 8

Re: [Numpy-discussion] Half baked C API?
by Ralf Juengling 17 Mar '05

17 Mar '05

konrad.hinsen(a)laposte.net wrote: > > How much do you expect to gain compared to a Python loop in such > a case? I'd expect a factor 5 to 10. > > Did you consider Pyrex? It lets you move from pure Python to pure C > with Python syntax, mixing both within a single function. I looked at it, but haven't tried it out yet. As far as I understand it, if I'd give Pyrex the example code in my previous posting to translate it to C, the result would contain calls to the Python interpreter to have it eveluate unknown functions like 'dot', 'sum' etc. That would be quite slow. So besides having counterparts in the C API, the tool that does the translation also needs to know about those. Ralf

5 7

Re: [Numpy-discussion] Current thoughts on future directions
by konrad.hinsen＠laposte.net 16 Mar '05

16 Mar '05

On 16.03.2005, at 20:57, Stephen Walton wrote: > Well, how much is their time worth is then the question? If they can > afford Not exactly. The first question they ask is how much is the code worth that requires them to buy a Fortran license or to install g77. They might then well choose my competitor's code that doesn't have such requirements. It doesn't help if I tell them that my code doesn't require Fortran at all, but that it relies on a library that can't be installed without a Fortran compiler. > an SGI workstation, I'd think they could afford a copy of SGI's > compiler, From memory, a compiler license is about half the price of a workstation. Depending on particular circumstances (special offers, campus licenses, etc.) the prices can be lower. > I might also point out, not to be argumentative, that the real > difficulty is installing gcc. The extra effort to install g77 once > gcc is working is very small. Are the users you mention doing without > C as well? I guess some do. On all the workstations I have ever used, there was a minimal C compiler for recompiling the kernel, which was also good enough for installing software, though not necessarily a pleasure for development. > Back when I was administering HP-UX, I found a community supported > archive with contributed pre-compiled software in HP's package format. > Is there a similar thing in the SGI community? I don't know. I have used SGI machines for only two years (ten years ago), and in an environment where compilers and development tools were considered necessary. Konrad. -- ------------------------------------------------------------------------ ------- Konrad Hinsen Laboratoire Leon Brillouin, CEA Saclay, 91191 Gif-sur-Yvette Cedex, France Tel.: +33-1 69 08 79 25 Fax: +33-1 69 08 82 61 E-Mail: khinsen(a)cea.fr ------------------------------------------------------------------------ -------

1 0

Re: Record array dimension
by Jin-chung Hsu 16 Mar '05

16 Mar '05

> I was surprised to find out that when I read a field from file into a record > array with format string "8f4" the corresponding field has dimension (1,8) > and not (8,) that I would intuitively expect. Am I possibly spesifying the > format incorrectly? If not, why is there an extra dimension? The first dimension(s) in a record array is always referring to the number of "rows". So, if you have: >>> import numarray.records as rec >>> r=rec.array(formats='f4', shape=8) >>> r.field(0).shape (8,) which will be what you might have expected. But if >>> r.rec.array(formats='8f4', shape=10) >>> r.field(0).shape (10, 8) I assume you have the shape=1, and that's why you get (1, 8) for the field shape. You can have complicated tables like: --> r=rec.array(formats='(4,5)f4', shape=(2,3)) --> r.field(0).shape (2, 3, 4, 5) So, the field shape always has the record array's shape as first dimension(s). JC Hsu

1 0

Record array field dimensions
by Antti Korvenoja 16 Mar '05

16 Mar '05

Hi! I was surprised to find out that when I read a field from file into a record array with format string '8f4' the corresponding field has dimension (1,8) and not (8,) that I would intuitively expect. Am I possibly spesifying the format incorrectly? If not, why is there an extra dimension? Antti Korvenoja

1 0

Future directions for SciPy in light of meeting at Berkeley
by Travis Oliphant 15 Mar '05

15 Mar '05

I wanted to send an update to this list regarding the meeting at Berkeley that I attended. A lot of good disscussions took place at the meeting that should stimulate larger feedback. Personally, I had far more to discuss before I had to leave, and so I hope that the discussions can continue. I was looking to try and understand why with an increasing number of Scientific users of Python, relatively few people actually seem to want to contribute to scipy, regularly, even becoming active developers. There are lots of people who seem to identify problems (though very often vague ones), but not many who seem able (either through time or interest contraints) to actually contribute to code, documentation, or infrastructure. Scipy is an Open source project and relies on the self-selection process of open source contributors. It would seem that while the scipy conference demonstrates a continuing and even increasing use of Python for scientific computing, not as many of these users are scipy devotees. Why? I think the answers come down to a few issues which I will attempt to answer with proposals. 1) Plotting -- scipy's plotting wasn't good enough (we knew that) and the promised solution (chaco) took too long to emerge as a simple replacement. While the elements were all there for chaco to work, very few people knew that and nobody stepped up to take chaco to the level that matplotlib, for example, has reached in terms of cross-gui applicability and user-interface usability. Proposal: Incorporate matplotlib as part of the scipy framework (replacing plt). Chaco is not there anymore and the other two plotting solutions could stay as backward compatible but not progressing solutions. I have not talked to John about this, though I would like to. I think if some other packaging issues are addressed we might be able to get John to agree. 2) Installation problems -- I'm not completely clear on what the "installation problems" really are. I hear people talk about them, but Pearu has made significant strides to improve installation, so I'm not sure what precise issues remain. Yes, installing ATLAS can be a pain, but scipy doesn't require it. Yes, fortran support can be a pain, but if you use g77 then it isn't a big deal. The reality, though, is that there is this perception of installation trouble and it must be based on something. Let's find out what it is. Please speak up users of the world!!!! Proposal (just an idea to start discussion): Subdivide scipy into several super packages that install cleanly but can also be installed separately. Implement a CPAN-or-yum-like repository and query system for installing scientific packages. Base package: scipy_core -- this super package should be easy to install (no Fortran) and should essentially be old Numeric. It was discussed at Berkeley, that very likely Numeric3 should just be included here. I think this package should also include plotting, weave, scipy_distutils, and even f2py. Some of these could live in dual namespaces (i.e. both weave and scipy.weave are available on install). scipy.traits scipy.weave (weave) scipy.plt (matplotlib) scipy.numeric (Numeric3 -- uses atlas when installed later) scipy.f2py scipy.distutils scipy.fft scipy.linalg? (include something like lapack-lite for basic but slow functionality, installation of improved package replaces this with atlas usage) scipy.stats scipy.util (everything else currently in scipy_core) scipy.testing (testing facilities) Each of these should be a separate package installable and distributable separately (though there may be co-dependencies so that scipy.plt would have to be distributed with scipy. Libraries (each separately installable). scipy.lib -- there should be several sub-packages that could live under hear. This is simply raw code with basic wrappers (kind of like a /usr/lib) scipy.lib.lapack -- installation also updates narray and linalg (hooks to do that) scipy.lib.blas -- installation updates narray and linalg scipy.lib.quadpack etc... Extra sub-packages: named in a hierarchy to be determined and probably each dependent on a variety of scipy-sub-packages. I haven't fleshed this thing out yet as you can tell. I'm mainly talking publicly to spur discussion. The basic idea is that we should force ourselves to distribute scipy in separate packages. This would force us to implement a yum-or-CPAN-like package repository, so that we define the interface as to how an additional module could be developed by someone, even maintained separately (with a different license), and simply inserted into an intelligent point under the scipy infrastructure. It also allow installation/compilation issues to be handled on a more per-module basis so that difficult ones could be noted. I think this would also help interested people get some of the enthought stuff put in to the scipy hierarchy as well. Thoughts and comments (and even half-working code) welcomed and encouraged... -Travis O.

22 44

Half baked C API?
by Ralf Juengling 15 Mar '05

15 Mar '05

I recently took a closer at Numeric's and numarray's C APIs for the first time and was surprised not to find the counterparts for all the array functions that are available in the Python API. Did I overlook anything, or do I really have to re-implement things like 'sum', 'argmax', 'convolve', 'cos' in C? Ralf

3 5

SourceForge.net: A04. Site Status (en)
by Colin J. Williams 15 Mar '05

15 Mar '05

Travis, In view of the difficulty at Sourceforge (see below), would it make sense to make the draft PEP available on the Python site? I haven't been able to read the most recent update. Colin W. http://sourceforge.net/docman/display_doc.php?group_id=1&docid=2352#1107968…

2 1

cvs access is broken
by Nils Wagner 14 Mar '05

14 Mar '05

cvs access is broken cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/numpy co -P Numerical cvs [checkout aborted]: unrecognized auth response from cvs.sourceforge.net: M PserverBackend::PserverBackend() Connect (Connection refused)

1 0

Slightly altered multidimensional slicing behavior
by Travis Oliphant 11 Mar '05

11 Mar '05

Hi all, I've updated the PEP on the numeric web page to reflect an improved (I think) usage of Ellipsis and slice objects when mixed with integer indexing arrays. Basically, since partial indexing already assumed an ending ellipsis. The presence of ellipsis or slice objects in the tuple, allow the user to move the position of the partial indexing. It does get a little mind blowing, but is actually not "too" bad using the mapiter object to implement. -Travis

1 0