Mailman 3 August 2007 - NumPy-Discussion

NumPy-1.0.3.x
by Jarrod Millman 10 Aug '07

10 Aug '07

Hello everyone, I made a mumpy-1.0.3.x branch from the 1.0.3 tag and tried to get everything working (see changesets 3957-3961). I added back get_path to numpy/distutils/misc_util.py, which is used by Lib/odr/setup.py in scipy 0.5.2. I also tried to clean up a few issues by doing the same thing that was done to the trunk in: http://projects.scipy.org/scipy/numpy/changeset/3845 http://projects.scipy.org/scipy/numpy/changeset/3848 I am still seeing 2 problems: 1) http://projects.scipy.org/scipy/numpy/ticket/535 2) when I run scipy.test(1,10), I get: check_cosine_weighted_infinite (scipy.integrate.tests.test_quadpack.test_quad)Illegal instruction If anyone has any ideas as to what is wrong, please let me know. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

2 1

I am volunteering to be the release manager for NumPy 1.0.3.1 and SciPy 0.5.2
by Jarrod Millman 10 Aug '07

10 Aug '07

I volunteer to be the release manager for NumPy 1.0.3.1 and SciPy 0.5.3. In order to actually get them both released I will obviously need some help. But given the amount of work required and the number of people who have offered to help, I believe this will be doable. Given the extensive discussion about what is needed for these releases, I am fairly confident that I know what needs to be done. I will try to be very specific about what I will do and what I will need help with. Basically, I am just rewriting the plan described by Robert Kern last month. Please let me know if you have any suggestions/comments/problems with this plan and please let me know if you can commit to helping in any way. [[NOTE: I just (on Monday) hired 2 full-time programmers to work on the neuroimaging in python (NIPY) project, so they will be able to help out with bug fixing as well as testing the pre-releases on different platforms.]] Releasing NumPy 1.0.3.1 =================== On July 24th, Robert suggested making a numpy 1.0.3.1 point release. He was concerned that there were some changes in numpy.distutils that needed to cook a little longer. So I am offering to make a 1.0.3.1 release. If Travis or one of the other core NumPy developers want to make a 1.0.4 release in the next week or so, then there won't be a need for a 1.0.3.1 release. First, I will branch from the 1.0.3 tag: svn cp http://svn.scipy.org/svn/numpy/tags/1.0.3 http://svn.scipy.org/svn/numpy/branches/1.0.3 Second, I will apply all the patches necessary to build scipy from svn, but nothing else. Then I will just follow the NumPy release instructions: http://projects.scipy.org/scipy/numpy/wiki/MakingReleases I will make the tarball and source rpm; but will need help with everything else. Things will go faster if someone else can build the Windows binaries. If not, my new programmers and I will make the binaries. Finally, one of the sourceforge admins will need upload those files once we are done. (I am happy to be made an admin and upload the files myself, if it would be more convenient.) Releasing SciPy 0.5.3 ================= I will make a 0.5.3 scipy branch: svn cp http://svn.scipy.org/svn/scipy/trunk http://svn.scipy.org/svn/scipy/branches/0.5.3 >From then on normal development will continue on the trunk, but only bug fixes will be allowed on the branch. I will ask everyone to test the branch for at least 1 week depending on whether we get any bug reports. Once we are able to get the most serious bugs fixed, I will start working with everyone to build as many binaries as possible. I will rely on David Cournapeau and Andrew Straw to provide RPMs and DEBs. Again, things will go faster if someone else can build the Windows binaries. But if not, my new programmers and I will figure out how to make the binaries for Windows. We can also make the OS X binaries especially if Robert Kern is stilling willing to help. I will also draft a release announcement and give everyone time to comment on it. I will either need to get access to the sourceforge site and the PyPi records or someone will have to update them for me. Timeline ======= If this is agreeable to everyone, I will make the NumPy branch on Friday and apply the relevant patches. Then if I can get someone else to make the Windows executables and upload the files, we should be able to have a new NumPy release before the beginning of the SciPy conference. As for the 0.5.3 SciPy branch, we can discuss this in some detail if everyone is OK with the basic plan. In general, I hope that I will be able to have a 1.0.3.1 NumPy release before August 20th. Perhaps we could even make the 0.5.3 branch by the 20th. Fortunately, as David said earlier the main issue is getting a new release of NumPy out. Resources ======== As I mentioned I just hired 2 full-time programmers to work on NIPY who will be able to help me get the binaries built and tested for the different platforms. All 3 of us will be at the SciPy conference next week. So we will hopefully be able to solve whatever problems we run into very quickly given that it will be so easy to get help. Additionally, David Cournapeau has said that he is willing to help get a new release of SciPy out. He has already been busy at work squashing bugs. Sincerely, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

5 5

Re: [Numpy-discussion] Fail to compile Numpy on Cygwin
by Pearu Peterson 10 Aug '07

10 Aug '07

On Fri, August 10, 2007 11:27 am, Javier Maria Torres (MI/EEM) wrote: > Hi Pearu, > > Using just "python setup.py build" I get the following change in the > same error: > > ... > compile options: '-I/usr/local/include/python2.5 -Inumpy/core/src > -Inumpy/core/include -I/usr/local/include/python2.5 -c' > gcc: _configtest.c > gcc _configtest.o -L/usr/local/lib -L/usr/lib -o _configtest.exe > /usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld: > crt0.o: No such file: No such file or directory > collect2: ld returned 1 exit status > /usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld: > crt0.o: No such file: No such file or directory > collect2: ld returned 1 exit status > failure. Check were is the crt0.o file in your system, if it exists then your cygwin environment is not set up properly, I guess. If not, you might need to install compiler development libraries to cygwin system. Pearu

1 0

Re: [Numpy-discussion] Fail to compile Numpy on Cygwin
by Pearu Peterson 10 Aug '07

10 Aug '07

On Fri, August 10, 2007 11:01 am, Javier Maria Torres (MI/EEM) wrote: > Hi, > > I get the following output when trying to compile the latest Numpy SVN > snapshot on Cygwin (gcc 3.4.4) and Python (Cygwin-installed, 2.5.1; I > also have the Windows version installed, this might cause problems?). I > also include the (meager) site.cfg used. I would appreciate any comment. In your build command python setup.py config --compiler=mingw32 build --compiler=mingw32 install you are not using cygwin gcc compiler but mingw32. I think you cannot do this - don't ask why, some time ago I failed to determine the cause. Anyway, under cygwin just try python setup.py build then it should just pick up cygwin compiler. Or, execute python setup.py config --compiler=mingw32 build --compiler=mingw32 install from Windows cmd line. HTH, Pearu

2 1

Fail to compile Numpy on Cygwin
by Javier Maria Torres (MI/EEM) 10 Aug '07

10 Aug '07

Hi, I get the following output when trying to compile the latest Numpy SVN snapshot on Cygwin (gcc 3.4.4) and Python (Cygwin-installed, 2.5.1; I also have the Windows version installed, this might cause problems?). I also include the (meager) site.cfg used. I would appreciate any comment. Thanks a lot, and greetings, Javier Torres

1 0

rant against from numpy import * / from pylab import *
by Sebastian Haase 09 Aug '07

09 Aug '07

Hi! I use the wxPython PyShell. I like especially the feature that when typing a module and then the dot "." I get a popup list of all available functions (names) inside that module. Secondly, I think it really makes code clearer when one can see where a function comes from. I have a default import numpy as N executed before my shell even starts. In fact I have a bunch of my "standard" modules imported as <some single capital letter>. This - I think - is a good compromise to the commonly used "extra typing" and "unreadable" argument. a = sin(b) * arange(10,50, .1) * cos(d) vs. a = N.sin(b) * N.arange(10,50, .1) * N.cos(d) I would like to hear some comments by others. On a different note: I just started using pylab, so I did added an automatic "from matplotlib import pylab as P" -- but now P contains everything that I already have in N. It makes it really hard to *find* (as in *see* n the popup-list) the pylab-only functions. -- what can I do about this ? Thanks, Sebastian

2 3

Working with lists
by Nils Wagner 09 Aug '07

09 Aug '07

Hi all, I have a list e.g. >>> bounds [(1950.0, 2100.0), (1800.0, 1850.0), (1600.0, 1630.0), (1400.0, 1420.0), (1200.0, 1210.0), (990, 1018.0), (10, 12), (12.0, 14.0), (14.0, 16.0), (16.0, 18.0), (18.0, 20)] How can I extract the first value of each pair given in parenthesis i.e. 1950,1800,1600,1400,... ? Nils

5 5

numpy arrays, data allocation and SIMD alignement
by David Cournapeau 09 Aug '07

09 Aug '07

Hi, Following an ongoing discussion with S. Johnson, one of the developer of fftw3, I would be interested in what people think about adding infrastructure in numpy related to SIMD alignement (that is 16 bytes alignement for SSE/ALTIVEC, I don't know anything about other archs). The problem is that right now, it is difficult to get information for alignement in numpy (by alignement here, I mean something different than what is normally meant in numpy context; whether, in my understanding, NPY_ALIGNED refers to a pointer which is aligned wrt his type, here, I am talking about arbitrary alignement). For example, for fftw3, we need to know whether a given data buffer is 16 bytes aligned to get optimal performances; generally, SSE needs 16 byte alignement for optimal performances, as well as altivec. I think it would be nice to get some infrastructure to help developers to get those kind of information, and maybe to be able to request 16 aligned buffers. Here is what I can think of: - adding an API to know whether a given PyArrayObject has its data buffer 16 bytes aligned, and requesting a 16 bytes aligned PyArrayObject. Something like NPY_ALIGNED, basically. - forcing data allocation to be 16 bytes aligned in numpy (eg define PyDataMem_Mem to a 16 bytes aligned allocator instead of malloc). This would mean that many arrays would be "naturally" 16 bytes aligned without effort. Point 2 is really easy to implement I think: actually, on some platforms (Mac OS X and FreeBSD), malloc returning 16 bytes aligned buffers anyway, so I don't think the wasted space is a real problem. Linux with glibc is 8 bytes aligned, I don't know about windows. Implementing our own 16 bytes aligned memory allocator for cross platform compatibility should be relatively easy. I don't see any drawback, but I guess other people will. Point 1 is more tricky, as this requires much more changes in the code. Do main developers of numpy have an opinion on this ? cheers, David

10 33

Where to put misc C function in numpy ?
by David Cournapeau 09 Aug '07

09 Aug '07

Hi, Following the thread on facilities for SIMD friendly allocations, I have a basic private branch ready for review, but I have one problem: where to put the allocation functions ? The problem is the following: data buffers are allocated/deallocated with functions defined in ndarrayobject,h PyMemData_NEW(ptr) malloc(ptr) ... Which I would replace with PyMemData_NEW(ptr) npy_aligned_alloc(ptr, DEF_ALIGNMENT) Where to define npy_aligned_alloc ? As PyMemData_NEW is used outside numpy.core, the function needs to be available somewhat "publically", but as far as I understand numpy code structure, there is no such facility available (eg a pure C library, totally unaware of python, which would contain some useful tools for numpy), right ? David

1 0

APL2007 Update
by Steven H. Rogers 09 Aug '07

09 Aug '07

Attached is an updated announcement for APL2007: Arrays and Objects. 21-23 October 2007 Montreal, Canada APL = Array Programming Languages

1 0