Mailman 3 March 2005 - SciPy-Dev

cvxopt
by Alan G Isaac 30 Mar '05

30 Mar '05

Why isn't cvxopt integrating with SciPy? Here is the author's answer: http://www.ee.ucla.edu/~vandenbe/cvxopt/cvxopt-faq fwiw, Alan Isaac

1 0

Re: [SciPy-dev] Re: [Numpy-discussion] Trying out Numeric3
by Pearu Peterson 28 Mar '05

28 Mar '05

On Sun, 27 Mar 2005, Michiel Jan Laurens de Hoon wrote: > Having a separate scipy_distutils that fixes some bugs in Python's distutils > is a design mistake in SciPy that we should not repeat in Numeric3. Not that > I don't think the code in scipy_distutils is not useful -- I think it would > be very useful. But the fact that it is not integrated with the existing > Python distutils makes me wonder if this package really has been thought out > that well. I don't think that part of scipy_distutils design was to fix Python's distutils bugs. As we found a bug, its fix was added to scipy_distutils as well as reported to distutils bug tracker. The main reason for adding bug fixes to scipy_distutils was to continue the work with scipy instead of waiting for the next distutils release (i.e. Python release), nor we could expect that SciPy users would use CVS version of Python's distutils. Also, SciPy was meant to support Python 2.1 and up, so the bug fixes remained relevant even when the bugs were fixed in Python 2.2 or 2.3 distutils. So much of history.. > As far as I can tell, scipy_distutils now fulfills four functions: > 1) Bug fixes for Python's distutils for older Python versions. As Numeric3 > will require Python 2.3 or up, these are no longer relevant. > 2) Bug fixes for current Python's distutils. These should be integrated with > Python's distutils. Writing your own package instead of contributing to > Python gives you bad karma. > 3) Fortran support. Very useful, and I'd like to see them in Python's > distutils. Another option would be to put this in SciPy.fortran or something > similar. But since Python's distutils already has a language= option for C++ > and Objective-C, the cleanest way would be to add this to Python's distutils > and enable language="fortran". > 4) Stuff particular to SciPy, for example finding Atlas/Lapack/Blas > libraries. These we can decide on a case-by-case basis if it's useful for > Numeric3. Plus I would add the scipy_distutils ability to build sources on-fly feature (build_src command). That's a very fundamental feature useful whenever swig or f2py is used, or when building sources from templates or dynamically during a build process. Btw, I have started scipy_core clean up. The plan is to create the following package tree under Numeric3 source tree: scipy.distutils - contains cpuinfo, exec_command, system_info, etc scipy.distutils.fcompiler - contains Fortran compiler support scipy.distutils.command - contains build_src and config_compiler commands plus few enhancements to build_ext, build_clib, etc commands scipy.base - useful modules from scipy_base scipy.testing - enhancements to unittest module, actually current scipy_test contains one useful module (testing.py) that could also go under scipy.base and so getting rid of scipy.testing scipy.weave - scipy.f2py - not sure yet how to incorporate f2py2e or weave sources here. As a first instance people are assumed to download them to Numeric3/scipy/ directory but in future their sources could be added to Numeric3 repository. For Numeric3 f2py and weave are optional. scipy.lib.lapack - wrappers to Atlas/Lapack libraries, by default f2c generated wrappers are used as in current Numeric. For backwards compatibility, there will be Packages/{FFT,MA,RNG,dotblas+packages from numarray}/ and Lib/{LinearAlgebra,..}.py under Numeric3 that will use modules from scipy. Pearu

2 1

Re: [SciPy-dev] Re: [Numpy-discussion] Trying out Numeric3
by Pearu Peterson 25 Mar '05

25 Mar '05

On Fri, 25 Mar 2005, Michiel Jan Laurens de Hoon wrote: > Pearu Peterson wrote: >> I got an impression from earlier threads that scipy_distutils will be >> included to scipy_base. So, I am proposing to use scipy_distutils tools and >> our scipy experience for dealing with this issue, scipy.lib.lapack >> would be a good working prototype here. > > Have you tried integrating scipy_distutils with Python's distutils? My guess > is that Python's distutils can benefit from what is in scipy_distutils, > particularly the parts dealing with C compilers. A clean integration will > also prevent duplicated code, avoids Pearu having to keep scipy_distutils up > to date with Python's distutils, and will enlarge the number of potential > users. Having two distutils packages seems to be too much of a good thing. No, I have not. Though a year or so ago there was a discussion about this in distutils list, mainly for adding Fortran compiler support to distutils. At the time I didn't have resources to push scipy_distutils features to distutils and even less so for now. So, one can think that scipy_distutils is an extension to distutils, though it also includes few bug fixes for older distutils. On the other hand, since Scipy supports Python starting at 2.2 then it cannot relay much on new features added to distutils of later Python versions. Instead, if these features happen to be useful for Scipy then they are backported for Python 2.2 through implementing them in scipy_distutils. "Luckily", there are not much such features as scipy_distutils has evolved with new very useful features much quicker than distutils. But, for Numeric3, scipy.distutils would be a perfect place to clean up scipy_distutils a bit, e.g. removing some obsolete features and assuming that Numeric3 will support Python 2.3 and up. Based on that, integrating scipy_distutils features to standard distutils can be made less pain if someone decides to do that. Pearu

1 0

Re: [Numpy-discussion] Trying out Numeric3
by Travis Oliphant 25 Mar '05

25 Mar '05

Michiel Jan Laurens de Hoon wrote: > Arnd's comment raises the question of how to try out or contribute to > Numeric3 if the code base is changing from day to day. It may be a > good idea to set up some division of labor, so we can contribute to > Numeric3 without getting in each other's way. For example, I'd be > interested in working on setup.py and putting different parts of > Numeric3/scipy_base together. > Michiel, you are free to work on setup.py all you want :-) Putting the parts of scipy_base together is a good idea. Exactly how to structure this is going to require some thought and need to be coordinated with current scipy. I want a package that is as easy to install as current Numeric (so the default will have something like lapack_lite). But, this should not handicap nor ignore a speed-conscious user who wants to install ATLAS or take advantage of vendor-supplied libraries. There should be a way to replace functionality that is clean and does not require editing setup.py files. Anybody with good ideas about how to do this well is welcome to speak up. Perhaps, the easiest thing to do is to keep the basic Numeric structure (with C-based easy-to-install additions) and call it scipylite (with backwards compatibility provided for Numeric, LinearAlgebra, RandomArray, and MLab names). This also installs the namespace scipy which has a little intelligence in it to determine if you have altas and fortran capabilities installed or not. Then, provide a scipyatlas package that can be installed to take advantage of atlas and vendor-supplied lapack/blas. Then, a scipyfortran package that can be installed if you have a fortran compiler which provides the functionality provided by fortran libraries. So, there are three divisions here. Feedback and criticisms encouraged and welcomed..... -Travis

2 3

check_syevr_irange/check_syevr_vrange ?
by Arnd Baecker 24 Mar '05

24 Mar '05

Hi, to figure out a question related to a recent snapshot for windows by Joe Cooper I installed on my debian sarge machine a pretty recent scipy from CVS (see below). For scipy.test(1) there are 4 errors, all related to check_syevr_irange/check_syevr_vrange I am not worried too much about the errors, as the results seem pretty close. Still I thought it would be better to report those here. Best, Arnd Python 2.3.5 (#1, Mar 22 2005, 11:11:34) Type "copyright", "credits" or "license" for more information. IPython 0.6.13_cvs -- An enhanced Interactive Python. ? -> Introduction to IPython's features. %magic -> Information about IPython's 'magic' % functions. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]:import scipy numerix Numeric 23.8 In [2]:scipy.__version__ Out[2]:'0.3.3_303.4584' In [3]:scipy.test(1) ====================================================================== FAIL: check_heevr_irange_high (scipy.lib.lapack.test_lapack.test_flapack_complex) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 80, in check_heevr_irange_high def check_heevr_irange_high(self): self.check_syevr_irange(sym='he',irange=[1,2]) File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 68, in check_syevr_irange assert_array_almost_equal(dot(a,v[:,i]),w[i]*v[:,i]) File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy_test/testing.py", line 740, in assert_array_almost_equal assert cond,\ AssertionError: Arrays are not almost equal (mismatch 33.3333333333%): Array 1: [ 3.6929266+0.j 4.0951085+0.j 7.3420534+0.j] Array 2: [ 3.6929276+0.j 4.0951101+0.j 7.3420528+0.j] ====================================================================== FAIL: check_heevr_vrange (scipy.lib.lapack.test_lapack.test_flapack_complex) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 102, in check_heevr_vrange def check_heevr_vrange(self): self.check_syevr_vrange(sym='he') File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 94, in check_syevr_vrange assert_array_almost_equal(dot(a,v[:,i]),w[i]*v[:,i]) File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy_test/testing.py", line 740, in assert_array_almost_equal assert cond,\ AssertionError: Arrays are not almost equal (mismatch 33.3333333333%): Array 1: [ 3.6929266+0.j 4.0951085+0.j 7.3420534+0.j] Array 2: [ 3.6929276+0.j 4.0951101+0.j 7.3420528+0.j] ====================================================================== FAIL: check_syevr_irange_high (scipy.lib.lapack.test_lapack.test_flapack_float) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 74, in check_syevr_irange_high def check_syevr_irange_high(self): self.check_syevr_irange(irange=[1,2]) File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 68, in check_syevr_irange assert_array_almost_equal(dot(a,v[:,i]),w[i]*v[:,i]) File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy_test/testing.py", line 740, in assert_array_almost_equal assert cond,\ AssertionError: Arrays are not almost equal (mismatch 33.3333333333%): Array 1: [ 3.6929266 4.0951085 7.3420534] Array 2: [ 3.6929276 4.0951101 7.3420528] ====================================================================== FAIL: check_syevr_vrange (scipy.lib.lapack.test_lapack.test_flapack_float) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy/lib/lapack/tests/esv_tests.py", line 94, in check_syevr_vrange assert_array_almost_equal(dot(a,v[:,i]),w[i]*v[:,i]) File "/home/abaecker/NBB/INSTALL_SOFT/TEST_NUMERIC3/InstDir/lib/python2.3/site-packages/scipy_test/testing.py", line 740, in assert_array_almost_equal assert cond,\ AssertionError: Arrays are not almost equal (mismatch 33.3333333333%): Array 1: [ 3.6929266 4.0951085 7.3420534] Array 2: [ 3.6929276 4.0951101 7.3420528] If I should send the full output of python scipy_core/scipy_distutils/system_info.py or any further info, just let me know...

1 0

Possible bug
by Gary Ruben 16 Mar '05

16 Mar '05

I recently tried installing Fernando Perez's iPython under win98 and experienced a small problem, which he suggests might be a scipy problem. Typing import scipy help(scipy) or help scipy in a Win98 DOS window running iPython starts displaying the help but freezes the DOS window once a screenful of help is displayed. If I run iPython from the IPython_shell.py shortcut and look at the running processes, I note two: Winoldap python <- DOS shell If I then type help scipy I notice that the DOS window title changes from Python to GNUPLO~1 and some gnuplot processes start up: Winoldap GNUPLO~1 <- DOS shell Gnuplot_helper Win32popenwin9x Wgnuplot The DOS window freezes up. I don't know why gnuplot is starting. The help> prompt system in iPython seems to work for other cases, so it seems scipy specific. It's not a showstopper for me by any means. Gary R.

1 0

Re: [Numpy-discussion] Re: [SciPy-user] Current thoughts on future directions
by Fernando Perez 15 Mar '05

15 Mar '05

Michiel Jan Laurens de Hoon wrote: > Perry Greenfield wrote: [weave & f2py in the core] >>That they are perfectly fine to go into the core. In fact, if they are >>used by any of the extra packages, they should be in the core to >>eliminate the extra step in the installation of those packages. >> > > -0. > 1) In der Beschraenkung zeigt sich der Meister. In other words, avoid > software bloat. > 2) f2py is a Fortran-Python interface generator, once the interface is > created there is no need for the generator. > 3) I'm sure f2py is useful, but I doubt that it has very general > usefulness. There are lots of other useful Python packages, but we're > not including them in scipy-core either. > 4) f2py and weave don't fit in well with the rest of scipy-core, which > is mainly standard numerical algorithms. I'd like to argue that these two tools are actually critically important in the core of a python for scientific computing toolkit, at its most basic layer. The reason is that python's dynamic runtime type checking makes it impossible to write efficient loop-based code, as we all know. And it is not always feasible to write all algorithms in terms of Numeric vector operations: sometimes you just need to write an indexed loop. At this point, the standard python answer is 'go write an extension module'. While writing extension modules by hand, from scratch, is not all that hard, it certainly presents a significant barrier for less experienced programmers. And yet both weave and f2py make it incredibly easy to get working compiled array code in no time at all. I say this from direct experience, having pointed colleagues to weave and f2py for this very problem. After handing them some notes I have to get started, they've come back saying "I can't believe it was that easy: in a few minutes I had sped up the loop I needed with a bit of C, and now I can continue working on the problem I'm interested in". I know for a fact that if I'd told them to write a full extension module by hand, the result would have been quite different. The reality is that, in scientific work, you are likely to run into this problem at a very early stage, much more so than for other kinds of python usage. For this reason, it is important that the basic toolset provides a clean solution from the start. At least that's been my experience. Regards, f

1 0

Re: [Numpy-discussion] Current thoughts on future directions
by Stephen Walton 14 Mar '05

14 Mar '05

konrad.hinsen(a)laposte.net wrote: > On Mar 10, 2005, at 18:33, Stephen Walton wrote: > >> >> Is it really that hard to install g77 on non-Linux platforms? > > > It takes some careful reading of the instructions... I have to admit that I hadn't remembered the pain I went through building gcc/g77 on HP-UX until Konrad reminded me. He's right, it's _really_ hard. That said, when I was on HP-UX I never used g77, I used HP's Fortran and ANSI C compilers. So, when we talk about Fortran in Scipy (and I've redirected this message to scipy-dev), for non-Linux Unix platforms I think we mean supporting the vendor-provided compilers. And, since Enthought can't keep an in-house copy of every variant, that means we'll be dependent on users of that platform to report what doesn't work, and maybe even try to fix it. The scripts for supporting various compilers in Scipy are simple and easy to understand. I've messed with the Absoft one myself.

1 0

Re: [Numpy-discussion] Current thoughts on future directions
by Stephen Walton 14 Mar '05

14 Mar '05

Michiel Jan Laurens de Hoon wrote: > I agree that Netlib should be in SciPy. But why should Netlib be in > scipy_base? It should not, and I'm sorry if my original message made it sound like I was advocating for that. I was mainly advocating for f2py to be in scipy_base.

1 0

Re: [Numpy-discussion] Re: [SciPy-user] Current thoughts on future directions
by Travis Oliphant 11 Mar '05

11 Mar '05

Michiel Jan Laurens de Hoon wrote: > Perry Greenfield wrote: > >> On Mar 9, 2005, at 11:41 PM, eric jones wrote: >> >>> 2. I do question whether weave really be in this core? I think it >>> was in scipy_core before because it was needed to build some of scipy. >>> 3. Now that I think about it, I also wonder if f2py should really be >>> there -- especially since we are explicitly removing any fortran >>> dependencies from the core. >> >> >> >> It would seem to me that so long as: >> >> 1) both these tools have very general usefulness (and I think they >> do), and >> 2) are not installation problems (I don't believe they are since they >> themselves don't require any compilation of Fortran, C++ or >> whatever--am I wrong on that?) >> >> That they are perfectly fine to go into the core. In fact, if they >> are used by any of the extra packages, they should be in the core to >> eliminate the extra step in the installation of those packages. >> > -0. > 1) In der Beschraenkung zeigt sich der Meister. In other words, avoid > software bloat. > 2) f2py is a Fortran-Python interface generator, once the interface is > created there is no need for the generator. > 3) I'm sure f2py is useful, but I doubt that it has very general > usefulness. There are lots of other useful Python packages, but we're > not including them in scipy-core either. > 4) f2py and weave don't fit in well with the rest of scipy-core, which > is mainly standard numerical algorithms. I'm of the opinion that f2py and weave should go into the core. 1) Neither one requires Fortran and both install very, very easily. 2) These packages are fairly small but provide huge utility --- inlining fortran or C code is an easy way to speed up Python. People who don't "need it" will never realize it's there 3) Building the rest of scipy will need at least f2py already installed and it would simplify the process. 4) Enthought packages (to be released in the future and of interest to scientists) rely on weave. Why not make that process easier with a single initial install. 5) It would encourage improvements of weave and f2py from the entire community. 6) The developers of f2py and weave are both scipy developers and so it would make sense for their code that forms a foundation for other work to go into scipy_core. -Travis

2 1