Mailman 3 February 2005 - SciPy-Dev

Segmentation fault in optimize
by Manuel Metz 11 Feb '05

11 Feb '05

Hi, I now know that I invoked the scipy.optimize.fmin_bfgs not correctly, but I think a 'segmentation fault' even in this case is not a desired result. :-( #----fminerror.py---- import scipy from Numeric import * def f1(x): return (x-5.3)**4 def fd1(x): return 4*(x-5.3)**3 # This works fine r1 = scipy.optimize.fmin_bfgs(f1,1) print r1 # Here I get a segmentation fault r2 = scipy.optimize.fmin_bfgs(f1,1,fd1) print r2 #----fminerror.py---- ----------------------------------------------------- Output: > python fminerror.py Optimization terminated successfully. Current function value: 0.000000 Iterations: 19 Function evaluations: 63 Gradient evaluations: 0 [ 5.31255046] Segmentation fault ----------------------------------------------------- I have a debian linux (sid), python2.3.5 and scipy 0.3.2 installed. Manuel -- ------------------------------------- Manuel Metz Sternwarte der Universitaet Bonn Auf dem Huegel 71 (room 3.06) D - 53121 Bonn E-Mail: mmetz(a)astro.uni-bonn.de Phone: (+49) 228 / 73-3660 Fax: (+49) 228 / 73-3672 -------------------------------------

1 0

Re: Scipy-dev Digest, Vol 16, Issue 4
by Brendan Simons 10 Feb '05

10 Feb '05

OK, I'm asking. The OS-X build requirements are way too high a barrier for me, which means no binary - no scipy on my mac (which is my primary development platform). I recognize that I'm not contributing anything towards that end, but you asked. -Brendan On 4-Feb-05, at 8:36 PM, scipy-dev-request(a)scipy.net wrote: > A binary for OS X is a desireable thing. I think more of a stink > should > be made, so that one gets provided by default. I know that Robert Kern > has worked on such a thing and it is not out of the question, but is on > the horizon. More users asking for it would help.

2 1

Re: [Numpy-discussion] Numeric3
by Travis Oliphant 08 Feb '05

08 Feb '05

Thanks for your comments. Nadav Horesh wrote: > Most of my work is carried using the numarray/Numeric package, and use > scipy not so often for three reasons: > > 1. nummaray is well equip. > It is certainly missing many things. > 1. scipy is mostly undocumented. > SciPy is decently self documented. And, with http://www.scipy.org/livedocs/ self-documentation leads to rapid online documentation. > 1. scipy had bugs in many of it high-end (=rarely used) routines. > Interesting claim. Yes bugs crop up, but they are typically fixed quickly when found. Basing much of the code on standard compiled routines helps to alleviate this problem. > This is a high barrier for a causal scipy users like me, and thus it > hard for me to contribute back. For instance, I wrote a bicoherence > routine some time ago, but found no proper place to contribute it. > Scipy would be a natural place if it would be more accessible. > One of the good reasons to provide bare Numeric3 and to put fft, > linear algebra, etc. in scipy, is that it would force scipy into the > painful process of being user friendly (=accessible) One very important reason I would like to do it. It is hard to know if scipy is accessible or not if people who have problems don't complain but instead walk away or just start their own package. -Travis

1 0

Re: [Numpy-discussion] Numeric3
by Travis Oliphant 08 Feb '05

08 Feb '05

> And NOT a matlab-like environment? Or is it trying to be both? Very loosely a matlab or IDL-like environment. Nobody has really defined what it means to be a MATLAB-like environment. The idea is that someone could do what one can do in MATLAB or IDL, not write code in exactly the same way. A MATLAB translator is an entirely different project. So, I'm not sure. There is no requirement that contributions "not be classed based", or conform to some command in MATLAB. Perhaps what you are referring to is the fact that from scipy import * makes a lot of command-like interaction possible. Is this seen as MATLAB-like? I guess right now, scipy is trying to bring lots of modules under a common naming convention. I think the goals of scipy are very much what people are talking about here. SciPy ultimately becomes what its users and developers want it to become. That is why I'm so unclear as to why we have not received more support in its development --- even in the way of ideas as to how to make it better --- until the great ideas that have been suggested on this list recently. -Travis

1 0

Re: [Numpy-discussion] Numeric3
by Travis Oliphant 07 Feb '05

07 Feb '05

Gary Strangman wrote: > To be in keeping with other python objects, it seems this object would > need to be tight, fast and easily extensible. I *think* this is > exactly what Numeric3 is intended to do. Getting this right is tricky, > but it seems like current solutions are EXTREMELY close. I think it's close too, that's why I'm willing to do this, right now. > already exists. Granted, the notion of what's "basic" vs. advanced is > relative (e.g., where do you put fft, or linear_algebra?). But if made > modular and encapsulated (e.g., an fft.py, linear_algebra.py, > integration.py, morphology.py) and made available both individually > and as part of one or more suites--see #4 below, it's easier to build > on existing code rather than reinvent. There seems to be several users on this list that agree. I would agree too, but would also state that Python also has the idea of "batteries included" meaning that the html, cgi, and xml libraries are all installed using a common installer. SciPy has tried to be modular, too but have a common installer. SciPy is an umbrella package with several sub-packages. Sometimes it is hard to be modular when a sub-package depends on linear algebra routines for example. I think that a basic install should always include linear algebra, random numbers, and fft code. I don't think this should live under a Numeric package because if we are to get arrayobjects into the core, then we need to develop some other place for these things to live. That is essentially what scipy is trying to be -- a place for these extras and additional numeric routines to live. Perhaps we underestimated the problems with using FORTRAN (but I still don't think so given that g77 is available wherever gcc is). With f2py, FORTRAN is incredibly quick to develop Python modules with. So, I would have a hard time saying that we should not use FORTRAN. I can conceed that we should separate modules that need FORTRAN from those that need only C(++). > problem (thanks!!). John indicates that the ultimate goal with > matplotlib is to provide plotting, not just scientific plotting, which > is even better! In that case, though, and in keeping with my previous > comment, perhaps the name matplotlib is a little misleading > (suggesting scientific plotting only). Again, if I were familiar with > python but just starting timeseries analysis, I would expect to load > my data into a (multiarray) python object, import timeseries.py, > import plotlib.py (i.e., matplotlib) and go to work doing timeseries > analysis ... be that at LLNL, Wall St, or in my neuro lab. I think matplotlib is great. SciPy's plotting methods were always considered stop-gap (functional but not much else). I would have much rather seen matplotlib integrate in with the scipy effort, though. Perhaps these modularity issues would be solved already, had this been attempted from the start. -Travis

1 0

Re: [Numpy-discussion] Numeric3
by Travis Oliphant 07 Feb '05

07 Feb '05

konrad.hinsen(a)laposte.net wrote: > On 06.02.2005, at 01:06, Travis Oliphant wrote: > >> But MATLAB is a huge package. SciPy has been trying to do the same >> thing. Plotting was needed and matplotlib did a great thing in >> providing excellent plotting. I've never understood the reason for >> matplotlib to remain separate except as evidence of the great schism >> that was occuring in the community. >> >> In fact, when Eric and I began scipy, I wanted to call it pylab >> (you'll notice I have the pylab.sourceforge.net site). I have fond >> ties to MATLAB myself. > > > That's actually one of the non-technical issues I have with SciPy. > SciPy "smells" like Matlab. Which is fine for those who know and like > Matlab, but not for those (like me) who find Matlab rather > incompatible with their brains. SciPy, like Matlab, is based on the > principle that all operations should be expressed in terms of arrays. > My personal approach is that operations should be expressed in terms > of problem-specific classes in which the internal use of arrays is > an implementation detail. I think I understand better what Konrad is saying than I did several years ago. SciPy may have started with the idea of being something like MATLAB, but in fact, scipy has lot's of class-based approaches: polynomials, random variables, gridding, ode solvers, just to name a few. Basically, scipy just wants code that works and can be useful to others. I don't think you'll find that scipy is opposed to class-based solutions at all. We do tend to like names that are shorter (or at least aliases that are shorter, so that command-line interactive use is easier). Now, I think of scipy really just as a gathering place for Numerical algorithms. I think it is clear that scipy needs to work on it's modularity, taking into account some of the great comments that have been made on this list if it is to be what it is intended to be. -Travis > > There are arguments for and against both approaches, and I think > there is space (meme space) for both. I just mention this to point > out that I don't expect SciPy to become the one and only scientific > Python library that includes everything. I don't expect to contribute > code to SciPy, for example. I wouldn't mind using it, of course, in > the implementation of my classes, if and when the installation issues > become less of a problem. > > Konrad. > -- > ------------------------------------------------------------------------ > ------- > Konrad Hinsen > Laboratoire Leon Brillouin, CEA Saclay, > 91191 Gif-sur-Yvette Cedex, France > Tel.: +33-1 69 08 79 25 > Fax: +33-1 69 08 82 61 > E-Mail: khinsen(a)cea.fr > ------------------------------------------------------------------------ > ------- > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting > Tool for open source databases. Create drag-&-drop reports. Save time > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. > Download a FREE copy at http://www.intelliview.com/go/osdn_nl > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

Preparing Patch Release of LAPACK and ScaLAPACK
by Nils Wagner 07 Feb '05

07 Feb '05

This message from the latest NA Digest, V. 05, # 06 might be of interest ... Nils From: Julien Langou <langou(a)cs.utk.edu> Date: Fri, 4 Feb 2005 13:54:41 -0500 (EST) Subject: Preparing Patch Release of LAPACK and ScaLAPACK We are in the process of preparing a patch for the current LAPACK and ScaLAPACK software. We would like to hear about any problems you may have encountered with the software. Please send us bugs in the software and documentation that you are aware of. You can submit a bug report by going to http://icl.cs.utk.edu/lapack-forum/lapack_bugs.html . Please be as specific as possible. The best way would be by providing us a way to reproduce the problem on our computers. We would like to receive all the information within two weeks, by February 28th, 2005. Thanks, The LAPACK/ScaLAPACK Team.

1 0

SciPy
by Travis Oliphant 06 Feb '05

06 Feb '05

Konrad, Since I've always respected your opinion, I really would like to know what it would take for you to 1) recommend and 2) require your users to install something like (or a reduced subset of something like) scipy. What threshold of "easy to install" do you have -- if you can define it. Also, What platforms do your users use? -Travis Oliphant

1 0

Re: [Numpy-discussion] Numeric3
by John Hunter 05 Feb '05

05 Feb '05

>>>>> "Peter" == Peter Verveer <verveer(a)embl.de> writes: Peter> BTW it was mentioned before that it would be a problem to Peter> remove packages like LinearAlgebra and FFT from the core Peter> Numeric. matplotlib was mentioned as an example of a Peter> package that depends on them. I think that points however Peter> to a fundamental problem with matplotlib: why does a Peter> plotting library need FFTs and linear algebra? So I don't Peter> think anybody can really argue that things like an FFT Peter> should be in a core array package. The dependence on fft arises from the original mission of matplotlib, which was to provide a matlab compatible python plotting. matlab has functions like psd, csd and cohere, which in addition to computing the spectral density and coherence functions, generate the plots. So I implemented Welch's Average Periodogram on top of fft to compute spectral densities for these plotting commands. I use Matrix a lot for matrix multiplications to do text rotations and layout. If Matrix disappeared, it can be easily replace by wrapping dot -- Todd did that already for the numarray equivalent in the numerix module. The matplotlib pylab module aspires to provide a matlab like environment -- the focus is on plotting of course -- but as I've needed things for my own work I've added commands to the matplotlib.mlab module to provide matlab commands in python that do require linear algebra, eg polyfit and polyval. Basically matplotlib.mlab and matplotlib.pylab extend Numeric's MLab. This section of the code could of course be folded into scipy.mlab, Numeric.mlab, or wherever appropriate, but in my view it is already in a pretty sensible place, in a module that attempts to provide a matlab environment for python users. If linear algebra, fft and random numbers disappeared from Numeric, matplotlib would survive. As you say, they don't go to the core of plotting. I could raise an exception if the user called psd and didn't have scipy installed. And everyone else who has written code based on the functionality of Numeric's extra modules could rewrite their code as well. It would mean more work for me and less functionality for some matplotlib users, but would be manageable. We've put a fair amount of work in making Numeric and numarray work transparently in matplotlib at the python and extension code level. Having a third array package in the wild which is incompatible with either is an unattractive option for me, but one which I can deal with if need be. And if the goal is to get an array package into the standard library that has the core functionality of Numeric and numarray, and Numeric3 is the most sensible means to that end, then I am all for it. JDH

1 0

Re: [Numpy-discussion] Numeric3
by Travis Oliphant 05 Feb '05

05 Feb '05

> > I have in the past tried to install SciPy on OSX and I had big > problems with it. Motivated by the recent discussions on the list I > tried again today, and found it much improved, but still quite > involved. It would still be a pain to distribute software that has a > dependency on SciPy to 'average users' that would have trouble > installing SciPy. There does not seem to be a binary distribution on > scipy.org for OS X, which would solve that problem. This is not > unimportant, since I guess OS X is gaining popularity in scientific > computing. A binary for OS X is a desireable thing. I think more of a stink should be made, so that one gets provided by default. I know that Robert Kern has worked on such a thing and it is not out of the question, but is on the horizon. More users asking for it would help. >> I'm very glad to hear these comments, as I constantly wonder why >> people seem to be duplicating SciPy's efforts instead of helping SciPy. > > > I agree with that, but fact is that the split between Numarray and > Numeric prevented me from contributing to SciPy. I have developed for > numarray (the image processing package nd_image, part of numarray, is > done by me) because I felt numarray was much better then Numeric for > that application, and that also ruled out SciPy. There seems now to be > an effort going to either try to merge the two or to bring Numeric up > to numarray standard. Thats great, and if that works out I will try to > rewrite my package so it works with both. Would be happy to > contribute it to SciPy. But I would make sure it remains a package > that can be installed separately from SciPy if required. > "What scipy is" is quite open to interpretation. The idea of scipy is just to have a super-package where different packages can live and have some kind of inter-operability. It is made so that sub-packages can be easily added and built upon request. We try hard to make sure that the only real dependency is scipy_base which should not be hard to install anywhere (does not require ATLAS or FORTRAN). It would be great to add nd_image as a sub-package to SciPy. In fact, nd_image was one of the things that most alarmed me about the emerging split in the community, as it reproduces some of what is already in SciPy (N-D convolution) while adding some great things that I've always wanted in SciPy (morphology). I believe scipy can be easily installed as a collection of packages if that is desired. Pearu has done an incredible job with scipy_disutils to make that process simple. Making a single super package helps users see that they have only a part of what's available (but maybe all they need), and you get the added benefit of standard documentation efforts. Nobody has taken up the ball to actually try and distribute individual packages of scipy, but it is a welcome effort. www.scipy.org has places where such packages could live. Enthought has mainly been interested in distributing "batteries-included" packages for windows because it is what their customers need. Other are welcome to pick up the slack. Perhaps it would be good to get a list of which platforms actually need binaries for people to look more favorably at SciPy. I can see that OS X is on the top of that list, surely somebody could distribute a copy of a binary for that platform (I know of people who have it installed). Also, scipy is still version 0.3 not because the code is unstable but because the layout of the packages is still open to change. Come and discuss your visions of what it should look like with us on the scipy-dev(a)scipy.net mailing list. > Making SciPy modular is in my opinion a very good approach. Why do I > need to install all of it, if I really only need a few parts? If I > can install only the subset I need, the likelihood that I will have an > easier job doing that will increase. That would make it also possible > to have sub-sets that are free of dependencies such as FORTRAN. > > On the other hand, I would think it would be a good idea to write > software that does not have dependencies on big packages such as > SciPy when possible, but only on Numeric/Numarray. That does not > exclude them from being included in SciPy. It seems to me that it > would be a desirable property if a package can be installed just with > Numeric (or numarray), with an appropriate sub-set of SciPy, or come > with the big SciPy package. I would rather see a "collection of SciPy > packages" then a "SciPy package". I'd like to see more of the development work that goes on happen under a common standard, that is all. SciPy has done a lot of work to provide an infrastructure within which to distribute packages. The field is built, but people don't seem to want to come. I understand that the Numeric / Numarray dichotomy is a big source of this problem. That is the whole reason I'm devoting a big portion of my time to the problem right now. My very ambitious goal with Numeric3 is to replace both Numeric and Numarray (heavily borrowing code from each). When I say replace them, I'm talking about the array object and umath module. I'm not talking about the other packages that use them. I want those to live under a common standard (i.e. scipy). It would be fine with me if that common standard include a scipy_lite or something like that which was easy to install, if that is desired. But, if the problem is really just availability of binary copies, then why don't we solve that instead of worrying about installation on platforms that nobody uses. In addition, even though I've developed a lot of scipy, I'm willing to throw it all out or alter it considerably in order to satisfy the demands that exist for package developers to want to put their weight behind something like SciPy. So, don't equate history with an unwillingness to cooperate. I think you'll find the people involved with scipy to be incredibly willing to cooperate to achieve mutual success. If you have users of a package that demand a certain binary of scipy to be installed, then bring them over. I can almost guarantee you that a copy of scipy will be available for them. While I do recognize the value of repeated work. When it comes to interoperability, single standards seem to work best. So, the longer this unnecessary division of resources goes on, the farther behind Python will get for doing anything but niche work. -Travis

1 0