Mailman 3 December 2006 - SciPy-Dev

Question about Kaiser Implementation in firwin
by Buehler, Eric (AGRE) 04 Oct '08

04 Oct '08

Hello, I have been using the signal.firwin FIR design tool and I have a question about the implementation. Scipy-0.4.8 Numpy-0.9.6 I am generating a kaiser window with the following parameters: N = 128 Cutoff = ~4e7 width = .3 All of the special kaiser generation parameters in the function follow Oppenheim and Schafer well. Even the sinc function for linear phase is follows exactly. However, the last line of the firwin function is causing me some heartburn. filter_design.py 1538 win = get_window(window,N,fftbins=1) 1539 alpha = N//2 1540 m = numpy.arange(0,N) 1541 h = win*special.sinc(cutoff*(m-alpha)) 1542 return h / sum(h) Line 1542 of filter_design.py, "return h / sum(h)", normalizes the function where it doesn't seem necessary, at least in the kaiser window case. Without the normalization, the kaiser window already returns a value of 1 at the zero frequency point. This normalization scales all of the data, making the window difficult to use in the frequency domain. Can someone point me to the rationale for this line? Looking at the code, this seems to be a pretty recent change (within the last year/year and a half). Thanks, Eric Buehler eric <dot> buehler <at> smiths-aerospace <dot> com ****************************************** The information contained in, or attached to, this e-mail, may contain confidential information and is intended solely for the use of the individual or entity to whom they are addressed and may be subject to legal privilege. If you have received this e-mail in error you should notify the sender immediately by reply e-mail, delete the message from your system and notify your system manager. Please do not copy it for any purpose, or disclose its contents to any other person. The views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the company. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused, directly or indirectly, by any virus transmitted in this email. ******************************************

3 2

ZVODE, a Double Precision Complex ODE Solver
by Nils Wagner 20 Feb '08

20 Feb '08

This might be of interest. Nils From: Alan Hindmarsh <alanh(a)llnl.gov> Date: Wed, 6 Dec 2006 18:20:32 -0500 Subject: ZVODE, a Double Precision Complex ODE Solver Of the many solvers freely available for ODE initial value problems, there seem to be very few, if any, for the case of double precision complex dependent variables, even though there have been occasional user calls for this functionality. (Of course a complex system can be treated as a real system of twice the size, but direct treatment as a complex system is preferable.) I have provided such a solver: ZVODE. ZVODE is a straightforward modification of the Fortran ODE solver DVODE, for double precision real systems, making it part of a trio, SVODE/DVODE/ZVODE. ZVODE is available in two locations: (1) Netlib, in /ode (see file zvode.f), and (2) the LLNL CASC software site, www.llnl.gov/CASC/software.html (see VODE). The latter site also includes a demonstration program, zvdemo. Alan Hindmarsh Center for Applied Scientific Computing Lawrence Livermore National Laboratory

4 5

sparsetools - new and (hopefully) improved!
by Nathan Bell 10 Jan '07

10 Jan '07

I've rewritten sparse/sparsetools to speed up some of the slower bits in current code. The previous implementation of CSR/CSC matrix addition, multiplication, and conversions were all very slow, often orders of magnitude slower than MATLAB or PySparse. In the new implementation: - CSR<->CSC conversion is cheap (linear time) - sparse addition is fast (linear time) - sparse multiplication is fast (noticeably faster than MATLAB in my testing) - CSR column indices do not need to be sorted (same for CSC row indices) - the results of all the above operations contain no explicit zeros - COO->CSR and COO->CSC is linear time and sums duplicate entries This last point is useful for constructing stiffness/mass matrices. Matrix multiplication is essentially optimal (modulo Strassen-like algorithms) and is based on the SMMP algorithm I mentioned a while ago. I implemented the functions in C++ with SWIG bindings. The C++ consists of a single header file with templated functions for each operation. SWIG makes it easy to instantiate templates for each data type (float,double,long double,npy_cfloat,npy_cdouble,npy_clongdouble). For the complex types I had some trouble at first, as they do no support arithmetic operations out of the box, that is, A+B works when A and B are floats or doubles, but not npy_cfloats or npy_cdoubles. To remedy this I wrote C++ operators for the complex types (in another header file) to implement the necessary operations. I used the SWIG example in the NumPy codebase (with some alterations), so strided arrays should be handled correctly. Also, when two different types are used together (e.g. a CSR matrix of type float64 is added to a CSR matrix of type float32) SWIG overloading takes care of the conversion. It will determine that float32->float64 is a safe cast and use the float64 version of the function. Some care is needed to ensure that the most economical function is chosen, but I think this can be accomplished just by ordering the functions appropriately (e.g float32 before float64). Currently the indices are assumed to be C ints. If we wanted both 32bit and 64bit ints it would be possible to template the index type as well. One challenge in writing sparse matrix routines is that the size of the output is not known in advance. For example, multiplying two sparse matrices together leads to an unpredicitable number of nonzeros in the result. To get around this I used STL vectors as SWIG ARGOUTs. This produces a nice pythonic interface for most of the functions. For functions operating on dense 2D arrays I chose to use an INPLACE array, since the size is known in advance. The code is available here: http://graphics.cs.uiuc.edu/~wnbell/sparse/sparse.tgz The sparse directory in the archive is indented to replace the sparse directory in scipy/Lib/ I've successfully installed the above code with the standard scipy setup.py and all the unittests pass. A simple benchmark for sparse matrix multiplication SciPy and MATLAB is available here: http://graphics.cs.uiuc.edu/~wnbell/sparse/sparse_times.m http://graphics.cs.uiuc.edu/~wnbell/sparse/sparse_times.py On my laptop SciPy takes about 0.33 seconds to MATLAB's 0.55, so 40% faster. The code was all written my myself, aside from the bits pilfered from the NumPy SWIG examples and SciPy UMFPACK SWIG wrapper, so there shouldn't be any license issues. I welcome any comments, questions, and criticism you have. If you know a better or more robust way to interface the C++ with Scipy please speak up!. Also, who should I contact regarding SVN commit privileges? -- Nathan Bell wnbell(a)gmail.com

6 26

ScipyTest vs. NumpyTest
by Jarrod Millman 09 Jan '07

09 Jan '07

Hey everyone, I have a couple of fairly trivial changes that I would like to make, but I wanted to run them by everyone before going ahead. I was looking at the testing system and it seems a little confusing at this point. I would like to clean this up and make the testing more consistent. First, I wasn't easily able to find documentation. The DEVELOPERS.txt file states that every SciPy module should contain: 75 + a directory ``tests/`` that contains files ``test_<name>.py`` 76 corresponding to modules ``xxx/<name>{.py,.so,/}``. See below for 77 more details. But there is nothing more below. The old Plone SciPy site had a development/testguide.html page, but I couldn't find a corresponding page on the new sites. I migrated the information from the old Plone site: http://projects.scipy.org/scipy/scipy/wiki/TestingGuidelines I need to finish the migration to reflect the current state of affairs, which I will do as I find time. Ultimately, more documentation about the testing is needed in the code. Second, it seems that after ScipyTest was changed to NumpyTest in changeset 2029 there is some confusion about whether to use the new name or not: http://projects.scipy.org/scipy/numpy/changeset/2029 Currently the scipy codebase uses both. I think that for consistency we should only use NumpyTest. I am happy to make the changes, but I wanted to make sure that everyone agreed that this is reasonable. I also think that the NumpyTest docstring should be updated: http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/testing/info.py If no one objects, I will file a ticket with numpy. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 1

scipy.io.mmwrite() of sparse matrices
by Nils Wagner 06 Jan '07

06 Jan '07

Hi all, It's currently not possible to store sparse matrices using scipy.io.mmwrite(). Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib64/python2.4/site-packages/scipy/io/mmio.py", line 269, in mmwrite typecode = a.gettypecode() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 237, in __getattr__ raise AttributeError, attr + " not found" AttributeError: gettypecode not found http://projects.scipy.org/scipy/scipy/ticket/317 Please can someone fix this problem in the near future ? Thanks in advance. Nils >>> scipy.__version__ '0.5.3.dev2439'

4 3

new function regrid for fitpack2
by John Travers 03 Jan '07

03 Jan '07

Hi all, I've just submitted a ticket (#286) with patches to add support for the regrid function from dierckx (fitpack) to the scipy interface. regrid is used for fitting smoothing or interpolating splines to rectangular mesh data. It is more efficient and stable than using surfit (which is designed for scattered data). The patch makes use of the newstyle fitpack2 interface based on hand coded fitpack.pyf. The added class is RectBivariateSpline?. A simple test case is included. Can somebody tell me why the dierckx function surfit is currently used in interp2d? As far as I can tell from the documentation (and the fitpack author's book) surfit shouldn't be used for interpolation. It is designed for smoothing scattered data. I say this as the reason I implemented the regrid function is because interp2d wouldn't work for my data. regrid does (and *is* designed for interpolation or smoothing of rectangular data). Can I suggest that interp2d uses regrid instead (or do people need to interpolate scattered data, in which case a different library alltogether would be required)? I'm willing to work out the required patches to make these changes if people think I should. Best regards, John Travers

3 15

Time to require python 2.4?
by Tim Leslie 27 Dec '06

27 Dec '06

Hi All, I'd like to start a discussion to see whether it's time for scipy to require Python 2.4. 2.4 has been out for over two years now (released in Nov 2004) and there is at least one outstanding patch for scipy which requires 2.4[1]. AFAIK there are no major distributions for which 2.3 is the default version. What do other people think? Are there any plans to make 2.4 a requirement in the near future? Are there many people who still use 2.3? Is this something that might be considered for the next release? Cheers, Tim [1] http://projects.scipy.org/scipy/scipy/ticket/196

3 3

Astronomy stuff
by Adam Curtis 26 Dec '06

26 Dec '06

So I just remembered this "moonphase" package I have lying around my library that was given to me by an astronomer guy. Think we could possibly use some of his code (with his permission) and make some changes to it so that it fits into the library?

1 0

Fwd: [sage-devel] numpy in SAGE, etc.
by Fernando Perez 23 Dec '06

23 Dec '06

Hi Scipy-dev, I'm forwarding this from William Stein, the lead developer of SAGE (http://modular.math.washington.edu/sage/). William, I won't have any email access until next Monday, but hopefully others may pitch in. It's probably also worth mentioning these two pages: http://scipy.org/Numpy_Example_List_With_Doc http://www.hjcb.nl/python/Arrays.html Their contents may serve as the starting point for material for docstring examples (this has been suggested several times before, it just hasn't happened). Cheers, f ---------- Forwarded message ---------- From: William Stein <wstein(a)gmail.com> Date: Dec 5, 2006 11:41 PM Subject: [sage-devel] numpy in SAGE, etc. To: "sage-devel(a)googlegroups.com" <sage-devel(a)googlegroups.com> Cc: "Fernando.Perez(a)colorado.edu" <Fernando.Perez(a)colorado.edu>, oliphant(a)ee.byu.edu, pearu(a)cens.ioc.ee Hello, In case you don't know, SAGE-1.5 (http://sage.math.washington.edu/sage) will included numpy by default. Inclusion of the scipy distribution might not be far off either. We will also definitely continue to include GSL in SAGE and develop its unique functionality. I want SAGE to develop into a truly viable alternative to MATLAB (in addition to everything else it is), and it's clear to me that numpy/scipy/vtk/mayavi/gsl are crucial pieces of software if there is any hope of succcess. Anyway, to the first point of this email. I want to try some functions in numpy, so I type numpy.[tab], then say numpy.array? and I see some _minimal_ documentation but ABSOLUTELY NO EXAMPLES. The same is true for tons of the functions in numpy (and Numeric), and even Python for that matter. Anyway, this is simply *not* up to snuff for what is needed for SAGE. For SAGE my goal is that every mathematical function in the system is illustrated by examples that the user can paste into the interpreter and have work (and moreover, they are autotested). Currently there are about 12000 lines of such input already. I think this is unrelated to the whole issue with the official numpy documentation being commercial. Given the extremely limited number of examples in Numeric, numpy, and the official Python docs, it must be a conscience design decision to *not* have lots and lots of doctests. In SAGE, often files have way more docs and doctes than actual code -- again this is a design decision. The question, then, it was to do if numpy is to be included in SAGE in a way that satisfies our design goals? Some options include: (1) The file numpy/add_newdocs.py in the numpy distribution defines somehow docstrings for a lot of the numpy constructors. A SAGE developer could simply add tons of examples to this file, based on playing around, and reading the numpy book to learn what is relevant to illustrate. As each numpy distribution is released, we would *merge* this file with the one in the new numpy distribution (e.g., very easily using Mercurial). (2) You might think it would be possible to change the docstrings at runtime, but I think they may be hardcoded in (many are for code defined in extension classes). OK, so I don't have many options. Thoughts? Does anybody want to help? Any person who wants to learn numpy could probably easily write these examples along the way. Instead of just learning numpy, you could more systematically learn numpy and at the same time contribute tons of useful doctests. And finally, am I just wrong -- would Travis, etc., want these docstrings with tons of examples? Travis -- since I cc'd you, maybe you can just answer. I can completely understand if you don't want tons of doctests; it's fine if your design goals are different. By the way, SAGE Days 3 is in LA at IPAM Feb 17-21, and I hope both Fernando and Travis will consider coming. Some travel funding is available. By the way, most of the remarks above also apply to Networkx -- it's docs seem to have almost no examples. Actually, I don't think I know of *any* Python packages that do have much in the way of examples in the docstrings, at least nothing on the order of SAGE. ------------------------- Here's the official statement about the scipy module documentation, in the DEVELOPERS.txt file of the scipy distribution: "Currently there are * A SciPy tutorial by Travis E. Oliphant. This is maintained using LyX. The main advantage of this approach is that one can use mathematical formulas in documentation. * I (Pearu) have used reStructuredText formated .txt files to document various bits of software. This is mainly because ``docutils`` might become a standard tool to document Python modules. The disadvantage is that it does not support mathematical formulas (though, we might add this feature ourself using e.g. LaTeX syntax). * Various text files with almost no formatting and mostly badly out dated. * Documentation strings of Python functions, classes, and modules. Some SciPy modules are well-documented in this sense, others are very poorly documented. Another issue is that there is no consensus on how to format documentation strings, mainly because we haven't decided which tool to use to generate, for instance, HTML pages of documentation strings." -------- So evidently the scipy people have for some reason not even decided on a format for their documentation, which is partly why they don't have it. For the record, in SAGE there is a very precise documentation format that is systematically and uniformly applied throughout the system: * We liberally use latex in the docstrings. I wrote a Python function that preparses this to make it human readable, when one types foo? * The format of each docstring is as follows: function header """ 1-2 sentences summarizing what the function does. INPUT: var1 -- type, defaults, what it is var2 -- ... OUTPUT: description of output var or vars (if tuple) EXAMPLES: a *bunch* of examples, often a whole page. NOTES: misc other notes ALGORITHM: notes about the implementation or algorithm, if applicable AUTHORS: -- name (date): notes about what was done """ The INPUT and OUTPUT blocks are typeset as a latex verbatim environment. The rest is typeset using normal latex. It's good to use the itemize environment when necessary for lists, also. Essentially all the documtation in SAGE has exactly this format. VERY often I rewrite documentation for code people send me so that it is formated as above. (So if you send me code, please format it exactly as above!!!) The SAGE reference manual is autogenerated from the source code using a script I wrote that basically extracts the documentation (mainly using Python introspection), and puts it in the same format as the standard Python documentation suite, then runs the standard Python documentation tools on it. Note, however, that the structure of the reference manual is laid out by hand (i.e., order of chapters, some heading text, etc.), which is I think very important. I suggest scipy consider a similar strategy. Hey, I just got: "Successfully installed scipy-2006-12-05 Now cleaning up tmp files." on my MacBookPro, after installing the new Intel fortran compiler. -- William --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel(a)googlegroups.com To unsubscribe from this group, send email to sage-devel-unsubscribe(a)googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/ -~----------~----~----~----~------~----~------~--~---

12 51

Period table?
by CakeProphet 22 Dec '06

22 Dec '06

I think it would be fairly useful if scipy contained a module with the entire period table as a Python data type of some sort with various information (atomic number, mass, etc) of all the elements. This sound cool to anybody else?

7 11