Mailman 3 March 2014 - SciPy-Dev

Draft Proposal
by Jennifer Janani March 18, 2014

March 18, 2014

Here is a draft proposal, suggestions are really welcome. I would like to thank everyone especially Ralf and Pauli for guiding me till here. This is P.S.Janani (Jennifer), a undergraduate student, majoring in Computer Science (2nd Year). A self found love for python and high level math college courses naturally made me an ardent user of SciPy/NumPy. Beyond this I have had formal C course for past 3 years, so I am comfortable with that language too. I am comparitively new to open sourcing, and after few collaborrated projects at college level (PyCoursesync, Helios), SciPy is the first formal organization I have tried contibuting to. I am genuinely interested in contributing to SciPy this summer, particularly to scipy.special The tentative areas of work shall be: Gaussian Hypergeometric Functions (2F1) and Spherical Harmonic Functions. 1. *Gaussian Hypergeometric functions:* The implementation of the hypergeometric functions in scipy.special is lacking in several respects, and is not optimal in many parameter regimes. Gaussian hypergeometric function being the most frequently used of hypergeometric functions, focus shall be to seal the present loop holes. 1.1.*Handling the errors that arise when one or more of a,b,c is large:* Here, We shall aim to overcome the lack of accuracy that occurs when attempting to compute the Gauss hypergeometric function when one or more of the values of |Re(a)|, |Re(b)| and |Re(c)| is large. At present inaccuracies have been sighted at these parametric domains. The possible approaches to overcome this can be the following: 1.To develop the present hyp2f1ra and expand the recurrence relation method according to the table given in Pg 52 of http://people.maths.ox.ac.uk/porterm/research/pearson_final.pdf<http://bl-1.com/click/load/VmMBNlQyBzYAZ1w5U2Q-b0231>(cross reference http://www.ams.org/journals/mcom/2007-76-259/S0025-5718-07-01918-7/<http://bl-1.com/click/load/XWhdalYwAjMDZFUwCDw-b0231> ). 2.To <http://bl-1.com/click/load/AzYOOQZgVGUCZV04VWA-b0231> increase the sensitivity of hys2f1 by decreasing the error tolerance, however this is heavy process and to be avoided. *1.**2.**Extending Gaussian Hypergeometric functions for complex a,b,c:* At present SciPy's hyp2f1(a,b,c,z) entertains only z as complex, and not a,b or c. The function is to be expanded to include complex inputs too, with the domains of the parameters wide and accurate. *1.**3.Sealing the unexpected and yet-unrecognized loopholes of hyp2f1: * Almost all possible combinations of a,b,c,z (real and complex) have been mentioned in Abramowitz and Stegan ( http://people.math.sfu.ca/~cbm/aands/page_560.htm<http://bl-1.com/click/load/U2YPOAdhADEFYlcyBDY-b0231>) and . A testing on all the mentioned cases may be considered as a thorough testing and also the many issues that have been reported scipy shall be attempted to be resolved. *2.Harmonic Functions:* *2.1.Improving spherical harmonic functions:* The function for spherical harmonic function, sph_harm at present calls lpmn thus evaluating all orders <N. On the other hand, lpmv avoids the storage of values small N's by using recursion. Therefore an attempt can be made to avoid the overheads by directly evaluating the function for given degree and order. *2.2. Implementing ellipsoidal harmonic functions:* Further, we can introduce ellipsoidal harmonic functions. The thesis on O pen-source implementations of ellipsoidal harmonics ( http://arxiv.org/pdf/1204.0267v2.pdf<http://bl-1.com/click/load/BDEIP1I0UGFTNFQxCTo-b0231>), presents implementations of ellipsoidal harmonic expansions for solving problems of potential theory using separation of variables *TENTATIVE TIMELINE:* The project is for the span of 90 days ie approx 12 weeks = 480 hrs. Considering that it would be a bit slow in the starting, it would be most probably in clean synchronization, with sub-proposal 1 ending almost during mid-term evaluations. And the second part can be done post mid term evaluations. The second half may be a bit busy with me new semester beginning by August 1st.

2 5

PR for ordered QZ decompositions
by Skipper Seabold March 18, 2014

March 18, 2014

Hi All, There's also a PR [1] to add ordered QZ decompositions that should be ready to merge. scipy.linalg.ordqz It wraps the relevant LAPACK routines and provides an ordqz function similar to that of matlab or R. This is a very useful function for macroeconometrics applications. If you have comments about this PR, please speak up. Thanks, Skipper [1] https://github.com/scipy/scipy/pull/3107

1 0

TravisCI test failures on: test_discrete_basic.test_discrete_basic test_stats.test_chisquare_masked_arrays
by Andrew Nelson March 15, 2014

March 15, 2014

Hi all, I had a Travis failure on a pull request: test_discrete_basic.test_discrete_basic test_stats.test_chisquare_masked_arrays 7 tests failed in total. The test log is at: https://gist.github.com/andyfaff/9562609 https://travis-ci.org/scipy/scipy/jobs/20805700 I haven't touched code related to those failed tests, so am at a bit of a loss to explain the build failure. Can any help me? cheers, Andrew -- _____________________________________ Dr. Andrew Nelson _____________________________________

2 1

Multivariate ctypes integration PR
by Brian Lee Newsom March 14, 2014

March 14, 2014

Hello, I believe that my work on "Implement back end of faster multivariate integration #3262" has fulfilled all requests suggested and should be ready to merge. I am seeing ~2x increases in speed on single variable integration very similar to those in the current ctypes functionality. The code supports additional parameters, nquad, and no longer harms the flow of control in non ctypes cases. I would really appreciate feedback/comments, as I haven't heard back in quite a while and believe this is an important contribution. Thanks, Brian

4 3

Consideration of differential evolution minimizer being added to scipy.optimize.
by Andrew Nelson March 14, 2014

March 14, 2014

I have written some code implementing the differential evolution minimization algorithm, as invented by Storn and Price. It's a stochastic technique, not gradient based, but it's quite good at finding global minima of functions. (see http://www1.icsi.berkeley.edu/~storn/code.html, http://en.wikipedia.org/wiki/Differential_evolution) I'd like it to be considered for inclusion in scipy.optimize, and have tried to write it as such. Can anyone give advice of how to go about polishing the code, such that it's suitable for inclusion in scipy.optimize? https://github.com/andyfaff/DEsolver cheers, Andrew. -- _____________________________________ Dr. Andrew Nelson _____________________________________

6 11

GSoC application template available
by Ralf Gommers March 13, 2014

March 13, 2014

Hi GSoC students, The PSF just made their application template for this year available: https://wiki.python.org/moin/SummerOfCode/ApplicationTemplate2014. There are a few things in there that are required (for one, submit a patch to numpy or scipy if you haven't done so yet), and some good recommendations. Cheers, Ralf

1 1

Re: [SciPy-Dev] Algorithms for "best fitting geometries"
by Saullo Castro March 13, 2014

March 13, 2014

The algorithm to fit a cylinder is already working with leastsq. I will have to develop the algorithm for cones very soon, and by then I would like to know if the SciPy community has an interest for such kind of implementation. We have compared the algorithm for cylinders with some Matlab packages and the developed codes are considerably faster and require less memory. Would it be better to create a new module scipy.fitting or something similar? Regards, Saullo 2014-03-12 11:44 GMT+01:00 <scipy-dev-request(a)scipy.org>: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)scipy.org > > You can reach the person managing the list at > scipy-dev-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Algorithms for "best fitting geometries" (Saullo Castro) > 2. Configuration in tox.ini tells pep8 to ignore E501 (line too > long). (Warren Weckesser) > 3. Re: Algorithms for "best fitting geometries" (Andrew Nelson) > 4. GSoC 2014 : Discrete Wavelets Transform (Ankit Agrawal) > 5. Re: Configuration in tox.ini tells pep8 to ignore E501 (line > too long). (Tim Leslie) > 6. Re: GSoC 2014 : Discrete Wavelets Transform (Skipper Seabold) > 7. Optimizing scipy.misc.comb (Neal Donnelly) > 8. Re: Optimizing scipy.misc.comb (Richard Tsai) > 9. Re: Optimizing scipy.misc.comb (Eric Moore) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 11 Mar 2014 19:09:41 +0100 > From: Saullo Castro <saullogiovani(a)gmail.com> > Subject: [SciPy-Dev] Algorithms for "best fitting geometries" > To: Scipy-Dev <scipy-dev(a)scipy.org> > Message-ID: > < > CAHbwRz7WfYgaDk5goMbi9kLfmY3Xkr64gbttuqGW9T+6iQKKmw(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > I've been working with some algorithms for finding a best fit cylinder and > cone given a group of 3-D points. > > I thought it would be nice to add this in SciPy but I need some advise > about where to add this. > > I did not find a "fitting" module. The best I can see is inside the > "optimize" module. By the way, I find the best fit using the > "scipy.optimize.leastsq". > > Thank you, > > Saullo >

2 1

Optimizing scipy.misc.comb
by Neal Donnelly March 12, 2014

March 12, 2014

Hey everyone, I'm working on a project that relies on scipy, numpy, and scikits to segment three-dimensional images of brain tissue into discrete neurons. The project is here <https://github.com/janelia-flyem/gala> iand the contest is here <http://brainiac2.mit.edu/SNEMI3D/>. I just profiled the code to discover that 8.8% of our (gigantic) runtime is spent in the comb (n-choose-k) function in scipy/misc/common.py. I re-implemented the n-choose-k functionality in Cython and found dramatic speed improvements. Scipy (float) time across 100000 trials: 5.127797 seconds Scipy (long) time across 100000 trials: 1.583159 seconds Cython time across 100000 trials: 0.074289 seconds I also wrote tests that ensure that it produces the same answer as the scipy implementation. The code is at the end of the email. It currently only handles one number at a time rather than an ndarray, and it always does it with exact-long-int. I have two questions - 1) Why is float precision the default for comb? By my measurements, its both slower and less precise. What's the idea there? 2) Should I contribute this, and if so where should it go? I'm happy to write tests, a wrapper to allow it to handle ndarrays, etc, but I need to know that I'm barking up the right tree. I also am unclear on how scipy handles Cython and where would be the appropriate place to move the function. Thanks so much! Neal Donnelly code: cdef long _nchoosek(int n, int k): > cdef long accumulator = 1 > cdef long i > for i in range(1, k+1): > accumulator *= (n+1-i) > accumulator /= i > return accumulator > > def nchoosek(n, k): > return _nchoosek(n, k)

3 2

Configuration in tox.ini tells pep8 to ignore E501 (line too long).
by Warren Weckesser March 12, 2014

March 12, 2014

Currently, tox.ini tells pep8 to ignore line length, but it also sets `max_line_length=79`. I don't recall if there was a decision to allow long lines, or if this is just to avoid all the output that would be generated if we didn't ignore it. I'd like to remove E501 from the `ignore` list. Any objections? Warren

2 1

Algorithms for "best fitting geometries"
by Saullo Castro March 12, 2014

March 12, 2014

I've been working with some algorithms for finding a best fit cylinder and cone given a group of 3-D points. I thought it would be nice to add this in SciPy but I need some advise about where to add this. I did not find a "fitting" module. The best I can see is inside the "optimize" module. By the way, I find the best fit using the "scipy.optimize.leastsq". Thank you, Saullo

2 1