Mailman 3 March 2015 - SciPy-Dev

scipy.stats improvements
by Abraham Escalante 15 Mar '15

15 Mar '15

Hello, My name is Abraham Escalante. I would like to make a proposal for the "scipy.stats improvements" project for the Google Summer of Code. I am new to the Open Source community (although I do have experience with git and github) and this seems to me like a perfect place to start contributing. I forked the scipy/scipy project and I've been perusing some of the StatisticsCleanup issues since I would like to make my first contribution before I actually make my formal proposal (and I know it would be a great way for me to become acquainted with the code, guidelines, tests and the like). I have a few questions that I would like to trouble you with: 1) Most of the StatisticsCleanup open issues mention a "need for review" and also "StatisticsReview guidelines". *Could you refer me to the StatisticsReview guidelines?* (I have been looking but I have not been able to find it in the forked project nor the scipy documentation). *What does it mean to have an issue flagged as "review"?* see https://github.com/scipy/scipy/issues/693 for an example of what I mean. 2) I am currently going through the code (using the StatisticsCleanup issues as a guide) and starting to read the SciPy statistics tutorial. *Do you have any suggested reading* to get more familiarised with SciPy (the statistics part in particular), Numpy or to brush up on my statistics knowledge? (pretty much anything to get me up the learning curve would be useful). Thanks in advance, Abraham Escalante.

3 13

Prediction framework for (time) series
by Thomas Spura 14 Mar '15

14 Mar '15

Dear list, I like to add a prediction framework to the interpolation module and submitted pull request #4632 [1]. After a first review by Ralf Gommers, I'm writing here to discuss the broader applicability of this method and the possible API. There are two additions for now: - The `Cache` class that keeps track of the last n values of the data, that can be added with a `.add` method and the new value in the series can be predicted with `.predict`. This class should be independant of the method to predict the new value in the series, so also other methods such as splines could be added in principle so it can be applied to various cases from molecular dynamics to stock marked prediction. - The `always_stable_projector` method takes a given series and tries to predict the next value in the row with the method of Kolafa [2]. One feature is that it is designed to be time reversible, which makes it favorable to use it in molecular dynamics. For other problems, splines might be better, so how about the following prediction framework: * Adding the current `predict.py` as `_predict.py` to the interpolate module. * Rename the class to `Predict` and a `method` keyword to the constructor, similar to what `scipy.optimize.minimize` does. Do you think this would be usefull to add to scipy? Greetings, Thomas [1] https://github.com/scipy/scipy/pull/4632 [2] http://dx.doi.org/10.1002/jcc.10385

2 1

building documentation with python 3
by josef.pktd＠gmail.com 12 Mar '15

12 Mar '15

Did anyone ever try to build the sphinx documentation with python3? I'm trying to build the statsmodels documentation with python 3.4, and I run into problems all over the place. One problem is that we still keep an old/ancient copy of numpydoc included in our source. Another problem seems to be that sphinx autosummary extension uses some system decoding instead of utf-8 on Windows. I hacked my way to a successful html build, but my changes to numpydoc are dirty and I had to change the autosummary code in the sphinx installation. Josef My new computer only has python 3.4

1 0

Arbitrary Resampling
by fizyxnrd 11 Mar '15

11 Mar '15

There is a routine in scipy.signal for resampling a (periodic) signal from one sample rate to another. I have found need for (and have written) a routine that will resample from an arbitrary collection of points to another arbitrary collection of points, in such a way so that some norm of the signal is preserved. scipy.interpolate.griddata accomplishes something similar. However, it has no notion of preserving norm. For example. $ x = np.array([1, 2, 3, 4, 5]) $ y = np.array([2, 2, 10, 2, 2]) $ newx = np.array([1, 3, 5]) $ newy = interp.griddata(x, y, newx) $ newy array([2., 10., 2.]) My expected result for this resampling would be [2, 3, 2] instead, so that trapz(newy, newx) == trapz(y, x). My question is this: Does this capacity already exist in a simple scipy routine, and if not, is there interest in such an addition?

1 0

Any ufunc in SciPy with two inputs and multiple outputs?
by Jaime Fernández del Río 11 Mar '15

11 Mar '15

Hi all, There has been some work on ufunc arguments going on in numpy, e.g. multiple output parameters can now be specified in a tuple to the 'out' keyword argument, see #5621 <https://github.com/numpy/numpy/pull/5621>. There is also an ongoing discussion, see #5662 <https://github.com/numpy/numpy/issues/5662>, on whether this taking of tuples of arrays as 'out' arguments should be extended to the ufunc methods that can support it, which seems to only be `outer`. We need a ufunc with two inputs and more than one output to test this, and there are none in NumPy (the only ufuncs with more than one output are frexp and modf, which take a single input). I haven't been able to spot any such ufunc in a quick search of scipy.special, but would appreciate if someone more knowledgeable of the innards of scipy could confirm this or point me in the right direction. Thanks! Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación ..

2 1

Code review for trapz update
by K. N. 11 Mar '15

11 Mar '15

I've proposed a change to trapz, so that bin widths (`x` array deltas) are always considered positive. This allows monotonically decreasing `x` sequences to be used, in addition to monotonically increasing sequences. I am open to the idea of accepting non-monotonic sequences as well, and performing a sort on `y` to make `x` monotonic, regardless of its input form, but I think this may make the code more complex than necessary. Thoughts and suggestions are welcome. https://github.com/fizyxnrd/numpy/compare/master...trapz_binwidth_fix

8 19

GSoC 2015 - Vector math library integration for NumPy
by Amine Ilidrissi 10 Mar '15

10 Mar '15

Hello everyone, Let me introduce myself, my name's Amine Ilidrissi and I'm a computer engineering student in France. I have already used SciPy and NumPy for my projects, both in optimisation and image processing (through OpenCV). I saw that SciPy/NumPy were accepted for GSoC 2015 and I got interested in contributing back to the two libraries that helped me so much. I took the time to read the Project Ideas page and I stumbled upon one project that I liked, "Vector math library integration". I have a solid experience of coding in C and a few years of undergraduate math under my belt, so I figured that I could be a good fit for this project. Is there someone in the NumPy community with whom I could discuss this issue further? Julian Taylor for example? He's listed as a potential mentor but I can't figure out how to contact him. Thanks in advance for replying. Cheers! Amine -- *Amine Ilidrissi* *Elève-ingénieur Civil des Mines de Nancy - Engineering student at Mines Nancy* *Département Information & Systèmes - Computer Engineering* *TEDxMinesNancy - Enactus Mines Nancy*

3 2

Re: [SciPy-Dev] GSOC Optimization Project
by Nikolay Mayorov 09 Mar '15

09 Mar '15

Gregor, many thanks for your input. I noted a few useful points: 1) The option to solve the normal equation directly is indeed useful when m >> n. 2) I read the tech. report and the approach looks really good considering how often the approximation is sought as the linear combination of basis functions (which in turn depend on tunable parameters). I didn't understand the reason to store Jacobian in transposed form. What significant difference will it make? Subject: Re: [SciPy-Dev] GSOC Optimization Project From: gregor.thalhammer(a)gmail.com Date: Mon, 9 Mar 2015 19:42:45 +0100 CC: n59_ru(a)hotmail.com To: scipy-dev(a)scipy.org Am 09.03.2015 um 13:06 schrieb Nikolay Mayorov <n59_ru(a)hotmail.com>:Hi! I was tricked by stackedit and I repeat my message with the working link. Sorry for that. I did some research on suitable algorithmic approaches and want to present you my pre-proposal: https://stackedit.io/viewer#!provider=gist&gistId=d806ad6048c7e3df4797&file… I have to admit that I'm a bit overwhelmed by the amount of papers, books, etc on this subject. I tried my best to come up with some kind of a practical approach. Now I need the feedback / review. Also I'm eager to hear a word from Pauli Virtanen as we don't know yet whether this project could happen at all. Dear Nikolay, I hope you will be successful with your proposal. In the past I have been unhappy with the current MINPACK base implementation and ended up writing my own Python based implementation of LM-algorithm, specialized for my needs. Several other translations are floating around, so it seems there is really some need for a more flexible (class based) implementation, that provides easy customization by users. Years ago, my use case was fitting of images (2D Gaussian or slightly more complicated models), directly calculating the Jacobian (no numeric differentiation). Speed was important. I just want to share some findings and ideas I would be happy to be covered by future improvements to the scipy code. * In my case (many observations, few parameters) directly solving the normal equations was a lot faster, this was the main reason for me not to use the MINPACK implementation. * For the QR decomposition using the scipy implementations instead of the MINPACK gives better performance, especially when using optimized libraries (MKL, ATLAS, …) * calculating the fitting function and the Jacobian at the same time gave another speed boost, since the function value can be reused to speed up the calculation of the Jacobian. * If I remember correctly, MINPACK expects that the derivative of the function for the k-th parameter is stored in the k-th column of a 2d array. This does not fit nicely with the standard C-contiguous layout of numpy. Instead, storing it in J[k] simplifies the code and makes extension to 2D functions more easy (no shape manipulation to squeeze a 2d array into a 1d shape) * In my use case several parameters acted as linear parameters, e.g., a scaling factor and an offset. For this special case the method of ‚separable nonlinear least squares‘ or ‚variable projection method‘ reduced the number of iterations a lot and improved the robustness. It essentially only modifies how the function and Jacobian is calculated, so does not require a modification to the LM algorithm itself. But providing some means to support this for a high-level interface would be nice. My starting point was a technical report by H. B. Nielsen [1]. I hope some of these ideas are useful for your proposal. If you are interested, I put my incomplete and poorly documented code on https://gist.github.com/geggo/92c77159a9b8db5aae73 Gregor [1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.70.6004&rep=rep1&t…

2 1

GSoC '15
by rahul .poruri 09 Mar '15

09 Mar '15

Hey, I'm Rahul and I'm interested in working on implementing numerical differentiation and improving interpolation. I have done a course in computational physics and am doing a course on numerical methods in programming, through which I've grown comfortable working with numerical derivatives and interpolation (polynomial and spline). I am currently going through numdifftools but I would like to point out that I am getting a 404 while trying to access numdiff.py from statsmodels. Is this <https://github.com/statsmodels/statsmodels/blob/master/statsmodels/sandbox/…> what was being referred to in the issue? And given that numdifftools goes well and beyond simply estimating the various derivatives of a function, which part of it would you like me to implement as a PR? Regards, Ra(h)ul

2 1

Greetings SciPy Community : GSOC queries
by Ankit Kumar 09 Mar '15

09 Mar '15

Greetings SciPy Community, My name is Ankit Kumar and I am a third year student of Materials Science Engineering at IIT Kanpur, India. I am interested in knowing more about a project listed on SciPy GSOC Ideas page: *Implement scipy.diff (numerical differentiation)* I have extensive experience in developing projects implementing scientific and mathematical ideas. I built a set of scripts using panda, numpy, scipy, matplotlib to perform algorithmic trading. I also implemented a grain strain measurement based on EBSD maps software last semester using Python, numpy, scipy and matplotlibs for my Mechanical Properties Lab. In Current semester as well I am heading the development of SaaS project a cloud based extractive metallurgy simulation again using Python and its web micro-framework flask. You may find my software source code here: https://github.com/ankitkmr *Kindly guide me as to how do I start working on this idea ? Which bug fix/enhancement should I code? Expectations in Proposals ?* Kindly share any details that you would like to share or any questions that you may have for me regarding this GSOC project. Let me know what you think. Thanks a lot. Yours Sincerely Ankit Kumar IIT Kanpur ᐧ

2 1