Mailman 3 March 2016 - SciPy-Dev

dense output option in integrate.ode
by Evgeni Burovski Feb. 20, 2017

Feb. 20, 2017

Hi David, Hi Jim, > I am new to SciPy-dev. Will the dense output option in scipy.integrate.ode > become available in 0.17.0? This is a feature already available in the > original FORTAN, but wasn't implemented in the wrapper. If the feature is sent as a pull request against the scipy master branch, the PR is reviewed by the maintainers of the integrate package and merged into master before the release split, then yes, it would be available in 0.17.0. So far I do not see any progress towards it. > I wrote these dense output extensions that you listed: > > https://github.com/jddmartin/scipy/tree/dense_output_from_dopri5_and_dop853 > https://github.com/jddmartin/dense_output_example_usage > > but I didn't issue any pull request to scipy. I sent this message to the > scipy developers list: > http://article.gmane.org/gmane.comp.python.scientific.devel/19635/ > explaining the changes. I was hoping for some feedback before issuing a > pull request. Ah, I see that the email likely fell through the cracks back in April. Sorry about that. You might want to ping that email thread once more or send a pull request on github (or both). <snip> Cheers, Evgeni

3 5

RFC: sparse DOK array
by Evgeni Burovski April 15, 2016

April 15, 2016

Hi, TL;DR: Here's a draft implementation of a 'sparse array', suitable for incremental assembly. Comments welcome! https://github.com/ev-br/sparr Scipy sparse matrices are generally great. One area where they are not very great is the formats suitable for incremental assembly (esp LIL and DOK matrices). This has been discussed for a while, see, for instance, https://github.com/scipy/scipy/pull/2908#discussion_r16955546 I spent some time to see how hard is it to build a dictionary-of-keys data structure without Python overhead. Turns out, not very hard. Also, when constructing a sparse data structure now, it seems to make sense to make it a sparse *array* rather than *matrix*, so that it supports both elementwise and matrix multiplication. A draft implementation is available at https://github.com/ev-br/sparr and I'd be grateful for any comments, large or small. Some usage examples are in the README file which Github shows at the front page above. First and foremost, I'd like to gauge interest in the community ;-). Does it actually make sense? Would you use such a data structure? What is missing in the current version? Short to medium term, some issues I see are: * 32-bit vs 64-bit indices. Scipy sparse matrices switch between index types. I wonder if this is purely backwards compatibility. Naively, it seems to me that a new class could just always use 64-bit indices, but this might be too naive? * Data types and casting rules. For now, I basically piggy-back on numpy's rules. There are several slightly different ones (numba has one?), and there might be an opportunity to simplify the rules. OTOH, inventing one more subtly different set of rules might be a bad idea. * "Object" dtype. So far, there isn't one. I wonder if it's needed or having only numeric types would be enough. * Interoperation with numpy arrays and other sparse matrices. I guess __numpy_ufunc__ would *the* solution here, when available. For now, I do something simple based on special-casing and __array_priority__. Sparse matrices almost work, but there are glitches. This is just a non-exhaustive list of things I'm thinking about. If there's something else (and I'm sure there is plenty!) I'm all ears --- either in this email thread or in the issue tracker. If someone feels like commenting on implementation details, sure, none of those is cast in stone and plumbing is definitely in a bit of a flux. Cheers, Evgeni

4 9

Blog-post that explains what Blaze actually is and where Pluribus project now lives.
by Travis Oliphant March 29, 2016

March 29, 2016

I have emailed this list in the past explaining what is driving my open source efforts now. Here is a blog-post that may help some of you understand at little bit of the history of Blaze, DyND Numba, and other related developments as they relate to scaling up and scaling out array-computing in Python. http://technicaldiscovery.blogspot.com/2016/03/anaconda-and-hadoop-story-of… This post and these projects do not have anything to do with the future of the NumPy and/or SciPy projects which are now in great hands guiding their community-driven development. The post is however, a discussion of additional projects that will hopefully benefit some of you as well, and for which your feedback and assistance is welcome. Best, -Travis -- *Travis Oliphant, PhD* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io

1 0

ANN: LAST CALL for SciPy (Scientific Python) 2016 Conference Talk / Poster Proposals - Due Friday 4/1
by Courtenay Godshall (Enthought) March 29, 2016

March 29, 2016

**ANN: LAST CALL for SciPy 2016 Conference Talk / Poster Proposals (Scientific Computing with Python) - Final Deadline - Friday April 1st** <http://www.scipy2016.scipy.org/> SciPy 2016, the 15th annual Scientific Computing with Python conference, will be held July 11-17, 2016 in Austin, Texas. SciPy is a community dedicated to the advancement of scientific computing through open source Python software for mathematics, science, and engineering. The annual SciPy Conference brings together over 650 participants from industry, academia, and government to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. The full program will consist of <http://scipy2016.scipy.org/ehome/146062/332967/?&&> 2 days of tutorials (July 11-12), <http://scipy2016.scipy.org/ehome/146062/332968/?&&> 3 days of talks (July 13-15), and 2 days of <http://scipy2016.scipy.org/ehome/146062/332969/?&&> developer sprints (July 16-17). More info is available on the conference website at <http://www.scipy2016.scipy.org/> http://scipy2016.scipy.org (where you can sign up for the mailing list); or follow <https://twitter.com/scipyconf> @scipyconf on Twitter. ---------------------------------------------------------------------------- --------------------------------- **SUBMIT A SCIPY 2016 TALK / POSTER PROPOSAL - DUE APRIL 1, 2016* ---------------------------------------------------------------------------- --------------------------------- Submissions for talks and posters are welcome on our website ( <http://scipy2016.scipy.org/> http://scipy2016.scipy.org). In your abstract, please provide details on what Python tools are being employed, and how. The talk and poster submission deadline is March 25th, 2016 SciPy 2016 will include 3 major topic tracks and 8 mini-symposia tracks. Major topic tracks include: - Scientific Computing in Python - Python in Data Science (Big data and not so big data) - High Performance Computing Mini-symposia will include the applications of Python in: - Earth and Space Science - Engineering - Medicine and Biology - Social Sciences - Special Purpose Databases - Case Studies in Industry - Education - Reproducibility If you have any questions or comments, feel free to contact us at: scipy-organizers(a)scipy.org ----------------------------------------------------------- **SCIPY 2016 REGISTRATION IS OPEN** ----------------------------------------------------------- Please register early. SciPy early bird registration until May 22, 2016. Register at <http://scipy2016.scipy.org/> http://scipy2016.scipy.org. Plus, enter our t-shirt design contest to win a free registration. (Send a vector art file to scipy(a)enthought.com by March 31 to enter). Important dates: April 1: Talk and Poster Proposals Due May 11: Plotting Contest Submissions Due Apr 22: Tutorials Announced Apr 22: Financial Aid Submissions Due May 4: Talk and Posters Announced May 22: Early Bird Registration Deadline Jul 11-12: SciPy 2016 Tutorials Jul 13-15: SciPy 2016 General Conference Jul 16-17: SciPy 2016 Sprints We look forward to an exciting conference and hope to see you in Austin in July! The Scipy 2016 Committee <http://scipy2016.scipy.org/> http://scipy2016.scipy.org/

1 0

Numpy 1.11.0 released
by Charles R Harris March 27, 2016

March 27, 2016

Hi All, I'm pleased to announce the release of Numpy 1.11.0. This release is the result of six months of work comprising 371 merged pull requests submitted by 112 authors containing many bug fixes and improvements. Highlights are: - The datetime64 type is now timezone naive. - A dtype parameter has been added to ``randint``. - Improved detection of two arrays possibly sharing memory. - Automatic bin size estimation for ``np.histogram``. - Speed optimization of A @ A.T and dot(A, A.T). - New function ``np.moveaxis`` for reordering array axes. Source files and documentation can be found on Sourceforge <https://sourceforge.net/projects/numpy/files/NumPy/1.11.0/>, while source files and OS X wheels for Python 2.7, 3.3, 3.4, and 3.5 can be installed from Pypi. Note that this is the last release to support Python 2.6, 3.2, and 3.3. Contributors are listed below in alphabetical order with the names of new contributors starred. Abdullah Alrasheed* Aditya Panchal* Alain* Alex Griffing Alex Rogozhnikov* Alex Willmer Alexander Heger* Allan Haldane Anatoly Techtonik* Andrew Nelson Anne Archibald Antoine Pitrou Antony Lee* Behzad Nouri Bertrand Lefebvre Blake Griffith Boxiang Sun* Brigitta Sipocz* Carl Kleffner Charles Harris Chris Hogan* Christoph Gohlke Colin Jermain* Cong Ma* Daniel David Freese David Sanders* Dmitry Odzerikho* Dmitry Zagorny* Eric Larson* Eric Moore Ethan Kruse* Eugene Krokhalev* Evgeni Burovski* Francis T. O'Donovan* François Boulogne* Gabi Davar* Gerrit Holl Gopal Singh Meena* Greg Yang* Greg Young* Gregory R. Lee Griffin Hosseinzadeh* Hassan Kibirige* Holger Kohr* Ian Henriksen Iceman9* Jaime Fernandez James Camel* Jason King* John Bjorn Nelson* John Kirkham Jonathan Helmus Jonathan Underwood* Joseph Fox-Rabinovitz* Julian Taylor Julien Dubois* Julien Lhermitte* Julien Schueller* Jörn Hees* Konstantinos Psychas* Lars Buitinck Luke Zoltan Kelley* MaPePeR* Mad Physicist* Mark Wiebe Marten van Kerkwijk Matthew Brett Matthias Geier* Maximilian Trescher* Michael K. Tran* Michael Behrisch* Michael Currie* Michael Löffler* Nathaniel Hellabyte* Nathaniel J. Smith Nick Papior* Nicolas Calle* Nicolás Della Penna* Olivier Grisel Pauli Virtanen Peter Iannucci Phaiax* Ralf Gommers Rehas Sachdeva* Ronan Lamy Ruediger Meier* Ryan Grout* Ryosuke Okuta* Rémy Léone* Sam Radhakrishnan* Samuel St-Jean* Sebastian Berg Simon Conseil* Stephan Hoyer Stuart Archibald* Stuart Berg Sumith* Tapasweni Pathak* Thomas Robitaille Tobias Megies* Tushar Gautam* Varun Nayyar* Vincent Legoll* Warren Weckesser Wendell Smith Yash Mehrotra* Yifan Li* endolith floatingpointstack* ldoddema* yolanda15* Thanks to all who worked on this release. Chuck

2 1

minimize reporting successful but not searching (with default method)
by David Mikolas March 22, 2016

March 22, 2016

I have a situation where scipy.optimize.minimize returns the initial starting value as the answer without really iterating, and returns "successful" status, using default method. It works fine if I specify Nelder-Mead, but my question is "why does minimize return successful when it isn's, and what can I do (if anything) to flag this behavior? I originally posted in scipy-user but decided it may be more appropriate here. The question is also here http://stackoverflow.com/q/36110998 and if someone can also leave an answer there as well, that would be great! but not necessary. One particular - I'm using calls to Skyfield, which may or may not have something to do with it. (http://rhodesmill.org/skyfield/) FULL OUTPUT using DEFAULT METHOD: status: 0 success: True njev: 1 nfev: 3 hess_inv: array([[1]]) fun: 1694.98753895812 x: array([ 10000.]) message: 'Optimization terminated successfully.' jac: array([ 0.]) nit: 0 FULL OUTPUT using Nelder-Mead METHOD: status: 0 nfev: 63 success: True fun: 3.2179306044608054 x: array([ 13053.81011963]) message: 'Optimization terminated successfully.' nit: 28

3 4

Slight change in lstsq accuracy for 0.17 - expected?
by Matthew Brett March 21, 2016

March 21, 2016

Hi, I just noticed that our (neuroimaging) test suite fails for scipy 0.17.0: https://travis-ci.org/matthew-brett/nipy/jobs/115471563 This turns out to be due to small changes in the result of `scipy.linalg.lstsq` between 0.16.1 and 0.17.0 - on the same platform - the travis Ubuntu in this case. Is that expected? I haven't investigated deeply, but I believe the 0.16.1 result (back into the depths of time) is slightly more accurate. Is that possible? Thanks for any insight, Matthew

5 21

Numpy 1.11.0rc2 released.
by Charles R Harris March 19, 2016

March 19, 2016

Hi All, Numpy 1.11.0rc2 has been released. There have been a few reversions and fixes since 1.11.0rc1, but by and large things have been pretty stable and unless something terrible happens, I will make the final next weekend. Source files and documentation can be found on Sourceforge <https://sourceforge.net/projects/numpy/files/NumPy/1.11.0rc2/>, while source files and OS X wheels for Python 2.7, 3.3, 3.4, and 3.5 can be installed from Pypi. Please test and yell loudly if there is a problem. Chuck

1 0

Gaussian Process hyperparameter tuning
by Jan-Samuel Wagner March 13, 2016

March 13, 2016

Is anyone busy developing a Gaussian Process strategy for hype-parameter tuning? Seems strange there is only GridSearchCV and RandomSearchCV methods for sklearn. Please let me know. I would love to develop this feature.

3 2

Fwd: [Numpy-discussion] GSoC?
by Evgeni Burovski March 3, 2016

March 3, 2016

---------- Forwarded message ---------- From: Ralf Gommers <ralf.gommers(a)gmail.com> Date: Wed, Feb 10, 2016 at 11:02 PM Subject: Re: [Numpy-discussion] GSoC? To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> On Wed, Feb 10, 2016 at 11:55 PM, Ralf Gommers <ralf.gommers(a)gmail.com> wrote: > > > > On Wed, Feb 10, 2016 at 11:48 PM, Chris Barker <chris.barker(a)noaa.gov> wrote: >> >> Thanks Ralf, >> >>> Note that we have always done a combined numpy/scipy ideas page and submission. For really good students numpy may be the right challenge, but in general scipy is easier to get started on. >> >> >> yup -- good idea. Is there a page ready to go, or do we need to get one up? (I don't even know where to put it...) > > > This is last year's page: https://github.com/scipy/scipy/wiki/GSoC-2015-project-ideas > > Some ideas have been worked on, others are still relevant. Let's copy this page to -2016- and start editing it and adding new ideas. I'll start right now actually. OK first version: https://github.com/scipy/scipy/wiki/GSoC-2016-project-ideas I kept some of the ideas from last year, but removed all potential mentors as the same people may not be available this year - please re-add yourselves where needed. And to everyone who has a good idea, and preferably is willing to mentor for that idea: please add it to that page. Here are a couple of vague ideas which I'd like to hash out on the list first. Both are mostly in the request-for-comment, feedback-welcome stage. (I am going to write them out even though I cannot guarantee that I'll be available to mentor either of them.) 1. Numeric differentiation. The current proposal focuses on bundling numdifftools. I think there's room for a complementary set of routines for numeric differentiation with fixed step, or limited intelligence w.r.t. the step selection. In fact, there is a fairly comprehensive implementation in scipy already, a by-product of last year's GSoC. It's not exposed publicly, and is hidden as scipy.optimize._numdiff.approx_derivative. I suspect that what's already there covers the bulk of the implementation, and what's left is a only a bit of work to spec out the user-facing API, especially for the broadcasting behavior, and then replacing semi-broken bits of scipy.misc.derivative, scipy.optimize.approx_fprime etc. The original PR has quite extensive discussion, https://github.com/scipy/scipy/pull/4884 And here's an unfinished attempt at following up on that discussion: https://gist.github.com/ev-br/7a1edb3f250bd375c46a 2. Optimization benchmarks. There are two quite large bodies of work on benchmarking various optimization algorithms: - Andrew Nelson's global optimizers benchmarks, https://github.com/scipy/scipy/pull/4191. - Nikolay Mayorov's nonliear LSQ benchmarks, some of which are available in scipy/benchmarks, and the most recent version is at https://github.com/nmayorov/lsq-benchmarks I think both can be very valuable for a broader scipy ecosystem, and both of these herculean efforts are currently blocked on the question of how to actually run them. It seems to me that it would be a useful project to come up with a standardized way of running these sorts of benchmarks and publishing the results. Ideally, it would include, * a runner. Maybe an ASV-based one, but then the project might include some enhancements to asv itself. * a way of pushlishing the results (again, asv has one, is it sufficient?) * a self-contained environment for running and publishing (a VM image? a docker container?) So that by the end of the project we would have a web page with the reference results, some automation for refreshing when needed, and a documented way of reproducing them. Thoughts? Evgeni

1 0