Mailman 3 March 2017 - SciPy-Dev

CUTEst in Scipy
by Antonio Ribeiro March 29, 2017

March 29, 2017

Hello, I am developing my google of summer proposal about constrained optimisation in Scipy and it will be very important to have a good set of benchmarks. There is a great library with diverse optimisation benchmarks called CUTEst <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is under LGPL 2.1 license. This CUTEst library include a huge amount of problem sets and is often used in optimisation papers. Furthermore, many of the available optimisation software provide some interface to it. I am interested in using problems from this set in my project and I want to ask how should I proceed? 1) Select a subset of tests from the CUTEst library and implement them in native python under scipy. 2) Include some interface to CUTEst in scipy. By what I understand LGPL license is more restrictive than BSD-3 used by Scipy. In this case, could we ask for permission? 3) There is an interface for CUTEst implemented in the library pyOpus (under GPL3 license) <http://fides.fe.uni-lj.si/pyopus/download1.html>. Since this is a library external to Scipy (with an incompatible license) how should I proceed in this case: can I make my benchmarks dependent on it? Can we ask permission for include this interface in Scipy? Any suggestions on how to proceed and what to include in my proposal is very welcome. Antônio

4 6

Fwd: [numfocus] Grants up to $3k available to NumFOCUS projects (sponsored & affiliated)
by Ralf Gommers March 27, 2017

March 27, 2017

Hi all, This call may be of interest. If anyone has a good idea for SciPy that would fit in this budget, let's hear it! Cheers, Ralf ---------- Forwarded message ---------- From: Gina Helfrich <gina(a)numfocus.org> Date: Fri, Mar 17, 2017 at 10:51 AM Subject: Re: [numfocus] Grants up to $3k available to NumFOCUS projects (sponsored & affiliated) To: numfocus(a)googlegroups.com, devs(a)numfocus.org There is no specific template, but proposals should be kept under 2 pages. Maximum 1 submission per project. Required elements of the proposal are: - title - project description - benefit to project/community - project team - and budget Submit proposals to info(a)numfocus.org Best, Gina On Thu, Mar 16, 2017 at 1:13 PM, Bob Carpenter <carp(a)alias-i.com> wrote: > Is there a format you're expecting for proposals? > > Are applications limited to one per project? > > Hard to imagine a project won't have something to do with $3K, > so I imagine you'll get applications from all of the projects if > the proposals take less than an hour or two to put together. > > - Bob > > > On Mar 16, 2017, at 12:14 PM, Gina Helfrich <gina(a)numfocus.org> wrote: > > > > Call for Proposals - Small Development Grants > > > > NumFOCUS is asking for proposals from its sponsored and affiliated > projects for targeted small development projects with a clear benefit to > those projects. This call is motivated by the success of our 2016 > end-of-year fundraising drive; we want to direct the donated funds to our > projects in a way that has impact and visibility to donors and the wider > community. > > > > There are no restrictions on what the funding can be used for. Whether > it’s code development, documentation work, an educational, sustainability > or diversity initiative, or yet something else, we trust the projects > themselves to understand what they need and explain that need in the > proposal. > > > > Available Funding: > > • Up to $3,000 per proposal > > • Allocated funding is $9,000; depending on the number and quality > of proposals this may be adjusted up or down. > > > > Eligibility: > > • Proposals must be approved by the leadership of a NumFOCUS > sponsored or affiliated project. > > • Proposed work must have a clear outcome, achievable within 2017. > > • The call is open to applicants from any nationality and can be > performed at any university, institute or business worldwide (US export > laws permitting). > > > > Timeline: > > • Mid-March 2017: Call for Proposals released > > • 3 April 2017: deadline for proposal submissions > > • 17 April: successful proposals announced > > > > -- > > Dr. Gina Helfrich > > Communications Director, NumFOCUS > > gina(a)numfocus.org > > 512-222-5449 > > > > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "NumFOCUS" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to numfocus+unsubscribe(a)googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "NumFOCUS" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to numfocus+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Dr. Gina Helfrich Communications Director, NumFOCUS gina(a)numfocus.org 512-222-5449 <(512)%20222-5449> <http://www.numfocus.org> <https://twitter.com/NumFOCUS> <https://www.facebook.com/NumFOCUS/> -- You received this message because you are subscribed to the Google Groups "NumFOCUS" group. To unsubscribe from this group and stop receiving emails from it, send an email to numfocus+unsubscribe(a)googlegroups.com. For more options, visit https://groups.google.com/d/optout.

1 0

Re: [SciPy-Dev] CUTEst in Scipy
by Ralf Gommers March 25, 2017

March 25, 2017

On Tue, Mar 21, 2017 at 12:35 PM, Matt Newville <newville(a)cars.uchicago.edu> wrote: > > > Date: Tue, 21 Mar 2017 09:21:59 +1100 >> From: Andrew Nelson <andyfaff(a)gmail.com> >> To: SciPy Developers List <scipy-dev(a)scipy.org> >> Subject: Re: [SciPy-Dev] CUTEst in Scipy >> Message-ID: >> <CAAbtOZc2JnOvgvzH4jEB3mpLBm_YZNCj9xy1q_PG2YN2+EMD4w(a)mail.gm >> ail.com> >> Content-Type: text/plain; charset="utf-8" >> >> There are some least squares problems from the NIST test suite in the >> benchmarks/benchmarks/go_benchmark_functions, but they're couched in >> terms >> of a simple scalar (chi2) minimisation problem, and they are all bounded >> problems. >> >> > The NIST StRD test suite (http://www.itl.nist.gov/ > div898/strd/nls/nls_main.shtml) could be considered for inclusion as a > test suite for scipy.optimize. These are generally small but non-trivial > problems for unconstrained non-linear optimization. Each of the 20+ data > sets comes with 2 sets of starting values for the variables and certified > values for both best fit values and estimated uncertainties. I believe > that none of the optimizers in scipy.optimize will get the right answer to > published precision in every case, though leastsq and least_squares can > solve most of these problems reasonably well from at least 1 of the 2 > starting points. This may imply that these problems are not "sparse" in > the sense of what (as far as I understand) the GSOC project intends to > focus on. But, they are a good set of test cases to include in any > assessment of an optimization algorithm. > > The NIST StRD datasets are public domain. There is existing test code in > lmfit to use these data sets with most of the scipy optimizers. This could > easily be modified to be part of scipy. See https://github.com/lmfit/ > lmfit-py/tree/master/NIST_STRD and https://github.com/lmfit/ > lmfit-py/blob/master/tests/test_NIST_Strd.py for more details. Also, > note that the tests focus only on the quality of the results. They may > record number of iterations, but make no attempt to benchmark the runtime. > That may be a good thing - I think that there's in some places a lack of verified accuracy tests while there are checks on no increases in number of iterations. In scipy.stats we also have NIST tests for anova and linregress, those were quite useful. > I think that migrating this test suite to scipy would not be a huge > undertaking, though I cannot say whether that is an appropriate use of time > for the proposed GSOC project. > Can you open an issue for this to not lose track of the idea? Could be a nice extra for GSoC or done separately. Ralf

2 1

Adding a convenient method to create ufuncs in for scipy.stats
by Warren Weckesser March 25, 2017

March 25, 2017

I'm working on an update to the Frechet distribution in scipy.stats (see https://github.com/scipy/scipy/issues/3258 and https://github.com/scipy/scipy/pull/3275). Instead jumping through the "lazy_where" hoops that are required for conditional computations, it would be much easier to create a ufunc for the standard PDF, CDF and possibly other required functions. Easier, that is, if I use the ufunc generation tools that we have over in scipy.special. Would there be any objections to this? We already have quite a few functions for probability distributions in scipy.special: https://docs.scipy.org/doc/scipy/reference/special.html#raw-statistical-fun… I wouldn't mind creating ufuncs for some of the other distributions, too. A ufunc implementation is more efficient, simplifies the code in scipy.stats, and automatically handles broadcasting. I'm bringing this up here to see if anyone has any objections to the expansion of the statistical functions in scipy.special. Warren

4 7

GSoc-17: scipy.diff
by ashwin.pathak March 25, 2017

March 25, 2017

Hello everyone, I was writing proposal about my project and was stuck at some points as to what all has to be included in scipy.diff and what kind of mathematical tools and formulations are to be incorporated. I was thinking of using numdiff tools from scipy.optimize in order to calculate the jacobian. Can someone suggest me some reading material or useful links. Also, can someone also point out exactly what other features and mathematical formulations to include.

1 0

License / Copyright Stuff
by Tyler Reddy March 24, 2017

March 24, 2017

Hi, I'm just wondering about the 'lowest friction' way to continue contributing to scipy when your employer owns the copyright to your open source contributions (but respects the license used by the project). In short, Los Alamos would want me to put one line in one file that says they 'own' my contributions under the license of the project after Date xx/yy/zzzz. Hopefully that's a reasonable compromise (they originally wanted to put a line in each file & I said that definitely would not fly). Not sure how annoyed people would be by that, or what specific file would be used, but they have been accommodating so if there's a request on the scipy end I could bring it up with them. The biggest drawback I'm aware of is that changes to licenses may sometimes require (?) the involvement of the contributors (copyright holders)--if one such copyright holder is a national lab instead of a single individual, this might make the process a bit clunkier. Best wishes, Tyler

4 4

Re: [SciPy-Dev] Sketching-based Matrix Computations
by Jordi Montes March 24, 2017

March 24, 2017

Short answer: Yes. Long answer: Yes but not for all kind of matrices. What I am proposing here is given a matrix A create a matrix S (using CWT) with the following property with probability of at least 9/10: ||SAx||_2 = (1+-epsilon)||Ax||_2 for all x in R. These matrices (S) are called subspace embeddings. SA can be computed in O(nnz(A)) and the property So basically it is an improvement for overconstrained least-squares regression, low-rank approximation, approximation all leverage scores and l_p-regession. On top of that, implementing Blendenpik would remove the uncertainty (I would use the result of CWT as a preconditioner). Blendenpik beats LAPACK's direct dense least-squares solver by a large margin on essentially any dense *tall *matrix. On 23 March 2017 at 12:24, Matt Haberland <haberland(a)ucla.edu> wrote: > Do I understand that this is intended to be a faster but otherwise drop-in > replacement for scipy.linalg.lstsq or scipy.sparse.linalg.lsqr? > I would definitely be interested in an alternative sparse least squares > solver. I could use it. > > On Mar 21, 2017 11:09 PM, "Jordi Montes" <jomsdev(a)gmail.com> wrote: > >> *# I started this discussion on GitHub >> <https://github.com/scipy/scipy/issues/7122>. This email is a digested >> version of the conversation to check what people think about this on the >> mailing list.* >> >> >> TL;DR: *I think having an implementation for some randomize matrix >> algorithms would be great. I would like to discuss if the community finds >> it useful too.* >> >> >> Hello, >> >> I am Jordi. I have been working on a xdata >> <http://www.darpa.mil/program/xdata> open source project for the last >> year called libSkylark <https://xdata-skylark.github.io/libskylark/>. >> The library is suitable for general statistical data analysis and >> optimization applications, but it is heavily focused on distributed systems. >> >> *Randomized matrix algorithms* have been a hot topic in research the >> last years. Recent developments have shown their utility in large-scale >> machine learning and statistical data analysis applications. >> >> Although many ML applications would take advantage of the implementation >> of these algorithms the scope of them goes far beyond. I believe it >> would be a good idea integrate some of them in Scipy and I would like to >> know the opinion of others to check if it would be worth it. >> >> To my eyes, Randomized Numerical Linear Algebra is big/useful enough to >> has its submodule. However, this is something that the community has to >> decide. >> >> Meanwhile, to demonstrate that these methods can be useful I would >> implement them under, scipy.linalg where the scipy linear algebra >> functions live. The idea is to bring value since the first moment. I will >> implement Blendenpik <http://epubs.siam.org/doi/10.1137/090767911>, a least-squares >> solver based on these techniques that outperforms LAPACK by >> significant factors and scales significantly better than any QR-based >> solver. >> >> The proposal is: >> >> - >> >> Implement CWT >> <http://delivery.acm.org/10.1145/2490000/2488620/p81-clarkson.pdf?ip=198.4.8…> >> . CWT lets us create a matrix *S* such as *SA* is shorter than *A >> but conservating some properties*. >> - >> >> Compute *QR = SA* using scipy. >> - >> >> From here, there are two potential features: >> - >> >> Implement Blendenpik <http://epubs.siam.org/doi/10.1137/090767911> for >> solving Linear Square problems faster without losing any accuracy. >> - >> >> Compute the squared row norms to obtain leverage scores. >> >> Would be better to start with Blendenpik, because it solves a more >> common problem by far. >> >> I am thinking about doing it for sparse matrices first because CWT can >> take advantage of it. However, I wonder if the QR method on script takes >> advantage of sparsity too. >> >> I have already set up the development environment so I could start as >> soon as we decide what to do. As I said, I am open to discussion, so anyone >> who could be interested is welcome. >> >> Thank you, >> >> Jordi. >> >> >> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev(a)scipy.org >> https://mail.scipy.org/mailman/listinfo/scipy-dev >> >> > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev(a)scipy.org > https://mail.scipy.org/mailman/listinfo/scipy-dev > >

1 0

mailing list moved
by Ralf Gommers March 24, 2017

March 24, 2017

Hi all, This mailing list moved, it's now scipy-dev(a)python.org. Please update the stored address in the contacts list of your email client - messages sent to @scipy.org will not arrive anymore. Cheers, Ralf

1 0

GSoC 2017: BSpline support
by Aman Pratik March 23, 2017

March 23, 2017

Hello Developers, This is Aman Pratik. I am currently pursuing my B.Tech from Indian Institute of Technology, Varanasi. I am a keen software developer and not very new to the open source community. I am interested in your project "Improve support for B-splines" for GSoC 2017. I have been working in Python for the past 2 years and have good idea about Mathematical Methods and Techniques. I am quite familiar with scipy mainly as a user and also as a developer. I also have some experience with Cython. These are the PRs I have worked/working on for the past month. ENH: Add evaluation of periodic splines #6730 <https://github.com/scipy/scipy/pull/7185> DOC: Bad documentation of k in sparse.linalg.eigs See #6990 (Merged) <https://github.com/scipy/scipy/pull/6995> ENH: scipy.linalg.eigsh cannot get all eigenvalues #7004 <https://github.com/scipy/scipy/pull/7004> My GitHub Profile: https://www.github.com/amanp10 I am interested in Mathematical models and functions and therefore willing to work on splines. Also, I am familiar with Benchmark tests, Unit tests and other technical knowledge I would require for this project. I have started my study for the subject and am looking forward to guidance from the potential mentors or other developers. Thank You

3 4

migration of all scipy.org mailing lists
by Ralf Gommers March 22, 2017

March 22, 2017

Hi all, The server for the scipy.org mailing list is in very bad shape, so we (led by Didrik Pinte) are planning to complete the migration of active mailing lists to the python.org infrastructure and to decommission the lists than seem dormant/obsolete. The scipy-user mailing list was already moved to python.org a month or two ago, and that migration went smoothly. These are the lists we plan to migrate: astropy ipython-dev ipython-user numpy-discussion numpy-svn scipy-dev scipy-organizers scipy-svn And these we plan to retire: Announce APUG Ipython-tickets Mailman numpy-refactor numpy-refactor-git numpy-tickets Pyxg scipy-tickets NiPy-devel There will be a short period (<24 hours) where messages to the list may be refused, with an informative message as to why. The mailing list address will change from listname(a)scipy.org to listname(a)python.org This will happen asap, likely within a day or two. So two requests: 1. If you see any issue with this plan, please reply and keep Didrik and myself on Cc (we are not subscribed to all lists). 2. If you see this message on a numpy/scipy list, but not on another list (could be due to a moderation queue) then please forward this message again to that other list. Thanks, Ralf

1 1