is there a C-API function for numpy which implements Python's
multidimensional indexing? Say, I have a 2d-array
PyArrayObject * M;
and an index
how do I extract the i-th row or column M[i,:] respectively M[:,i]?
I am looking for a function which gives again a PyArrayObject * and
which is a view to M (no copied data; the result should be another
PyArrayObject whose data and strides points to the correct memory
portion of M).
I searched the API documentation, Google and mailing lists for quite a
long time but didn't find anything. Can you help me?
quite often I work with block matrices. Matlab offers the convenient notation
[ a b; c d ]
to stack matrices. The numpy equivalent is kinda clumsy:
I wrote the little function `stack` that does exactly that:
stack([[a, b], [c, d]])
In my case `stack` replaced `hstack` and `vstack` almost completely.
If you're interested in including it in numpy I created a pull request
. I'm looking forward to getting some feedback!
F2py is a great tool, but my impression is that it is being left behind
by the evolution of Fortran from F90 onward. This is unfortunate; it
would be nice to be able to easily wrap new Fortran libraries.
I'm curious: has anyone been looking into what it would take to enable
f2py to handle modern Fortran in general? And into prospects for
getting such an effort funded?
I recently developed a Cython-based, OpenMP-accelerated quartic (and
cubic, quadratic) polynomial solver to address a personal research need
for quickly solving a very large number of independent low-degree
polynomial equations for both real and complex coefficients.
For example, on an Intel i7-4710MQ, running with 8 threads this code
solves approximately 1.2e7 quartic polynomial equations per second.
(With OMP_NUM_THREADS=4 the solver gives approximately 8.9e6 solutions
per second, so it seems HyperThreading in this case helps about 30%.)
The algorithms for cubics and quadratics come from Numerical Recipes
(3rd ed.), and the quartic problem is internally reduced to a cubic and
two quadratics, using well-known standard tricks.
Since to my understanding, for solving polynomial equations NumPy only
provides roots(), which works one problem at a time, I'm writing to
inquire whether there would be interest to include this functionality to
NumPy, if I contribute the code (and clean it up for integration)?
I have some previous open source contribution experience. I have
authored the IVTC and Phosphor deinterlacers for VLC, and a modular
postprocessing filter framework for the Panda3D game engine (posted at
the forums on panda3d.org, projected for version 1.10).
Currently the polynomial solver is available in a git repository hosted
by our university, the relevant files being:
I'm working on a research grant, which allows me to hold the copyright;
license is BSD.
Polysolve2 is the fast Cython+OpenMP version, while polysolve is an
earlier (slower) experimental version using only NumPy array operations.
The example module contains a usage example, and setup (in the standard
manner) instructs distutils and Cython as to how to build polysolve2
(including setting the relevant math flags and enabling OpenMP).
(The setup script includes also some stuff specific to my solver for the
Ziegler problem; basically, the "tworods" module can be ignored. I
apologize for the inconvenience.)
Juha Jeronen, Ph.D.
University of Jyväskylä
Department of Mathematical Information Technology
Due to a recent commit, Numpy master now raises an error when applying the
sign function to an object array containing NaN. Other options may be
preferable, returning NaN for instance, so I would like to open the topic
for discussion on the list.
[splitting this out from the thread-o-doom]
On Wed, Sep 23, 2015 at 8:47 AM, Chris Barker - NOAA Federal
> So here is what I think is on the table:
> We have the steering council. Which leaves two questions:
> -How big should it be?
> -Who will be on the original, "seed" council?
> Note that as I understand the current draft of the governance doc,
> once established, the council itself decides who is on the council. So
> at this point we really are ONLY deciding how it's going to start. It
> has to be bootstrapped somehow.
> However, that had been contentious enough that it would probably be a
> good idea to hash out some guidelines about the council membership.
We actually do have some of those already -- dunno if they're perfect,
but they exist :-). To make sure everyone's on the same page, here's a
condensed summary of what the draft currently says:
Joining the council: contributor must have produced "substantial" and
"sustained" contributions over at least one year + be voted on by the
current council + be interested and willing to serve. (Then there's
some language emphasising that "contributions" should *not* be read
narrowly as a euphemism for "lines of code".)
Leaving the council: Happens by choice of member, or if inactive for
one year and can't be contacted, or if inactive for two years. Former
members are listed as "emeritus" to recognize their past service.
Rejoining the council: aside from their entry on the list of emeritus
members, a former-member and a never-member are treated identically in
general, and the rules for re-joining are the same as the rules for
Proposal for seed council: "everyone who's merged a pull request since
Jan 1, 2014".
(Actual text is here:
http://thread.gmane.org/gmane.comp.python.numeric.general/61106 , see
section "Council membership".)
We didn't talk much about these -- I think mostly on the theory that
the exact details really aren't going to matter much in the end. These
specific rules are exactly the rules that Jupyter/IPython use, stolen
My interpretation is that these rules were designed to produce a
council consisting of a broad spectrum of contributors who are
actively engaged, in tune and up-to-date with the issues currently
facing the project, and broadly respected by the broader community.
The rationale for doing things this way (if we keep it) would be that
the steering council's "primary responsibility is to facilitate the
ordinary community-based decision making procedure", so you need
people who are actively engaged in community discussions; and, if
things break down then the people best positioned to resolve it are
the ones who have the best view of what went wrong, understand the
personalities involved, and so forth.
In practice, I'm sure any council interventions would involve most
members deferring to whoever they judge has the most expertise,
whether or not that person is on the council -- it's not like they'll
ever be making a decision in a vacuum.
Regarding the seed council, I just tried to pick an objective
criterion and an arbitrary date that seemed generally in keeping with
idea of "should be active in the last 1-to-2-years-ish". Fiddling with
the exact date in particular makes very little difference -- between
pushing it back to 2 years ago today or forward to 1 year ago today,
the only thing that changes is whether Pauli makes the list or not.
(And Pauli is obviously a great council candidate, though I don't know
whether he even wants to be on it.)
> Personally, I have no idea how big the council should be. Too big, and
> there is no point, consensus is harder to reach the larger the group,
> and the main (only?) role of the council is to resolve issues where
> consensus has not been reached in the larger community. But what is
> too big?
I'm a little wary of the idea of capping the council size. Assuming
you're pre-filtering for basic competence and good faith (as we are),
then the only way making the council smaller helps with decision
making is that it arbitrarily throws away the opinion of some
dedicated and valuable contributors. Plus then we have to start making
judgements like "well, person A has been around for a while but pretty
inactive recently, and person B is doing awesome stuff, should we kick
person A off the council to let person B on or...?" Judging whether
someone is or isn't a "substantial contributor" is fine, we can do
that. Having to make a relative judgement of which of two people is
the *more* "substantial contributor", though -- that sounds awful.
And given how conflict-adverse groups can be, I suspect that capping
the council size would in practice just have the effect that it never
takes in new blood. (The old effect where "science advances one
retirement at a time".)
I'd be interested to hear what Jupyter/IPython's experience with this
is, though: they currently have a 10 (!) person steering council, I
assume they'd love to have more. Having lots of contributors who are
active and engaged enough to meet the steering council qualifications
is a good problem to have :-). Technically their situation is slightly
different because their council runs on regular voting rather than
Apache-style consensus voting (a luxury they can afford because they
have a BDFL to step in if regular voting ends up disenfranchising the
minority), but I sort of get the impression that in practice they just
don't vote unless they know it will be unanimous, and it's worked out
for them so far. (To understate the case.)
> As for make-up of the council, I think we need to expand beyond people
> who have recently contributed core code.
> Yes, the council does need to have expertise to make technical
> decisions, but if you think about the likely contentious issues like
> ABI breakage, a core-code focused view is incomplete. So there should
> be representation by:
> Someone(s) with a long history of working with the code -- that
> institutional memory of why decisions were made the way they were
> could be key.
Sure -- though I can't really imagine any way of framing a rule like
this that *wouldn't* be satisfied by Chuck + Ralf + Pauli, so my guess
is that such a rule would not actually have any effect on the council
membership in practice.
> Someone(s) that may not have worked on the core code, but is a major
> player in the community, perhaps as the leader of a Numpy-dependent
> project. This would provide representation for the broad community.
Pointing out features of the current draft again for reference: In the
current text, the "NumFOCUS subcommittee" does have an external member
to provide some oversight. (So mathematically speaking, this means
that the subcommittee is not a subset -- go figure. I blame IPython.)
But, this oversight is specifically for financial matters only, not
technical ones: "This Subcommittee shall NOT make decisions about the
direction, scope or technical direction of the Project."
Thomas Caswell (one of the leaders of matplotlib) volunteered to be
our external member to start. We certainly could ask him to sit on the
steering council as well, but honestly my guess is that this would
have no effect, either positive or negative. (I know if someone asked
me to serve on a hypothetical matplotlib steering council, then I
would just... never do anything, because who am I to second-guess
matplotlib's actual developers.)
It's not like we don't already hear from downstream projects on a
regular basis. And if things have gone *so* pear-shaped that we have a
situation where the steering council feels they need to issue a
ruling, *and simultaneously* the members of the council are so
out-of-touch that they don't know or care about the needs of
downstream projects, then the situation is unsalvageable and we should
fork and start over.
But I mean, it probably wouldn't hurt either. I'm not super-wedded to
the current text. I just think we should limit how much effort we
spend trying to cover every possible situation ahead of time. If we're
clever enough to solve a problem now hypothetically, then we're
probably also clever enough to solve it later when it becomes
Nathaniel J. Smith -- http://vorpus.org
We have just released a new version of the "scipy lecture notes":
These are a consistent set of materials to learn the core aspects of the
scientific Python ecosystem, from beginner to expert. They are written
and maintained by a set of volunteers and published under a CC-BY
Highlights of the latest version includes:
* a chapter giving a introduction to statistics in Python
* a new layout with emphasis on readability including on small devices
* fully doctesting for Python 2 and 3 compatibility
We hope that you will find these notes useful, for you, your colleagues,
or your students.