[Numpy-discussion] last call for fixes for numpy 1.8.1rc1

Francesc Alted francesc at continuum.io
Fri Feb 28 11:17:10 EST 2014


On 2/28/14, 3:00 PM, Charles R Harris wrote:
>
>
>
> On Fri, Feb 28, 2014 at 5:52 AM, Julian Taylor 
> <jtaylor.debian at googlemail.com <mailto:jtaylor.debian at googlemail.com>> 
> wrote:
>
>     performance should not be impacted as long as we stay on the stack, it
>     just increases offset of a stack pointer a bit more.
>     E.g. nditer and einsum use temporary stack arrays of this type for its
>     initialization:
>     op_axes_arrays[NPY_MAXARGS][NPY_MAXDIMS]; // both 32 currently
>     The resulting nditer structure is then in heap space and dependent on
>     the real amount of arguments it got.
>     So I'm more worried about running out of stack space, though the limit
>     is usually 8mb so taking 128kb for a short while should be ok.
>
>     On 28.02.2014 13:32, Francesc Alted wrote:
>     > Well, what numexpr is using is basically NpyIter_AdvancedNew:
>     >
>     >
>     https://github.com/pydata/numexpr/blob/master/numexpr/interpreter.cpp#L1178
>     >
>     > and nothing else.  If NPY_MAXARGS could be increased just for
>     that, and
>     > without ABI breaking, then fine.  If not, we should have to wait
>     until
>     > 1.9 I am afraid.
>     >
>     > On the other hand, increasing the temporary arrays in nditer
>     from 32kb
>     > to 128kb is a bit worrying, but probably we should do some
>     benchmarks
>     > and see how much performance would be compromised (if any).
>     >
>     > Francesc
>     >
>     > On 2/28/14, 1:09 PM, Julian Taylor wrote:
>     >> hm increasing it for PyArrayMapIterObject would break the
>     public ABI.
>     >> While nobody should be using this part of the ABI its not
>     appropriate
>     >> for a bugfix release.
>     >> Note that as it currently stands in numpy 1.9.dev we will break
>     this ABI
>     >> for the indexing improvements.
>     >>
>     >> Though for nditer and some other functions we could change it
>     if thats
>     >> enough.
>     >> It would bump some temporary arrays of nditer from 32kb to 128kb, I
>     >> think that would still be fine, but getting to the point where
>     we should
>     >> move them onto the heap.
>
>
> These sort of changes can have subtle side effects and need lots of 
> testing in a release cycle. Bugfix release cycles are kept short by 
> restricting changes to those that look simple and safe.

Agreed.  I have just opened a ticket for having this in mind for NumPy 1.9:

https://github.com/numpy/numpy/issues/4398

-- 
Francesc Alted




More information about the NumPy-Discussion mailing list