[Numpy-discussion] neighborhood iterator speed

Nadav Horesh nadavh at visionsense.com
Mon Oct 24 08:23:41 EDT 2011


* I'll try to implement the 2D iterator as far as far as my programming expertise goes. It might take few days.

* There is a risk in providing a buffer pointer, and for my (and probably most) use cases it is better for the iterator constructor to provide it. I was thinking about the possibility to give the iterator a shared memory pointer, to open a door for multiprocessing. Maybe it is better instead to provide a contiguous ndarray object to enable a sanity check.

   Nadav.


-----Original Message-----
From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of David Cournapeau
Sent: Monday, October 24, 2011 1:57 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] neighborhood iterator speed

On Mon, Oct 24, 2011 at 10:48 AM, Nadav Horesh <nadavh at visionsense.com> wrote:
> * Iterator mode: Mirror. Does the mode make a huge difference?

It could, at least in principle. The underlying translate function is
called often enough that a slight different can be significant.

> * I can not find any reference to PyArrayNeightborhoodIter_Next2d, where can I find it?

I think it would look like:

static NPY_INLINE int
PyArrayNeighborhoodIter_Next2d(PyArrayNeighborhoodIterObject* iter)
{
    _PyArrayNeighborhoodIter_IncrCoord2d(iter);
    iter->dataptr = iter->translate((PyArrayIterObject*)iter,
iter->coordinates);

    return 0;
}

The ...IncrCoord2 macro avoid one loop, which may be useful (or not).
The big issue here is the translate method call that cannot be inlined
because of the "polymorphism" of neighborhood iterator. But the only
way to avoid this would be to have many different iterators so that
the underlying translate function is known.

Copying the data makes the call to translate unnecessary (but adds the
penalty of one more conditional on every PyArrayNeighborhood_Next.

> * I think that making a copy on reset is (maybe in addition to the creation), since there is a reset for every change of the parent iterator, and after this change, the neighborhood can be determined.

you're right of course, I forgot about the parent iterator.

> * What do you think about the following idea?
>    * A neighbourhood iterator generator that accepts also a buffer to copy in the neighbourhood.
>    * A reset function that would refill the buffer after each parent iterator modification

The issue with giving the buffer is that one needs to be carefull
about the size and all. What's your usecase to pass the buffer ?

David
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


__________ Information from ESET NOD32 Antivirus, version of virus signature database 4628 (20091122) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com





More information about the NumPy-Discussion mailing list