[Numpy-discussion] convert strides/shape/offset into nd index?

[Anne Archibald]

2009/11/30 James Bergstra <bergstrj at iro.umontreal.ca>:
> Your question involves a few concepts:
>
> - an integer vector describing the position of an element
>
> - the logical shape (another int vector)
>
> - the physical strides (another int vector)
>
> Ignoring the case of negative offsets, a physical offset is the inner
> product of the physical strides with the position vector.
>
> In these terms, you are asking how to solve the inner-product equation
> for the position vector.  There can be many possible solutions (like,
> if there is a stride of 1, then you can make that dimension account
> for the entire offset.  This is often not the solution you want.).
> For valid ndarrays though, there is at most one solution though with
> the property that every position element is less than the shape.
>
> You will also need to take into account that for certain stride
> vectors, there is no way to get certain offsets.  Imagine all the
> strides were even, and you needed to get at an odd offset... it would
> be impossible.  It would even be impossible if there were a dimension
> with stride 1 but it had shape of 1 too.
>
> I can't think of an algorithm off the top of my head that would do
> this in a quick and elegant way.

Not to be a downer, but this problem is technically NP-complete. The
so-called "knapsack problem" is to find a subset of a collection of
numbers that adds up to the specified number, and it is NP-complete.
Unfortunately, it is exactly what you need to do to find the indices
to a particular memory location in an array of shape (2,2,...,2).

What that means in practice is that either you have to allow
potentially very slow algorithms (though you know that there will
never be more than 32 different values in the knapsack, which might or
might not be enough to keep things tractable) or use heuristic
algorithms that don't always work. There are probably fairly good
heuristics, particularly if the array elements are all at distinct
memory locations (arrays with overlapping elements can arise from
broadcasting and other slightly more arcane operations).

Anne

>
> James
>
> On Sun, Nov 29, 2009 at 10:36 AM, Zachary Pincus
> <zachary.pincus at yale.edu> wrote:
>> Hi all,
>>
>> I'm curious as to what the most straightforward way is to convert an
>> offset into a memory buffer representing an arbitrarily strided array
>> into the nd index into that array. (Let's assume for simplicity that
>> each element is one byte...)
>>
>> Does sorting the strides from largest to smallest and then using
>> integer division and mod (in the obvious way) always work? (I can't
>> seem to find a counterexample, but I am not used to thinking too
>> deeply about bizarrely-strided configurations.) Is there a method that
>> doesn't involve sorting?
>>
>> Thanks,
>> Zach
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
>
> --
> http://www-etud.iro.umontreal.ca/~bergstrj
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>