[Numpy-discussion] efficient 3d histogram creation

[Chris Colbert]

after looking at it for a while, I don't see a way to easily speed it up
using pure numpy.

As a matter of fact, the behavior shown below is a little confusing. Using
fancy indexing, multiples of the same index are interpreted as a single call
to that index, probably this a for a reason that I dont currently
understand. I would think multiple calls to the same index would cause
multiple increments in the example below.

For the life of me, I can't think of how to do this 3d histogram in numpy
without a for loop.

Chris

########## example code #############

>>> a = np.arange(0,8,1).reshape((2,2,2))

>>> a

array([[[0, 1],

       [2, 3]],

       [[4, 5],

       [6, 7]]])


>>> indx =
np.array([[0,0,0],[0,0,1],[0,1,0],[0,1,1],[1,0,0],[1,0,1],[1,1,0],[1,1,1]])
>>> indx = indx.reshape(4,2,3)

>>> a[indx[:,:,0], indx[:,:,1], indx[:,:,2]]+=1

>>> a

array([[[2, 3],

        [4, 5]],

       [[6, 7],

        [8, 9]]])


>>> indx2 = np.zeros((4,2,3)).astype(np.uint8)


>>> a[indx2[:,:,0], indx2[:,:,1], indx2[:,:,2]]+=1

>>> a

array([[[3, 3],

        [4, 5]],

       [[6, 7],

        [8, 9]]])

>>>


On Thu, May 7, 2009 at 5:35 PM, Charles R Harris
<charlesr.harris at gmail.com>wrote:

>
>
> 2009/5/7 Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
>
>> Stéfan van der Walt wrote:
>> > 2009/5/7 Chris Colbert <sccolbert at gmail.com>:
>> >> This was really my first attempt at doing anything constructive with
>> Cython.
>> >> It was actually unbelievably easy to work with. I think i spent less
>> time
>> >> working on this, than I did trying to find an optimized solution using
>> pure
>> >> numpy and python.
>> >
>> > One aspect we often overlook is how easy it is to write a for-loop in
>> > comparison to vectorisation.  Besides, for-loops are sometimes easier
>> > to read as well!
>> >
>> > I think the Cython guys are planning some sort of templating, but I'll
>> > CC Dag so that he can tell us more.
>>
>> We were discussing how it would/should look like, but noone's committed
>> to implementing it so it's pretty much up in the blue I think -- someone
>> might jump in and do it next week, or it might go another year, I can't
>> tell.
>>
>> While I'm here, also note in that code Chris wrote that you want to pay
>> attention to the change of default division semantics on Cython 0.12
>> (especially for speed).
>>
>> http://wiki.cython.org/enhancements/division
>>
>
> Hi Dag,
>
> Numpy can now do separate compilations with controlled export of symbols
> when the object files are linked together to make a module. Does Cython have
> anyway of controlling the visibility of symbols or should we just include
> the right files in Numpy to get the needed macros?
>
> Chuck
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090507/86dcbfb0/attachment.html>