[Numpy-discussion] Need help for implementing a fast clip in numpy (was slow clip)

[Francesc Altet]

El dt 09 de 01 del 2007 a les 23:19 +0900, en/na David Cournapeau va
escriure:
> Hi,
> 
>     I am finally implementing a C function to replace the current slow 
> implementation of clip in python as promised a few weeks ago. The idea 
> is to implement it as the following:
> 
> def clip(input, min, max):
>     a   = input.copy()
>     putmask(a, a <= min, min)
>     putmask(a, a >= max, max)
>     return a
> 
> I don't have that much experience in writing general numpy functions, so 
> I was wondering of other people could advise me on the following points.
> 

Sorry, but not real experience writing extensions directly in C.
However, you may want to experiment using numexpr for doing what you
want. It's relatively easy to extend numexpr and adding a new opcode to
its virtual machine. I'm attaching a patch for implementing such a clip
routine (only for floating point types, but given the example, it should
be straightforward to add support for integers as well).

Also, you should note that using the fancy indexing of numpy seems to
perform better than the putmask approach. Below are my figures for a
small benchmark (also attached) for testing the performance of clip
using several approaches:

time (putmask)--> 1.38
time (where)--> 2.713
time (numexpr where)--> 1.291
time (fancy+assign)--> 0.967
time (numexpr clip)--> 0.596

It is interesting to see there how fancy-indexing + assignation is quite
more efficient than putmask.

Unfortunately, integrating this clip version officially in numexpr
doesn't seem realistic as the main authors are trying hard to avoid
cluttering the VM with too many opcodes (they don't want to overload the
instruction cache of the CPU). Nevertheless, as the implementation of
clip in numexpr should be rather optimal, its performance can still be
used as a reference for other implementations. 

HTH and good luck with your own clip implementation,

-- 
Francesc Altet    |  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth
-------------- next part --------------
A non-text attachment was scrubbed...
Name: clip-bench.py
Type: text/x-python
Size: 922 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20070110/159bc9f5/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: numexpr-clip.patch
Type: text/x-patch
Size: 2116 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20070110/159bc9f5/attachment.bin>