[Numpy-discussion] Status of Numeric
Mike C. Fletcher
mcfletch at rogers.com
Thu Jan 22 06:08:10 CET 2004
>On Wed, 21 Jan 2004 16:22:43 -0700, Tim Hochberg
><tim.hochberg at ieee.org> wrote:
>>The first is that last I heard the crossover point where
>>Numarray becomes faster than Numeric is about 2000 elements. It would be
>>nice if that becomes smaller, but I certainly wouldn't call it extreme.
>>In fact I'd venture that the majority of cases where numeric operations
>>are a bottleneck would already be faster under Numarray. In my
>>experience, while it's not uncommon to use short arrays, it is rare for
>>them to be a bottleneck.
>Couldn't one argue that - for example - in 3d graphic applications the
>performance as to short arrays is crucial.
I would imagine so, but then I'm a 3D guy, and really, the numeric
community is not AFAIK mostly 3D guys. Most in the numeric community
seem to be the scientific guys, who want to deal with really big
arrays. Moderately sized scenegraphs are going to have 1000s of nodes
with 3 or 4 matrices each (each matrix being 3, 4, or occasionally 16
values) and then a few hundred larger matrices for defining
vertex/colour/normal/texture content, plus a few hundred matrices
created/destroyed per frame (1/60th of a second or so)). But then most
Python scenegraph engines (OpenGLContext being the only counterexample
of which I know) are going to simply expose such matrices from their
C/C++ engine, not use Num* arrays.
Memory would likely be the most critical question for such
applications. Not sure how much extra overhead is really being
discussed. 4000 extra dictionary-based instances could really add up
memory-wise. But a 20 or 30 byte overhead/matrix isn't likely going to
kill anything that performs adequately in Python to begin with.
>But hoping that in fact numarray does come to be accepted as the
AFAIK we still can't compile PyOpenGL with numarray, so I suppose I'm on
the fence :) . Should really figure out what makes it fail some day
soon I suppose.
>>The second point is the relative speediness of Numeric at low array
>>sizes is the result that nearly all of it is implemented in C, whereas
>>much of Numarray is implemented in Python.
>Good for me. I'll be able to understand more of it.
Hmm, this does concern me a little. 3D graphics needs to do a lot of a
very small number of operations (dot and cross product, and basic
add/multiply/divide mostly). If each of those calls is going through an
extra Python layer, there might be pretty significant slowdowns.
>>This results in a larger
>>overhead for Numarray, which is why it's slower for small arrays. As I
>>understand it, the decision to base most of Numarray in Python was
>>driven by maintainability; it wasn't an attempt to optimize large arrays
>>at the expense of small ones.
>I was indeed making an assumption here. Ifs its wrong, its indeed
Sounds as though the Python layer is being used to dispatch C or C++
coded functions on internal structures of the array, which would be
consistent with the optimisation pattern (larger the dispatched
operation, the less percentage overhead involved).
>For reasons that are non-technical, I believe it much to the best that
>the community continue to look to one package of the sort as a
>standard. To me. even better if it makes it into the core
Core or not doesn't really concern me. But definitely one standard.
Even now, we're starting to get users stymied by having Numarray and
having it pretend to be Numpy so that PyOpenGL craps out with nasty
errors when it tries to do anything with Numpy.
Mike C. Fletcher
Designer, VR Plumber, Coder
More information about the Python-list