[Numpy-discussion] Back to numexpr

[Tim Hochberg]

David M. Cooke wrote:

>On Tue, Jun 13, 2006 at 09:56:37AM -0700, Tim Hochberg wrote:
>  
>
>>[SNIP]
>>    
>>
>
>All the above is checked in already :-)
>  
>
So I see. Oops!

>  
>
>>   - Support for both numpy and numarray (use the flag --force-numarray
>>   in setup.py).
>>
>>At first glance this looks like it doesn't make things to messy, so I'm 
>>in favor of incorporating this.
>>    
>>
>
>... although I had ripped this all out. I'd rather have a numpy-compatible
>numarray layer (at the C level, this means defining macros like PyArray_DATA)
>than different code for each.
>  
>
Okey dokey. I don't feel strongly about this either way other than I'd 
rather have one version of numexpr around rather than two almost 
identical versions. Whatever makes that work would makes me happy.

>  
>
>>   - Added a new benchmark for testing boolean expressions and
>>   strided/unaligned arrays: boolean_timing.py
>>
>>Benchmarks are always good.
>>    
>>
>
>Haven't checked that in yet.
>
>  
>
>>   Things that I want to address in the future:
>>
>>   - Add tests on strided and unaligned data (currently only tested
>>   manually)
>>
>>Yep! Tests are good.
>>
>>   - Add types for int16, int64 (in 32-bit platforms), float32,
>>     complex64 (simple prec.)
>>
>>I have some specific ideas about how this should be accomplished. 
>>Basically, I don't think we want to support every type in the same way, 
>>since this is going to make the case statement blow up to an enormous 
>>size. This may slow things down and at a minimum it will make things 
>>less comprehensible.
>>    
>>
>
>I've been thinking how to generate the virtual machine programmatically,
>specifically I've been looking at vmgen from gforth again. I've got other
>half-formed ideas too (separate scalar machine for reductions?) that I'm
>working on too.
>
>But yes, the # of types does make things harder to redo :-)
>
>  
>
>>My thinking is that we only add casts for the extra 
>>types and do the computations at high precision. Thus adding two int16 
>>numbers compiles to two OP_CAST_Ffs followed by an OP_ADD_FFF, and then 
>>a OP_CAST_fF.  The details are left as an excercise to the reader ;-). 
>>So, adding int16, float32, complex64 should only require the addition of 
>>6 casting opcodes plus appropriate modifications to the compiler.
>>    
>>
>
>My thinking too.
>  
>
Yeah! Although I'm not in a hurry on this part.

 I'm remembering now that the next item on my agenda was to work on 
supporting broadcasting. I don't exactly know how this is going to work, 
although I recall having something of a plan at some point. Perhaps the 
easiest way to start out is to just test the shapes of the input array 
for compatibility. If they're compatible and don't require broadcasting, 
proceed as now. If they are incompatible, raise a "ValueError: shape 
mismatch: objects cannot be broadcast to a single shape" as numpy does. 
If they are compatible, but require broadcasting, raise a 
NotImplementedError. This should be relatively easy and makes the 
numexpr considerably more congruent with numpy.

I'm hoping that, while working on that, my plan will pop back into my 
head ;-)


[SNIP]

Regards,

-tim