[Numpy-discussion] Back to numexpr

David M. Cooke cookedm at physics.mcmaster.ca
Tue Jun 13 15:44:13 EDT 2006

On Tue, 13 Jun 2006 21:30:41 +0200
Francesc Altet <faltet at carabos.com> wrote:

> A Dimarts 13 Juny 2006 20:46, Tim Hochberg va escriure:
> > >Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre
> > >for some users (specially in 32-bit platforms), is a type with the same
> > >rights than the others and we would like to give support for it in
> > >numexpr. In
> > > fact, Ivan Vilata already has implemented this suport in our local copy
> > > of numexpr, so perhaps (I say perhaps because we are in the middle of a
> > > big project now and are a bit scarce of time resources) we can provide
> > > the patch against the latest version of David for your consideration.
> > > With this we can solve the problem with int64 support in 32-bit
> > > platforms (although addmittedly, the VM gets a bit more complicated, I
> > > really think that this is worth the effort)
> >
> > In addition to complexity, I worry that we'll overflow the code cache at
> > some point and slow everything down. To be honest I have no idea at what
> > point that is likely to happen, but I know they worry about it with the
> > Python interpreter mainloop.
> That's true. I didn't think about this :-/
> > Also, it becomes much, much slower to 
> > compile past a certain number of case statements under VC7, not sure
> > why. That's mostly my problem though.
> No, this is a general problem (I'd say much more in GCC, because the
> optimizer runs so slooooow). However, this should only affect to poor
> developers, not users and besides, we should find a solution for int64 in
> 32-bit platforms.

If I switch to vmgen, it can easily make two versions of the code: one using a
case statement, and another direct-threaded version for GCC (which supports
taking the address of a label, and doing a 'goto' to a variable). Won't solve
the I-cache problem, though. And there's always subroutine threading (each
opcode is a function, and the program is a list of function pointers).

We won't know until we try :)

> Or perhaps
> David can come with a better solution (vmgen from gforth? no idea what this
> is, but the name sounds sexy;-) 

The docs for it are at

|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca

More information about the NumPy-Discussion mailing list