A Wednesday 31 October 2007, Timothy Hochberg escrigué:
On Oct 31, 2007 3:18 AM, Francesc Altet <faltet@carabos.com> wrote:
[SNIP]
Incidentally, all the improvements of the PyTables flavor of numexpr have been reported to the original authors, but, for the sake of keeping numexpr simple, they decided to implement only some of them. However, people is encouraged to try out the Pytables flavor from:
My original concern was that we'd start overflowing the code cache and slow things down. Some experiments seemed to confirm this on some processors, but it then turned out those were in error. At least that's my recollection. Because of that, and because PyTables is, as far as I know, the major user of numexpr, I suspect I'd be fine putting those changes in now. I say "suspect" since it's been a long time since I looked at the relevant patches, and I should probably look at those again before I commit myself. I just haven't had the free time and motivation to go back and look at the patches. I can't speak for David though.
Yes, I remember that you were mainly concerned with overflowing the code cache, and yes, I think you can be confident this is not the case, as my benchmarking in many CPU's (ranging from 7 year old AMD Duron, Pentiums 4 and up to relatively recents AMD Opterons), doesn't show any slow down with the PyTables version of numexpr, but rather the contrary, it speeds things up, specially in contexts where booleans and/or strided/unaligned arrays are used, where the improvement can be up to 2x on recent CPU's (as shown in my previous post). If I remember correctly, another point where you (specially David) were not very keen to include was the support for strings, arguing that numexpr is meant mainly to deal with numerical data, not textual. However, our experience is that adding this support was not too complicated (mainly due to NumPy flexibility), and can be helpful in some instances. As for one, we use them for evaluating expressions like 'array_string == "some_string"', and this can be very convenient to use when you are in the middle of potentially complex boolean expressions that you want to evaluate *fast*. At any rate, we would be glad if you would like to integrate our patches in the main numexpr, as there is not much sense to have different implementations of numexpr (most specially when it seems that there are not much users out there). So, count on us for any question you may have in this regard. Cheers, --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"