A Wednesday 31 October 2007, Mathew Yeates escrigué: [...]
I also took a look at NumExpr. While it wasn't something I needed for vectorizing, it still looks very interesting. What kinds of performance improvements would be expected using this?
Well, for some speedup figures you can always check the first (and very exciting) message (and many of the interesting followups) of the original author, David Cooke, in this very same list: http://thread.gmane.org/gmane.comp.python.numeric.general/4081/focus=4138 This was back in March 2006. Since then, David Cooke and Tim Hochberg (with some additions of Ivan Vilata and myself) have added many interesting functionality, like the support for logical and comparison operators, many functions, like 'where', 'sqrt' and all the trigonometric and inverse trigonometric, reductions ('sum' and 'prod') and also optimizations for some specific operations, like division (a/b) or exponentation (a**b) with floating points, among others. I'm attaching a file (timing2-orig.out) with the results of the file 'timing.py' that comes with the numexpr distribution for more up-to-date figures (all the runs mentioned here have been executed on a Opteron at 2 GHz machine). On it, you can see that the speed-ups for a variety of tests range from 1.1x up to 8.7x, being the mean speedup of aproximately 2x, which is pretty good. Furthermore, and due to the specifics needs of Pytables (mainly, for doing 'in-core' selections in tables), Ivan and me have ended creating an enhanced version of numexpr that has improved support for more datatypes (long integers and strings, mainly), improvements for boolean operations and specific optimization for unidimensional arrays being strided and/or unaligned (these cases can be quite common in PyTables). You can see the result of these improvements in the attached 'boolean_timing-tables.out' file which is the output of the (also attached) benchmark 'boolean_timing.py' (see 'boolean_timing-orig.out' for the results using the original numexpr). The main things to notice is that, for these kind of 'boolean' computations, the Pytables version of numexpr can be more than 3x faster than plain Numpy and up to 1.8x faster (2.0x for the unidimensional strided case) than the original numexpr. Incidentally, all the improvements of the PyTables flavor of numexpr have been reported to the original authors, but, for the sake of keeping numexpr simple, they decided to implement only some of them. However, people is encouraged to try out the Pytables flavor from: http://www.pytables.org/trac/browser/trunk/tables/numexpr/ Or, by installing PyTables 2.x series and importing numexpr as: "from tables import numexpr" Cheers, --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"