[Numpy-discussion] vectorizing loops

[Timothy Hochberg]

On Nov 1, 2007 7:14 AM, David M. Cooke <cookedm at physics.mcmaster.ca> wrote:

> On Nov 1, 2007, at 08:56 , Francesc Altet wrote:
>
> > A Wednesday 31 October 2007, Timothy Hochberg escrigué:
> >> On Oct 31, 2007 3:18 AM, Francesc Altet <faltet at carabos.com> wrote:
> >>
> >> [SNIP]
> >>
> >>> Incidentally, all the improvements of the PyTables flavor of
> >>> numexpr have been reported to the original authors, but, for the
> >>> sake of keeping numexpr simple, they decided to implement only some
> >>> of them. However, people is encouraged to try out the Pytables
> >>> flavor from:
> >>
> >> My original concern was that we'd start overflowing the code cache
> >> and slow things down. Some experiments seemed to confirm this on some
> >> processors, but it then turned out those were in error. At least
> >> that's my recollection. Because of that, and because PyTables is, as
> >> far as I know, the major user of numexpr, I suspect I'd be fine
> >> putting those changes in now. I say "suspect" since it's been a long
> >> time since I looked at the relevant patches, and I should probably
> >> look at those again before I commit myself. I just haven't had the
> >> free time and motivation to go back and look at the patches.  I can't
> >> speak for David though.
> >
> > If I remember correctly, another point where you (specially David)
> > were
> > not very keen to include was the support for strings, arguing that
> > numexpr is meant mainly to deal with numerical data, not textual.
> > However, our experience is that adding this support was not too
> > complicated (mainly due to NumPy flexibility), and can be helpful in
> > some instances.  As for one, we use them for evaluating expressions
> > like 'array_string == "some_string"', and this can be very convenient
> > to use when you are in the middle of potentially complex boolean
> > expressions that you want to evaluate *fast*.
> >
> > At any rate, we would be glad if you would like to integrate our
> > patches
> > in the main numexpr, as there is not much sense to have different
> > implementations of numexpr (most specially when it seems that there
> > are
> > not much users out there).  So, count on us for any question you may
> > have in this regard.
>
> Well, I don't have much time to work on it, but if you make sure your
> patches on the scipy Trac apply clean, I'll have a quick look at them
> and apply them. Since you've had them working in production code, they
> should be good ;-)
>
> Another issue is that numexpr is still in the scipy sandbox, so only
> those who enable it will use it (or use it through PyTables). One
> problem with moving it out is that Tim reports the compile times on
> Windows are ridiculous (20 mins!).


While this is true at the default optimization (O2), it compiles reasonably
quickly at O1 and I've never been able to detect a speed difference between
versions compiled with O1 versus O2. It would probably be sufficient to
crank back the optimization on Windows.



> Maybe numexpr should become a
> scikit? It certainly doesn't need the rest of scipy.
>
>


-- 
.  __
.   |-\
.
.  tim.hochberg at ieee.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20071101/d7e156a7/attachment.html>