[Tutor] list comprehension, testing for multiple conditions

[Jerry Hill]

On Wed, Aug 22, 2012 at 5:23 PM, Pete O'Connell <pedrooconnell at gmail.com> wrote:
> OK maybe I am wrong about it being slow (I thought for loops were
> slower than lis comprehensions). But I do know I need it to be as fast
> as possible if I need to run it on a thousand files each with hundreds
> of thousands of lines

You're quite likely wrong about that.  The overall runtime of your
code is very likely to end up dominated by I/O with the hard drive,
not the microsecond differences in how you process each line.  The way
to know for sure is to write code that is easy to read and does the
correct thing.  Then run the code an see if it's too slow for your
purposes.  If it is, profile the code and see where it's spending all
of its time.  Only then can you actually optimize the right places.

Once you've identified where your code needs to be faster, you're
likely to get more mileage out of a clever algorithm change than a
micro-optimization to your filtering. Or maybe you'll discover that it
really is worthwhile to optimize your filtering, because it's inside a
tight loop, and that's where all of your time is spent.  There are a
lot of tricks you can use to slightly speed up your python code, but
none of them are worth doing until you know for sure that you're
optimizing in the right place.  And if it turns out that the code's
runtime is fine as first written, you don't have to waste your time
doing lots of work for little gain.

-- 
Jerry