how can this iterator be optimized?

[Steven D'Aprano]

On Wed, 11 Feb 2009 20:26:12 -0800, Basilisk96 wrote:

>> Don't use a generator then. If you're going to finally return a list
>> (and sorted does exactly that), build a list right from the start:
> 
> Good point. However, for the sake of argument, what if I removed the
> sorting requirement and simply wanted the generator? 

The same applies. You're problem is that you call an expensive function 
three times instead of once. Some compilers can optimize that away, but 
not in Python, because there's no guarantee that the function will be the 
same function between calls.


> I usually strive
> for comprehensions if a for loop can be reduced to such.

Any particular reason?


> Would there be
> any speed advantage in this case of a comprehension-style generator vs.
> a for-yield loop (assuming there is a way to call func(s) once per
> iteration in the comprehension)?  

I understand that comprehensions are slightly slower. I recently found a 
40% speed up on a trivial generator expression when converted to a for 
loop. Admittedly the gen expr did very little, and I haven't tested list 
comprehensions.



> In my example, func() operates on
> filename strings, so it's not too bad.. but it's possible for this
> pattern to apply to more substantial operations.  My conjecture is that
> higher func() loads would favor more the use of a simple for- yield
> loop.

If there's only one call to func(), and you ignore the (probably) fixed 
cost of jumping into a generator each time, then it shouldn't make any 
difference.

If you are comparing one call to func() in a for loop versus three calls 
to func() in a list comp or generator expression, then of course the for 
loop will be more efficient.


-- 
Steven