Generator function performance

Oren Tirosh oren-py-l at
Mon Dec 17 09:02:34 EST 2001

At first, I thought of generator function mostly in terms of convenience:
the resulting source is shorter and more readable compared to other 
implementations.  Then I found an unexpected benefit: generators are 
also very efficient.

A function call in Python is quite expensive.  Suspending and restoring 
the state of a generator is relatively cheap in comparison.  A dataflow 
process consisting of a chain of generators each reading its upstream 
generator is very fast, especially if each generator function only uses 
builtins and functions from binary modules.  

Writing such dataflows is very convenient because you only have to focus on 
one stage at a time. Each stage may read data from its upstream iterator 
more or less often than once for each result it yields and there is no need
for intermediate rate matching buffers.


More information about the Python-list mailing list