[Python-ideas] With clauses for generator expressions
abarnert at yahoo.com
Fri Nov 16 12:06:43 CET 2012
From: Steven D'Aprano <steve at pearwood.info>
Sent: Fri, November 16, 2012 1:53:42 AM
> > This seems to be an argument against with statements, or any other kind of
> > resource management at all besides "trust the GC".
> Certainly not. I'm saying that for many applications, explicit resource
> management is not critical -- letting the GC close the file (or whatever
> resource you're working with) -- is a perfectly adequate strategy. The mere
> existence of "faulty" gen expressions like the above example is not
> necessarily a problem.
> Think of it this way: you can optimize code for speed, for memory, and for
> resource usage. (Memory of course being a special case of resource usage.)
> You're worried about making it easy to micro-optimize generator expressions
> for resource usage.
It's not a micro-optimization, or an optimization at all. It has nothing to do
with performance, and everything to do with making your code work at all. (Or,
in some cases, making it robust—your code may work 99% of the time, or work with
CPython or POSIX but not PyPy or Windows.) For example, see Google's Python
for why they recommend always closing files.
> The only limitation here is that you can't use a context manager in a list
> comprehension or generator expression.
Yes, that's exactly the limitation (but only in generator expressions—in list
comprehensions, it can't ever matter).
> The beauty of generator expressions is that they are deliberately lean. The
> bar to fatten them up with more syntax is quite high, and I don't think you
> have come even close to getting over it.
This is one of those cases where it won't hurt you when you don't use it. You
don't have to put if clauses into generator expressions, or nest multiple
loops—and very often you don't, in which case they don't get in the way, and
your expression is concise and simple. Similarly, you won't have to put with
clauses into generator expressions, and very often you won't, in which case they
won't get in the way.
And I don't think anyone would have trouble learning or understanding it. The
expression still maps to a generator function that's just a simple tree of
one-line nested statements with a yield statement at the bottom, the only
difference is that instead of the two most common kinds of statements in such
functions, you can now use the three most common.
More information about the Python-ideas