[Python-ideas] With clauses for generator expressions

[Steven D'Aprano]

On 16/11/12 20:26, Andrew Barnert wrote:
> From: Steven D'Aprano<steve at pearwood.info>
> Sent: Thu, November 15, 2012 10:05:36 PM
>
>
>> As far as the given use-case is concerned:
>>
>> upperlines =  (line.upper() for line in open('foo'))
>>
>> I don't see what the concern is.  The file will remain open so long as the
>> generator is not exhausted, but that  has to be the case no matter what you
>> do. If the generator is thrown away  before being exhausted, the file will
>> eventually be closed by the garbage  collector, if only when the application
>> or script exits. For short-lived  scripts, the temporarily leakage of a file
>> handle or two is hardly likely to  be a serious problem.
>>
>> Presumably if you have a long-lived application  with many such opened
>> files, you might risk running out of file handles when  running under Jython
>> or IronPython. But I think that's a sufficiently unusual  and advanced use-
>> case that I'm not worried that this is a problem that needs  solving with
>> syntax instead of education.
>
>
> This seems to be an argument against with statements, or any other kind of
> resource management at all besides "trust the GC".


Certainly not. I'm saying that for many applications, explicit resource
management is not critical -- letting the GC close the file (or whatever
resource you're working with) -- is a perfectly adequate strategy. The mere
existence of "faulty" gen expressions like the above example is not
necessarily a problem.

Think of it this way: you can optimize code for speed, for memory, and for
resource usage. (Memory of course being a special case of resource usage.)
You're worried about making it easy to micro-optimize generator expressions
for resource usage. I'm saying that's usually premature optimization. It's
not worth new syntax complicating generator expressions to optimize the
closing of a few files.

If your application is not one of those applications where a laissez-faire
approach to resource management is acceptable, that's fine. I'm not saying
that nobody needs care about resource management! If you need to care about
your resources with more attention than benign neglect, then do so.

The only limitation here is that you can't use a context manager in a list
comprehension or generator expression. I don't care about that. Not every
problem that requires a function needs to be solvable with lambda, and not
every problem that requires a generator needs to be solvable with a generator
expression.

The beauty of generator expressions is that they are deliberately lean. The
bar to fatten them up with more syntax is quite high, and I don't think you
have come even close to getting over it.



-- 
Steven