[Python-ideas] should there be a difference between generators anditerators?
dangyogi at gmail.com
Thu Sep 4 18:30:25 CEST 2008
Raymond Hettinger wrote:
> From: "Bruce Frederiksen" <dangyogi at gmail.com>
>> 1. All of the itertools and map (and I've no doubt left some others
>> out here) be extended to propagate the extra generators methods:
>> close, throw and send.
> Try wrapping each input iterator with an autocloser:
> def closer(it):
> for elem in it:
> yield elem
> if hasattr(it, 'close'):
> map(somefunc, closer(someiterable))
I don't think that this adds anything. If the 'it' iterable is run to
completion (and, presumably, does what 'close' will do anyway), this
adds nothing. And if the 'it' iterable is not run to completion, then
neither is closer, and so it still adds nothing. (And placing the 'if
hasattr' clause under a 'finally' doesn't help, because the 'finally' is
only run if the generator runs to completion or 'close' is called, and
nobody is ever going to call close here on closer). What I'm looking
for is making the following work:
with contextlib.closing(map(somefunc, someiterable)) as it:
for i in it:
if somecondition: break
# at this point someiterable.close should have been run
Intuitively, I would think that the above should work, but it doesn't.
Intuitively, I would think that if Python defines 'close' and 'throw'
methods to allow generators to properly clean up after themselves, that
itertools and the 'for' statement would both honor these so that the
'with' statement isn't even required here. I can see arguments for not
calling 'close' automatically in 'for' statements (though none for not
calling 'throw' automatically); but I don't see any arguments against
itertools honoring these methods. If any of the itertools is passing
values back from a generator (rather than a simple iterator), it would
be very nice to retain the full semantics of the generator.
Now, for map, you could suggest that Python programmers understand these
subtleties and write instead:
with contextlib.closing(someiterable) as it:
for i in map(somefunc, it):
and this works for map. But what about chain?
someiterable))) as it:
for i in it:
This is my specific situation, and I need 'close' to be called on the
currently active somegenerator within chain before the line following
'with' is executed. In my particular case, I don't think I need close
on someiterable, but it seems to make sense to do this too.
Currently, I'm not using the 'with' clause and am relying on CPython's
garbage collector to immediately call __del__ (which is defined to call
close) when the 'for' statement abandons chain. But this doesn't work
in jython or ironpython...
And, BTW, thank you for adding from_iterable to chain!
More information about the Python-ideas