[Python-ideas] Generator syntax hooks?

[Nick Coghlan]

On 8 August 2017 at 09:06, Chris Barker <chris.barker at noaa.gov> wrote:
> It would be nice to have an easier access to an "slice iterator" though --
> one of these days I may write up a proposal for that.

An idea I've occasionally toyed with [1] is some kind of "iterview"
that wraps around an arbitrary iterable and produces lazy itertools
based results rather than immediate views or copies.

However, my experience is also that folks are *really* accustomed to
syntactic operations on containers producing either full live views
(e.g. memoryview or numpy slices, range as a dynamically computed
container), or actual copies (builtin container types). Having them
produce consumable iterators instead then gets confusing due to the
number of operations that will implicitly consume them (including
simple "x in y" checks).

The OP's proposal doesn't fit into that category though: rather it's
asking about the case where we have an infinite iterator (e.g.
itertools.count(0)), and want to drop items until they start meeting
some condition (i.e. itertools.dropwhile) and then terminate the
iterator as soon as another condition is no longer met (i.e.
itertools.takewhile).

Right now, getting the "terminate when false" behaviour requires the
use of takewhile:

    {itertools.takewhile(lambda x: x < 1000000, itertools.count(1000)}

In these cases, the standard generator expression syntax is an
attractive nuisance because it *looks* right from a mathematical
perspective, but hides an infinite loop:

    {x for x in itertools.count(0) if 1000 <= x < 1000000}

The most credible proposal to address this that I've seen is to borrow
the "while" keyword in its "if not x: break" interpretation to get:

    {x for x in itertools.count(0) if 1000 <= x while x < 1000000}

which would be compiled as equivalent to:

    x = set()
    for x in itertools.count(0):
        if 1000 <= x:
            set.add(x)
        if not x < 1000000:
            break

(and similarly for all of the other comprehension variants)

There aren't any technical barriers I'm aware of to implementing that,
with the main historical objection being that instead of the
comprehension level while clause mapping to a while loop directly the
way the for and if clauses map to their statement level counterparts,
it would instead map to the conditional break in the expanded
loop-and-a-half form:

    while True:
        if not condition:
            break

While it's taken me a long time to come around to the idea, "Make
subtle infinite loops in mathematical code easier to avoid" *is* a
pretty compelling user-focused justification for incurring that extra
complexity at the language design level.

Cheers,
Nick.

[1] https://mail.python.org/pipermail/python-ideas/2010-April/006983.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia