[Python-ideas] Generator syntax hooks?

[Guido van Rossum]

On Tue, Aug 8, 2017 at 10:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 8 August 2017 at 09:06, Chris Barker <chris.barker at noaa.gov> wrote:
> > It would be nice to have an easier access to an "slice iterator" though
> --
> > one of these days I may write up a proposal for that.
>
> An idea I've occasionally toyed with [1] is some kind of "iterview"
> that wraps around an arbitrary iterable and produces lazy itertools
> based results rather than immediate views or copies.
>
> However, my experience is also that folks are *really* accustomed to
> syntactic operations on containers producing either full live views
> (e.g. memoryview or numpy slices, range as a dynamically computed
> container), or actual copies (builtin container types). Having them
> produce consumable iterators instead then gets confusing due to the
> number of operations that will implicitly consume them (including
> simple "x in y" checks).
>
> The OP's proposal doesn't fit into that category though: rather it's
> asking about the case where we have an infinite iterator (e.g.
> itertools.count(0)), and want to drop items until they start meeting
> some condition (i.e. itertools.dropwhile) and then terminate the
> iterator as soon as another condition is no longer met (i.e.
> itertools.takewhile).
>

I don't think that's what the OP meant. The original proposal seemed to
assume that it would be somehow reasonable for the input ("integers" in the
example) to be able to see and parse the condition in the generator
expression ("1000 <= x < 100000" in the example, with "x" somehow known to
be bound to the iteration value). That's at least what I think the remark
"I like mathy syntax" referred to.


> Right now, getting the "terminate when false" behaviour requires the
> use of takewhile:
>
>     {itertools.takewhile(lambda x: x < 1000000, itertools.count(1000)}
>
> In these cases, the standard generator expression syntax is an
> attractive nuisance because it *looks* right from a mathematical
> perspective, but hides an infinite loop:
>
>     {x for x in itertools.count(0) if 1000 <= x < 1000000}
>
> The most credible proposal to address this that I've seen is to borrow
> the "while" keyword in its "if not x: break" interpretation to get:
>
>     {x for x in itertools.count(0) if 1000 <= x while x < 1000000}
>
> which would be compiled as equivalent to:
>
>     x = set()
>     for x in itertools.count(0):
>         if 1000 <= x:
>             set.add(x)
>         if not x < 1000000:
>             break
>
> (and similarly for all of the other comprehension variants)
>
> There aren't any technical barriers I'm aware of to implementing that,
> with the main historical objection being that instead of the
> comprehension level while clause mapping to a while loop directly the
> way the for and if clauses map to their statement level counterparts,
> it would instead map to the conditional break in the expanded
> loop-and-a-half form:
>
>     while True:
>         if not condition:
>             break
>
> While it's taken me a long time to come around to the idea, "Make
> subtle infinite loops in mathematical code easier to avoid" *is* a
> pretty compelling user-focused justification for incurring that extra
> complexity at the language design level.
>

I haven't come around to this yet. It looks like it will make explaining
comprehensions more complex, since the translation of "while X" into "if
not X: break" feels less direct than the translations of "for x in xs" or
"if pred(x)". (In particular, your proposal seems to require more
experience with mentally translating loops and conditions into jumps --
most regulars of this forum do that for a living, but I doubt it's second
nature for the OP.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170808/7b89d61c/attachment-0001.html>