[Python-ideas] Control Flow - Never Executed Loop Body

[Sven R. Kunze]

On 23.03.2016 00:56, Andrew Barnert via Python-ideas wrote:
> The only question is whether any of them are obvious enough (even for 
> novices). If not, we could argue whether any proposed change would be 
> significantly _more_ obvious. And, if so, then the question is whether 
> the change is worth the cost. But I think what we have is already 
> obvious enough.

It was reasonable for both Django and for jinja2 to add this. People 
actually asked for it https://github.com/mitsuhiko/jinja2/issues/77

> (Especially since 90% of the time, when you need to do something 
> special on empty, you explicitly have a sequence, not any iterable, so 
> it's just "if not seq:".)

Repeating that you think it will be a sequence will make it one.

We cannot keep track of all possible implementations in our code base. 
So, if somebody changes from sequence to iterator for whatever reason, 
it should work without weird errors.

> So really, this proposal is really just asking for syntactic sugar 
> that complicates the language in exchange for making some 
> already-understandable code a little more concise, which doesn't seem 
> worth it. 

Did you even think what you just said? Almost everything in Python is 
"just syntactic sugar" compared to most other Turing-complete languages.

To put it differently: every construct that abstracts away "goto" is 
"just syntactic sugar".

>>> What do you think
>>> about an alternative that can handle more than empty and else?
>>>
>>> for item in collection:
>>>     # do for item
>>> except NeverExecuted:
>>>     # do if collection is empty
>>>
>>> It basically merges "try" and "for" and make "for" emit EmptyCollection.
>> Does this mean that every single for-loop that doesn't catch
>> NeverExecuted (or EmptyCollection) will raise an exception?
> Elsewhere he mentioned that EmptyCollection would be a subclass of StopIteration.
>
> Presumably, every iterator type (or just the builtin ones, and hopefully "many" others?) would have special code to raise EmptyCollection if empty. Like this pseudocode for list_iterator:
>
>      def __next__(self):
>          if not self.lst:
>              raise EmptyCollection
>          elif self.i >= len(self.lst):
>              raise StopIteration
>          else:
>              i = self.i
>              self.i += 1
>              return self.lst[self.i]

Interesting. That would be an alternative approach.

> Or, alternatively, for itself would do this.

I think the most important question is: do we want to support the 
"ifempty" feature in the first place?

I can see a lot of discussion around it; that makes it controversial and 
that means there is half/half support/rejection (support by more people 
+ rejection by less people but those have many good questions we would 
need to answer before we get this proposal to work). The only real 
reason against it so far: "it makes the language more complicated 
because I don't need it". Not entirely compelling but understandable.

I further see that people would like this feature GIVEN a very GOOD 
syntax/approach and I entirely agree with that.

> The for loop bytecode would have to change to stash an "any values seen" flag somewhere such that if it sees StopIteration and hasn't seen any values, it converts that to an EmptyCollection. Or any of the other equivalents (e.g., the compiler could unroll the first PyIter_Next from loop from the rest of them to handle it specially).

Something like that.

> But this seems like it would add a lot of overhead and complexity to every loop whether desired or not.

If the "for" does not have any empty-equivalent clauses, there is no 
need to introduce that overhead in the first place.

So we can conclude:

1) none overhead for regular "for"s
2) less overhead for "for-ifempty" because it would be done in C and not 
in Python

>> If not, then how will this work? Is this a special kind of
>> exception-like process that *only* operates inside for loops?
>>
>> What will an explicit "raise NeverExecuted" do?
> Presumably that's the same question as what an explicit raise StopIteration does. Just as there's nothing stopping you from writing a __next__ method that raises StopIteration but then yields more values of called again, there's nothing stopping you from raising NeverExecuted pathologically, but you shouldn't do so. M
>
>>> So, independent of the initial "never executed loop body" use-case, one
>>> could also emulate the "else" clause by:
>>>
>>> for item in collection:
>>>     # do for item
>>> except StopIteration:
>>>     # do after the loop
>> That doesn't work, for two reasons:
>>
>> (1) Not all for-loops use iterators. The venerable old "sequence
>> protocol" is still supported for sequences that don't support __iter__.
>> So there may not be any StopIteration raised at all.
> I think there always is.
>
> IIRC, PyObject_Iter (the C API function used by iter and by for loops) actually constructs a sequence iterator object if the object doesn't have tp_iter (__iter__ for Python types) but does have  tp_sequence (__getitem__ for Python types, but, e.g., dict has __getitem__ without having tp_sequence). And the for loop doesn't treat that sequence iterator any different from "real" iterators returned by __iter__; it just calls tp_next (__next__) until StopIteration. (And the "other half" of the old-style sequence protocol, that lets old-style sequences be reversed if they have a length, is similarly implemented by the C API underneath the reversed function.)
>
> I'm on my phone right now, so I can't double-check any of the details, but I'm 80% sure they're all at least pretty close...

I think I would have to deal with the old protocol given the proposal is 
accepted.

>> (2) Even if StopIteration is raised, the for-loop catches it (in a
>> manner of speaking) and consumes it.
>>
>> So to have this work, we would need to have the for-loop re-raise
>> StopIteration... but what happens if you don't include an except
>> StopIteration clause? Does every bare for-loop with no "except" now
>> print a traceback and halt processing? If not, why not?
> I think this could be made to work: a for loop without an except clause handles StopIteration the same as today (by jumping to the else clause), but one that does have one or more except clauses just treats it like a normal exception.
>
> Of course this would mean for/except/else is now legal but useless, which could be confusing ("why does my else clause no longer run when I add an 'except ValueError' clause?").

One could disallow "else" in case any "except" is defined.

> More generally, I think the fact that for/except StopIteration is almost but not quite identical to plain for would be confusing more often than helpful.

You bet how people think about "else". "So, 'else' is always executed 
after the for?" "Yes, but only when there is no 'break' executed in the 
'for'" "... *thinking* ... okay ..."

> But I think it is a coherent proposal, even if it's not one I like. :)

Best,
Sven