unintuitive for-loop behavior

[Jussi Piitulainen]

Steve D'Aprano writes:

> On Sun, 2 Oct 2016 11:44 am, Gregory Ewing wrote:
>
>> Steve D'Aprano wrote:
>>
>>> Certainly when you call a function, the local bindings need to be
>>> created. Obviously they didn't exist prior to calling the function!
>>> I didn't think that was the difference you were referring to, and I
>>> fail to see how it could be relevant to the question of for-loop
>>> behaviour.
>> 
>> My proposed change is (mostly) equivalent to turning the
>> loop body into a thunk and passing the loop variable in as
>> a parameter.
>
> A thunk is not really well-defined in Python, because it doesn't
> exist, and therefore we don't know what properties it will have. But
> generally when people talk about thunks, they mean something like a
> light-weight anonymous function without any parameters:
>
> https://en.wikipedia.org/wiki/Thunk

According to a story (probably told in that page), thunks got their name
from being "already thunk of". Scheme at least doesn't have any special
"light-weight" anonymous functions, it just has (lambda () ...).

> You say "passing the loop variable in as a parameter" -- this doesn't
> make sense. Variables are not values in Python. You cannot pass in a
> *variable* -- you can pass in a name (the string 'x') or the *value*
> bound to the name, but there's no existing facility in Python to pass
> in a variable.

He means it like this:

   def g4159(i):
       <loop body here>

   g4159(next(g4158))

The loop variable becomes the parameter of a function. The values are
passed to that function, one by one.

[snip tangent]

> hard to talk about your hypothetical change except in hand-wavy terms:
>
> "Something magically and undefined happens, which somehow gives the
> result I want."

The following is intended to be more concrete. You, Steve, have already
declared (in a followup to me in this thread) that it is not magical
(but is obfuscated, convoluted, non-idiomatic, and so on, but then you
had not understood that the expansion is not intended to be actual
source code).

So a Python for-loop *could* behave (*could have been defined to*
behave) so that

   for v in s:
       c

is equivalent to

   try:
       gs = iter(s)
       while True:
           def gf(v):
               <nonlocal declarations>
               c
           gf(next(gs))
   except StopIteration:
       pass

where gs and gf are new names and <nonlocal declarations> make it so
that only v becomes local to gf.

>> This is the way for-loops or their equivalent are actually
>> implemented in Scheme, Ruby, Smalltalk and many other similar
>> languages, which is why they don't have the same "gotcha".
>
> Instead, they presumably have some other gotcha -- "why don't for
> loops work the same as unrolled loops?", perhaps.

I don't think so, but then I only know Scheme where tail recursion is
its own reward.

On the other hand, in Scheme I can actually implement any number of
different looping constructs if I find them desirable.  Instead of
trying to explain that something is, in principle, not only possible but
also quite straightforward, I could just show an actual implementation.

(I did that once, but then it was a beginner who thought that some
really basic thing would be impossible merely because they were such a
beginner.)

> In Python, the most obvious gotcha would be that if for-loops
> introduced their own scope, you would have to declare any other
> variables in the outer scope nonlocal. [snip]

The intention is that the for-loop would own the loop variables. Other
variables would stay in the outer scope.