[Python-ideas] Thunks (lazy evaluation) [was Re: Delay evaluation of annotations]

[Steven D'Aprano]

Let's talk about lazy evaluation in a broader sense that just function 
annotations.

If we had syntax for lazy annotation -- let's call them thunks, after 
Algol's thunks -- then we could use them in annotations as well as 
elsewhere. But if we special case annotations only, the Zen has 
something to say about special cases.


On Mon, Sep 26, 2016 at 02:57:36PM +1000, Nick Coghlan wrote:
[...]
> OK, that does indeed make more sense, and significantly reduces the
> scope for potential runtime compatibility breaks related to
> __annotations__ access. Instead, it changes the discussion to focus on
> the following main challenges:
> 
> - the inconsistency introduced between annotations (lazily evaluated)
> and default arguments (eagerly evaluated)
> - the remaining compatibility breaks (depending on implementation details)
> - the runtime overhead of lazy evaluation
> - the debugging challenges of lazy evaluation


Default arguments are a good use-case for thunks. One of the most common 
gotchas in Python is early binding of function defaults:

def func(arg=[]):
    ...

Nine times out of ten, that's probably not what you want. Now, to avoid 
all doubt, I do not want to change function defaults to late binding. 
I've argued repeatedly on comp.lang.python and elsewhere that if a 
language only offers one of early binding or late binding, it should 
offer early binding as Python does. The reason is, given early binding, 
it it trivial to simulate something like late binding:

def func(arg=None):
    if arg is None:
        arg = []
    ...

but given late binding, it is ugly and inconvenient to get a poor 
substitute for early binding when that's what you want. So, please, 
let's not have a debate over the behaviour of function defaults.

But what if we could have both? Suppose we use backticks `...` to make a 
thunk, then we could write:

def func(arg=`[]`):
    ...

to get the late binding result wanted.

Are there other uses for thunks? Potentially, they could be used for 
Ruby-like code blocks:

result = function(arg1, arg2, block=```# triple backticks
    do_this()
    do_that()
    while condition:
       do_something_else()
    print('Done')
    ```,
    another_arg=1)


but then I'm not really sure what advantage code blocks have over 
functions.


> The inconsistency argument is simply that people will be even more
> confused than they are today if default arguments are evaluated at
> definition time while annotations aren't. There is a lot of code out
> there that actively relies on eager evaluation of default arguments,
> so changing that is out of the question, which then provides a strong
> consistency argument in favour of keeping annotations eagerly
> evaluated as well.

Indeed. There are only (to my knowledge) only two places where Python 
delays evaluation of code:

- functions (def statements and lambda expressions);
- generator expressions;

where the second can be considered to be syntactic sugar for a generator 
function (def with yield). Have I missed anything?

In the same way that Haskell is fundamentally built on lazy evaluation, 
Python is fundamentally built on eager evaluation, and I don't think we 
should change that.

Until now, the only way to delay the evaluation of code (other than the 
body of a function, of course) is to write it as a string, then pass it 
to eval/exec. Thunks offer an alternative for delayed evaluation that 
makes it easier for editors to apply syntax highlighting: don't apply it 
to ordinary strings, but do apply it to thunks.

I must admit that I've loved the concept of thunks for years now, but 
I'm still looking for the killer use-case for them, the one clear 
justification for why Python should include them.

- Late-bound function default arguments? Nice to have, but we already 
have a perfectly serviceable way to get the equivalent behaviour.

- Code blocks? Maybe a Ruby programmer can explain why they're so 
important, but we have functions, including lambda.

- Function annotations? I'm not convinced thunks are needed or desirable 
for annotations.

- A better way to write code intended for delayed execution? Sounds 
interesting, but not critical.

Maybe somebody else can think of the elusive killer use-case for thunks, 
because I've been pondering this question for many years now and I'm no 
closer to an answer.


> There would likely still be some compatibility breaks around name
> access in method annotation definitions, and compatibility would also
> break for any code that actually did expect to trigger a side-effect
> at definition time. This is a much smaller scope for breakage than
> breaking __annotations__ access, but we can't assume it won't affect
> anyone as there's a lot of code out there that we'd judge to be
> questionable from the point of view of maintainability and good design
> aesthetics that nevertheless still solves the problem the author was
> aiming to solve.

It's not just published code. It's also one-off throw-away code, 
including code executed in the interactive interpreter then thrown away. 
It is really helpful to be able to monkey-patch or shadow builtins, 
insert some logging code or even a few print statements, or perhaps 
something that modifies and global variable, for debugging or to learn 
how something works. Could it be equally useful inside annotations? I 
expect so... complicated only by the fact that one needs to monkey-patch 
the *metaclass*, not the type itself.

It may be that I'm completely off-base here and this is a stupid thing 
to do. But I say that until the community has more experience with 
annotations, we shouldn't rule it out.

(Just to be clear: I'm mostly talking about interactive exploration of 
code, not production code. Python is not Ruby and we don't encourage 
heavy use of monkey-patching in production code. But it has its uses.)


> The runtime memory overhead of lazy evaluation isn't trivial. Using a
> naive function based approach:
> 
> >>> import sys
> >>> sys.getsizeof("")
> 49
> >>> sys.getsizeof(lambda: "")
> 136
> 
> And that's only the function object itself - it's not counting all the
> other objects hanging off the function object like the attribute
> dictionary. A more limited thunk type could reduce that overhead, but
> it's still going to be larger in most cases than just storing the
> evaluation result.

This is better:


py> sys.getsizeof((lambda: "").__code__)
80


> The impact on runtime speed overhead is less certain, but also likely
> to be a net negative - defining functions isn't particularly cheap
> (especially compared to literal references or a simple name lookup),
> and calling them if you actually access __annotations__ isn't going to
> be particularly cheap either.
> 
> The debugging challenge is the same one that arises with any form of
> delayed evaluation: by default, the traceback you get will point you
> to the location where the delayed evaluation took place *not* the
> location where the flawed expression was found. That problem can be
> mitigated through an exception chaining design that references the
> likely location of the actual error, but it's never going to be as
> easy to figure out as cases where the traceback points directly at the
> code responsible for the problem.

Nevertheless, an explicit thunk syntax will make this a matter of 
consenting adults: if you choose to shoot your foot off with a 
hard-to-debug thunk, you have nobody to blame but yourself.

Or whoever wrote the library that you're using. *wink*



-- 
Steve