[Python-ideas] Delayed Execution via Keyword

[Steven D'Aprano]

On Fri, Feb 17, 2017 at 12:24:53AM -0500, Joseph Hackman wrote:

> I propose a keyword to mark an expression for delayed/lazy execution, for
> the purposes of standardizing such behavior across the language.
> 
> The proposed format is:
> delayed: <expr>
> i.e. log.info("info is %s", delayed: expensiveFunction())

Keywords are difficult: since by definition they are not backwards 
compatible, they make it hard for people to write version independent 
code, and will break people's code. Especially something like "delayed", 
I expect that there is lots of code that used "delayed" as a regular 
name.

    if status.delayed:
        ...

A new keyword means it can't be back-ported to older versions, and will 
break code.


> Unlike 'lambda' which returns a function (so the receiver must be
> lambda-aware), delayed execution blocks are for all purposes values. The
> first time the value (rather than location) is read, 

What counts as "reading" a value? Based on your example below, I can't 
tell if passing the object to *any* function is enough to trigger 
evaluation, or specifically print is the magic that makes it happen.


> or any method on the delayed object is called, 

I don't think that can work -- it would have to be any attribute access, 
surely, because Python couldn't tell if the attribute was a method or 
not until it evaluated the lazy object. Consider:

spam = delayed: complex_calculation()
a = spam.thingy

What's `a` at this point? Is is still some sort of lazy object, waiting 
to be evaluated? If so, how is Python supposed to know if its a method?

result = a()


> the expression is executed and the delayed
> expression is replaced with the result. (Thus, the delayed expression is
> only every evaluated once).

That's easily done by having the "delayed" keyword cache each expression 
it sees, but that seems like a bad idea to me:

spam = delayed: get_random_string()
eggs = delayed: get_random_string()  # the same expression

spam.upper()  # convert to a real value

assert spam == eggs  # always true, as they are the same expression

Worse, suppose module a.py has:

spam = delayed: calculate(1)

and module b.py has:

eggs = delayed: calculate(1)

where a.calculate and b.calculate do completely different things. The 
result you get will depend on which happens to be evaluated first and 
cached, and would be a nightmare to debug. Truely spooky action-at-a-
distance code.

I think it is better to stick to a more straight-forward, easily 
understood and debugged system based on object identity rather than 
expressions.


> Ideally:
> a = delayed: 1+2
> b = a
> print(a) #adds 1 and 2, prints 3
> # a and b are now both just 3
> print(b) #just prints 3

That would work based on object identity too.

By the way, that's probably not the best example, because the keyhole 
optimizer will likely have compiled 1+2 as just 3, so you're effectively 
writing:

a = delayed: 3

At least, that's what I would want: I would argue strongly against lazy 
objects somehow defeating the keyhole optimizer. If I write:

a = delayed: complex_calculation(1+2+3, 4.5/3, 'abcd'*3)

what I hope will be compiled is:

a = delayed: complex_calculation(6, 0.6428571428571429, 'abcdabcdabcd')

same as it would be now (at least in CPython), apart from the "delayed:" 
keyword.
a = delayed: complex_calculation(1+2+3, 4.5/3, 'abcd'*3)


> Mechanically, this would be similar to the following:
> 
> class Delayed():
>     def __init__(self, func):
>         self.__func = func
>         self.__executed = False
>         self.__value = None
> 
>     def __str__(self):
>         if self.__executed:
>             return self.__value.__str__()
>         self.__value = self.__func()
>         self.__executed = True
>         return self.__value.__str__()

So you're suggesting that calling str(delayed_object) is the one way to 
force evaluation?



I have no idea what the following code is supposed to mean.


> def function_print(value):
>     print('function_print')
>     print(value)
> 
> def function_return_stuff(value):
>     print('function_return_stuff')
>     return value
> 
> function_print(function_return_stuff('no_delay'))
> 
> function_print(Delayed(lambda: function_return_stuff('delayed')))
> 
> delayed = Delayed(lambda: function_return_stuff('delayed_object'))
> function_print(delayed)
> function_print(delayed)




-- 
Steve