[Python-ideas] Delayed Execution via Keyword

[Joseph Hackman]

On 17 February 2017 at 06:10, Steven D'Aprano <steve at pearwood.info> wrote:

> On Fri, Feb 17, 2017 at 12:24:53AM -0500, Joseph Hackman wrote:
>
> > I propose a keyword to mark an expression for delayed/lazy execution, for
> > the purposes of standardizing such behavior across the language.
> >
> > The proposed format is:
> > delayed: <expr>
> > i.e. log.info("info is %s", delayed: expensiveFunction())
>
> Keywords are difficult: since by definition they are not backwards
> compatible, they make it hard for people to write version independent
> code, and will break people's code. Especially something like "delayed",
> I expect that there is lots of code that used "delayed" as a regular
> name.
>
>     if status.delayed:
>         ...
>

I think it would be key, like async/await, to narrowly define the scope in
which the word delayed functions as a keyword.

There is enough information for the compiler to know that you don't mean
the delayed keyword there because:
1. It's immediately after a dot, but more importantly
2. It's in a bare 'if'. There's no way the execution could be delayed.

delayed=False
if delayed:

is still protected by #2.

In a case where delayed would make sense, it also is unambiguous

if either_or(True, delayed:expensive_function()):

is clearly using the delayed keyword, rather than the delayed defined as
False above. (Notably, the built-in 'and' and 'or' shouldn't use delayed:,
as the short-circuiting logic is already well defined.

So, in short, in an if, for or while, the delayed keyword is only used if
it is inside a function call (or something like that).


> A new keyword means it can't be back-ported to older versions, and will
> break code.
>
>
async and await both work fine, for the reasons listed above. I'll admit
there may be more nuance required here, but it should be both possible, and
fairly intuitive based on when people would be using delayed execution.


>
> > Unlike 'lambda' which returns a function (so the receiver must be
> > lambda-aware), delayed execution blocks are for all purposes values. The
> > first time the value (rather than location) is read,
>
> What counts as "reading" a value? Based on your example below, I can't
> tell if passing the object to *any* function is enough to trigger
> evaluation, or specifically print is the magic that makes it happen.
>

So far I'm going with pretty much anything that isn't being the right-hand
of an assignment. So coercion to different types, hashing (for use as a key
in a dict or set), __repr__, etc would all be covered, as well as identity
and comparisons. i.e.:

def expensive_function(x,y):
    if x and y is not None:
        print('yippie skippy')

expensive_function(True, delayed: evaluates_to_none())

The idea put forth here would cover this, by evaluating to perform the is.


> > or any method on the delayed object is called,
>
> I don't think that can work -- it would have to be any attribute access,
> surely, because Python couldn't tell if the attribute was a method or
> not until it evaluated the lazy object. Consider:
>
> spam = delayed: complex_calculation()
> a = spam.thingy
>

Since spam.thingy is an access on spam, it would have been evaluated before
'thingy' was read.


> What's `a` at this point? Is is still some sort of lazy object, waiting
> to be evaluated? If so, how is Python supposed to know if its a method?
>
> result = a()
>
>
> > the expression is executed and the delayed
> > expression is replaced with the result. (Thus, the delayed expression is
> > only every evaluated once).
>
> That's easily done by having the "delayed" keyword cache each expression
> it sees, but that seems like a bad idea to me:
>
> spam = delayed: get_random_string()
> eggs = delayed: get_random_string()  # the same expression
>
spam.upper()  # convert to a real value
>
> assert spam == eggs  # always true, as they are the same expression
>

Since spam and eggs are two different instances of delayed expression, each
one would be evaluated separately when they are read from (as operands for
the equals operator). So no, even without the spam.upper(), they would not
match.


>
> Worse, suppose module a.py has:
>
> spam = delayed: calculate(1)
>
> and module b.py has:
>
> eggs = delayed: calculate(1)
>
> where a.calculate and b.calculate do completely different things. The
> result you get will depend on which happens to be evaluated first and
> cached, and would be a nightmare to debug. Truely spooky action-at-a-
> distance code.
>
> I think it is better to stick to a more straight-forward, easily
> understood and debugged system based on object identity rather than
> expressions.
>
>
The caching means that:
spam  = delayed: calculate(1)
eggs = spam

eggs == spam would be true, and calculate would have only been called once,
not twice.


> > Ideally:
> > a = delayed: 1+2
> > b = a
> > print(a) #adds 1 and 2, prints 3
> > # a and b are now both just 3
> > print(b) #just prints 3
>
> That would work based on object identity too.
>
> By the way, that's probably not the best example, because the keyhole
> optimizer will likely have compiled 1+2 as just 3, so you're effectively
> writing:
>
> a = delayed: 3
>
> At least, that's what I would want: I would argue strongly against lazy
> objects somehow defeating the keyhole optimizer. If I write:
>
> a = delayed: complex_calculation(1+2+3, 4.5/3, 'abcd'*3)
>
> what I hope will be compiled is:
>
> a = delayed: complex_calculation(6, 0.6428571428571429, 'abcdabcdabcd')
>
> same as it would be now (at least in CPython), apart from the "delayed:"
> keyword.
> a = delayed: complex_calculation(1+2+3, 4.5/3, 'abcd'*3)
>

I'm not sure what this is trying to do, so it's hard for me to weigh in.
It's totally fine if delayed:1+2 is exactly the same as delayed:3 and/or 3
itself.


> > Mechanically, this would be similar to the following:
> >
> > class Delayed():
> >     def __init__(self, func):
> >         self.__func = func
> >         self.__executed = False
> >         self.__value = None
> >
> >     def __str__(self):
> >         if self.__executed:
> >             return self.__value.__str__()
> >         self.__value = self.__func()
> >         self.__executed = True
> >         return self.__value.__str__()
>
> So you're suggesting that calling str(delayed_object) is the one way to
> force evaluation?
>

Not at all, pleas see above.


> I have no idea what the following code is supposed to mean.
>
>
> > def function_print(value):
> >     print('function_print')
> >     print(value)
> >
> > def function_return_stuff(value):
> >     print('function_return_stuff')
> >     return value
> >
> > function_print(function_return_stuff('no_delay'))
> >
> > function_print(Delayed(lambda: function_return_stuff('delayed')))
> >
> > delayed = Delayed(lambda: function_return_stuff('delayed_object'))
> > function_print(delayed)
> > function_print(delayed)
>

If you run this code block, it will demonstrate a number of different
orders of execution, indicating that the delayed execution does function as
expected.


>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170217/d59a0c6b/attachment-0001.html>