[Python-ideas] Delayed Execution via Keyword

[David Mertz]

Dask also has a function delayed() that may be used as an decorator and in
other ways like:

>>> from dask import delayed

>>> from operator import add, mul
>>> a = delayed(add)(1, 2)
>>> b = delayed(mul)(a, 3)

>>> b
Delayed('mul-1907f29b-60a4-48af-ba2a-938556555f9b')

>>> c = b.compute()

>>> c
9

>>> b.dask
{'add-d49ba000-dd5d-4031-8c37-6514626a3d81': (<function _operator.add>, 1,
2),
 'mul-1907f29b-60a4-48af-ba2a-938556555f9b': (<function _operator.mul>,
  'add-d49ba000-dd5d-4031-8c37-6514626a3d81',
  3)}


You *can* do pretty much anything you'd want to using this approach...
including the real job of Dask, to identify latent parallelism and execute
computations on many cores or many machines in a cluster.

But actual syntax to do all of this would be really elegant.  I think for
this to be as useful as I'd want, you'd sometimes need to be to continue
delaying computation rather than doing so on every access  I guess as
syntax, the `delayed:` construct would work (I think having no colon would
more closely model `yield` and `yield from` and `await` and `async` which
this is kinda-sorta akin to).

So for example, in a hypothetical Python 3.7+:

>>> a = delayed 1 + 2

>>> b = delayed b * 3

>>> c = delayed 12/3

>>> my_lazy_func(b, delayed c) # evaluates b but not yet c
>>> b
9
>>> delayed c
<delayed object at 0x123456789>


If you want to do something like Dask... or for Dask itself to be able to
use it in some eventual version, you'd need to be able to keep objects from
evaluating even while you passed them around.  The obvious use is for
finding parallelism, but other things like passing callbacks might find
this useful too.

Dask delayed objects stay lazy until you explicitly `.compute()` on them
(which does so recursively on every lazy object that might go into the
computation).  This hypothetical new keyword would have object evaluate
eagerly *unless* you explicitly kept them lazy.  But the idea is that the
programmer would still get to decide in their code.


On Feb 16, 2017 9:53 PM, "Joseph Jevnik" <joejev at gmail.com> wrote:

> You might be interested in https://github.com/llllllllll/lazy_python,
> which implements the features you describe but instead of a keyword it uses
> a decorator.
>
> On Fri, Feb 17, 2017 at 12:24 AM, Joseph Hackman <josephhackman at gmail.com>
> wrote:
>
>> Howdy All!
>>
>> This suggestion is inspired by the question on "Efficient debug logging".
>>
>>
> I propose a keyword to mark an expression for delayed/lazy execution, for
>> the purposes of standardizing such behavior across the language.
>>
>> The proposed format is:
>> delayed: <expr>
>> i.e. log.info("info is %s", delayed: expensiveFunction())
>>
>> Unlike 'lambda' which returns a function (so the receiver must be
>> lambda-aware), delayed execution blocks are for all purposes values. The
>> first time the value (rather than location) is read, or any method on the
>> delayed object is called, the expression is executed and the delayed
>> expression is replaced with the result. (Thus, the delayed expression is
>> only every evaluated once).
>>
>> Ideally:
>> a = delayed: 1+2
>> b = a
>> print(a) #adds 1 and 2, prints 3
>> # a and b are now both just 3
>> print(b) #just prints 3
>>
>> Mechanically, this would be similar to the following:
>>
>> class Delayed():
>>     def __init__(self, func):
>>         self.__func = func
>>         self.__executed = False
>>         self.__value = None
>>
>>     def __str__(self):
>>         if self.__executed:
>>             return self.__value.__str__()
>>         self.__value = self.__func()
>>         self.__executed = True
>>         return self.__value.__str__()
>>
>>
>> def function_print(value):
>>     print('function_print')
>>     print(value)
>>
>> def function_return_stuff(value):
>>     print('function_return_stuff')
>>     return value
>>
>> function_print(function_return_stuff('no_delay'))
>>
>> function_print(Delayed(lambda: function_return_stuff('delayed')))
>>
>> delayed = Delayed(lambda: function_return_stuff('delayed_object'))
>> function_print(delayed)
>> function_print(delayed)
>>
>> Unfortunately, due to https://docs.python.org/3/r
>> eference/datamodel.html#special-lookup , this magic delayed class would
>> need to implement many magic methods, as __getattribute__ is not _always_
>> called.
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170216/32795c88/attachment.html>