[Python-ideas] Delayed Execution via Keyword

[Joshua Morton]

David, can you elaborate on your example?

if we replaced line four with

    >>> x = my_lazy_func(b, delayed c)

what would the value of `x` be, and how would this differ from either

    >>> x = delayed my_lazy_func(b, delayed c)

or

    >>> x = delayed my_lazy_func(b, c)

To put it another way, why does my_lazy_func being called not evaluate c,
or are you implying that it too is some kind of special delayed function
that needs to be explicitly computed, like in dask?

--Josh

On Fri, Feb 17, 2017 at 1:23 AM David Mertz <mertz at gnosis.cx> wrote:

> Dask also has a function delayed() that may be used as an decorator and in
> other ways like:
>
> >>> from dask import delayed
>
> >>> from operator import add, mul
> >>> a = delayed(add)(1, 2)
> >>> b = delayed(mul)(a, 3)
>
> >>> b
> Delayed('mul-1907f29b-60a4-48af-ba2a-938556555f9b')
>
> >>> c = b.compute()
>
> >>> c
> 9
>
> >>> b.dask
> {'add-d49ba000-dd5d-4031-8c37-6514626a3d81': (<function _operator.add>, 1,
> 2),
>  'mul-1907f29b-60a4-48af-ba2a-938556555f9b': (<function _operator.mul>,
>   'add-d49ba000-dd5d-4031-8c37-6514626a3d81',
>   3)}
>
>
> You *can* do pretty much anything you'd want to using this approach...
> including the real job of Dask, to identify latent parallelism and execute
> computations on many cores or many machines in a cluster.
>
> But actual syntax to do all of this would be really elegant.  I think for
> this to be as useful as I'd want, you'd sometimes need to be to continue
> delaying computation rather than doing so on every access  I guess as
> syntax, the `delayed:` construct would work (I think having no colon would
> more closely model `yield` and `yield from` and `await` and `async` which
> this is kinda-sorta akin to).
>
> So for example, in a hypothetical Python 3.7+:
>
> >>> a = delayed 1 + 2
>
> >>> b = delayed b * 3
>
> >>> c = delayed 12/3
>
> >>> my_lazy_func(b, delayed c) # evaluates b but not yet c
> >>> b
> 9
> >>> delayed c
> <delayed object at 0x123456789>
>
>
> If you want to do something like Dask... or for Dask itself to be able to
> use it in some eventual version, you'd need to be able to keep objects from
> evaluating even while you passed them around.  The obvious use is for
> finding parallelism, but other things like passing callbacks might find
> this useful too.
>
> Dask delayed objects stay lazy until you explicitly `.compute()` on them
> (which does so recursively on every lazy object that might go into the
> computation).  This hypothetical new keyword would have object evaluate
> eagerly *unless* you explicitly kept them lazy.  But the idea is that the
> programmer would still get to decide in their code.
>
>
> On Feb 16, 2017 9:53 PM, "Joseph Jevnik" <joejev at gmail.com> wrote:
>
> You might be interested in https://github.com/llllllllll/lazy_python,
> which implements the features you describe but instead of a keyword it uses
> a decorator.
>
> On Fri, Feb 17, 2017 at 12:24 AM, Joseph Hackman <josephhackman at gmail.com>
> wrote:
>
> Howdy All!
>
> This suggestion is inspired by the question on "Efficient debug logging".
>
>
> I propose a keyword to mark an expression for delayed/lazy execution, for
> the purposes of standardizing such behavior across the language.
>
> The proposed format is:
> delayed: <expr>
> i.e. log.info("info is %s", delayed: expensiveFunction())
>
> Unlike 'lambda' which returns a function (so the receiver must be
> lambda-aware), delayed execution blocks are for all purposes values. The
> first time the value (rather than location) is read, or any method on the
> delayed object is called, the expression is executed and the delayed
> expression is replaced with the result. (Thus, the delayed expression is
> only every evaluated once).
>
> Ideally:
> a = delayed: 1+2
> b = a
> print(a) #adds 1 and 2, prints 3
> # a and b are now both just 3
> print(b) #just prints 3
>
> Mechanically, this would be similar to the following:
>
> class Delayed():
>     def __init__(self, func):
>         self.__func = func
>         self.__executed = False
>         self.__value = None
>
>     def __str__(self):
>         if self.__executed:
>             return self.__value.__str__()
>         self.__value = self.__func()
>         self.__executed = True
>         return self.__value.__str__()
>
>
> def function_print(value):
>     print('function_print')
>     print(value)
>
> def function_return_stuff(value):
>     print('function_return_stuff')
>     return value
>
> function_print(function_return_stuff('no_delay'))
>
> function_print(Delayed(lambda: function_return_stuff('delayed')))
>
> delayed = Delayed(lambda: function_return_stuff('delayed_object'))
> function_print(delayed)
> function_print(delayed)
>
> Unfortunately, due to
> https://docs.python.org/3/reference/datamodel.html#special-lookup , this
> magic delayed class would need to implement many magic methods, as
> __getattribute__ is not _always_ called.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170217/3550c572/attachment-0001.html>