Dask also has a function delayed() that may be used as an decorator and in other ways like:
from dask import delayed
from operator import add, mul a = delayed(add)(1, 2) b = delayed(mul)(a, 3)
b Delayed('mul-1907f29b-60a4-48af-ba2a-938556555f9b')
c = b.compute()
c 9
b.dask {'add-d49ba000-dd5d-4031-8c37-6514626a3d81': (<function _operator.add>, 1, 2), 'mul-1907f29b-60a4-48af-ba2a-938556555f9b': (<function _operator.mul>, 'add-d49ba000-dd5d-4031-8c37-6514626a3d81', 3)}
You *can* do pretty much anything you'd want to using this approach... including the real job of Dask, to identify latent parallelism and execute computations on many cores or many machines in a cluster. But actual syntax to do all of this would be really elegant. I think for this to be as useful as I'd want, you'd sometimes need to be to continue delaying computation rather than doing so on every access I guess as syntax, the `delayed:` construct would work (I think having no colon would more closely model `yield` and `yield from` and `await` and `async` which this is kinda-sorta akin to). So for example, in a hypothetical Python 3.7+:
a = delayed 1 + 2
b = delayed b * 3
c = delayed 12/3
my_lazy_func(b, delayed c) # evaluates b but not yet c b 9 delayed c <delayed object at 0x123456789>
If you want to do something like Dask... or for Dask itself to be able to use it in some eventual version, you'd need to be able to keep objects from evaluating even while you passed them around. The obvious use is for finding parallelism, but other things like passing callbacks might find this useful too. Dask delayed objects stay lazy until you explicitly `.compute()` on them (which does so recursively on every lazy object that might go into the computation). This hypothetical new keyword would have object evaluate eagerly *unless* you explicitly kept them lazy. But the idea is that the programmer would still get to decide in their code. On Feb 16, 2017 9:53 PM, "Joseph Jevnik" <joejev@gmail.com> wrote:
You might be interested in https://github.com/llllllllll/lazy_python, which implements the features you describe but instead of a keyword it uses a decorator.
On Fri, Feb 17, 2017 at 12:24 AM, Joseph Hackman <josephhackman@gmail.com> wrote:
Howdy All!
This suggestion is inspired by the question on "Efficient debug logging".
I propose a keyword to mark an expression for delayed/lazy execution, for
the purposes of standardizing such behavior across the language.
The proposed format is: delayed: <expr> i.e. log.info("info is %s", delayed: expensiveFunction())
Unlike 'lambda' which returns a function (so the receiver must be lambda-aware), delayed execution blocks are for all purposes values. The first time the value (rather than location) is read, or any method on the delayed object is called, the expression is executed and the delayed expression is replaced with the result. (Thus, the delayed expression is only every evaluated once).
Ideally: a = delayed: 1+2 b = a print(a) #adds 1 and 2, prints 3 # a and b are now both just 3 print(b) #just prints 3
Mechanically, this would be similar to the following:
class Delayed(): def __init__(self, func): self.__func = func self.__executed = False self.__value = None
def __str__(self): if self.__executed: return self.__value.__str__() self.__value = self.__func() self.__executed = True return self.__value.__str__()
def function_print(value): print('function_print') print(value)
def function_return_stuff(value): print('function_return_stuff') return value
function_print(function_return_stuff('no_delay'))
function_print(Delayed(lambda: function_return_stuff('delayed')))
delayed = Delayed(lambda: function_return_stuff('delayed_object')) function_print(delayed) function_print(delayed)
Unfortunately, due to https://docs.python.org/3/r eference/datamodel.html#special-lookup , this magic delayed class would need to implement many magic methods, as __getattribute__ is not _always_ called.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/