<div dir="ltr">David, can you elaborate on your example?<div><br></div><div>if we replaced line four with</div><div><br></div><div>    >>> x = my_lazy_func(b, delayed c)</div><div><br></div><div>what would the value of `x` be, and how would this differ from either</div><div><br></div><div>    >>> x = delayed my_lazy_func(b, delayed c)</div><div><br></div><div>or</div><div><br></div><div>    >>> x = delayed my_lazy_func(b, c)<br></div><div><br></div><div>To put it another way, why does my_lazy_func being called not evaluate c, or are you implying that it too is some kind of special delayed function that needs to be explicitly computed, like in dask?</div><div><br></div><div>--Josh</div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Feb 17, 2017 at 1:23 AM David Mertz <<a href="mailto:mertz@gnosis.cx">mertz@gnosis.cx</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div dir="auto" class="gmail_msg">Dask also has a function delayed() that may be used as an decorator and in other ways like:<div dir="auto" class="gmail_msg"><br class="gmail_msg"></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> from dask import delayed</font></div></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> from operator import add, mul</font></div><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> a = delayed(add)(1, 2)</font></div><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> b = delayed(mul)(a, 3)</font></div></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> b</font></div><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">Delayed('mul-1907f29b-60a4-48af-ba2a-938556555f9b')<br class="gmail_msg"></font></div></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> c = b.compute()</font></div></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> c</font></div><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">9</font></div></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg">>>> b.dask</font></div><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg"><div class="gmail_msg">{'add-d49ba000-dd5d-4031-8c37-6514626a3d81': (<function _operator.add>, 1, 2),</div><div class="gmail_msg"> 'mul-1907f29b-60a4-48af-ba2a-938556555f9b': (<function _operator.mul>,</div><div class="gmail_msg">  'add-d49ba000-dd5d-4031-8c37-6514626a3d81',</div><div class="gmail_msg">  3)}</div></font></div></div></blockquote><div style="font-family:monospace,monospace" class="gmail_msg"><font face="monospace, monospace" class="gmail_msg"><br class="gmail_msg"></font></div><font face="arial, helvetica, sans-serif" class="gmail_msg">You <b class="gmail_msg">can</b> do pretty much anything you'd want to using this approach... including the real job of Dask, to identify latent parallelism and execute computations on many cores or many machines in a cluster.</font><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg"><br class="gmail_msg"></font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">But actual syntax to do all of this would be really elegant.  I think for this to be as useful as I'd want, you'd sometimes need to be to continue delaying computation rather than doing so on every access  I guess as syntax, the `delayed:` construct would work (I think having no colon would more closely model `yield` and `yield from` and `await` and `async` which this is kinda-sorta akin to).</font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg"><br class="gmail_msg"></font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">So for example, in a hypothetical Python 3.7+:</font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg"><br class="gmail_msg"></font></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class="gmail_msg"><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">>>> a = delayed 1 + 2</font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class="gmail_msg"><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">>>> b = delayed b * 3</font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class="gmail_msg"><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">>>> c = delayed 12/3</font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px" class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">>>> my_lazy_func(b, delayed c) # evaluates b but not yet c<br class="gmail_msg">>>> b<br class="gmail_msg">9<br class="gmail_msg">>>> delayed c<br class="gmail_msg"><delayed object at 0x123456789></font></blockquote><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg"><br class="gmail_msg"></font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">If you want to do something like Dask... or for Dask itself to be able to use it in some eventual version, you'd need to be able to keep objects from evaluating even while you passed them around.  The obvious use is for finding parallelism, but other things like passing callbacks might find this useful too.</font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg"><br class="gmail_msg"></font></div><div class="gmail_msg"><font face="arial, helvetica, sans-serif" class="gmail_msg">Dask delayed objects stay lazy until you explicitly `.compute()` on them (which does so recursively on every lazy object that might go into the computation).  This hypothetical new keyword would have object evaluate eagerly <b class="gmail_msg">unless</b> you explicitly kept them lazy.  But the idea is that the programmer would still get to decide in their code.<br class="gmail_msg"></font></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_msg"><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="gmail_msg"><div dir="auto" class="gmail_msg"><div class="gmail_msg"><font face="monospace, monospace" class="gmail_msg"><br class="gmail_msg"></font></div></div></blockquote><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg">On Feb 16, 2017 9:53 PM, "Joseph Jevnik" <<a href="mailto:joejev@gmail.com" class="gmail_msg" target="_blank">joejev@gmail.com</a>> wrote:<br type="attribution" class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg">You might be interested in <a href="https://github.com/llllllllll/lazy_python" class="gmail_msg" target="_blank">https://github.com/llllllllll/lazy_python</a>, which implements the features you describe but instead of a keyword it uses a decorator.<br class="gmail_msg"></div><div class="gmail_extra gmail_msg"><br class="gmail_msg"><div class="gmail_quote gmail_msg">On Fri, Feb 17, 2017 at 12:24 AM, Joseph Hackman <span dir="ltr" class="gmail_msg"><<a href="mailto:josephhackman@gmail.com" class="gmail_msg" target="_blank">josephhackman@gmail.com</a>></span> wrote:<br class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg">Howdy All!<div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">This suggestion is inspired by the question on "Efficient debug logging".</div><div class="gmail_msg"> <br class="gmail_msg"></div></div></blockquote></div></div></blockquote><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_msg"></div><div class="gmail_msg">I propose a keyword to mark an expression for delayed/lazy execution, for the purposes of standardizing such behavior across the language.</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">The proposed format is:<br class="gmail_msg">delayed: <expr></div><div class="gmail_msg">i.e. <a href="http://log.info" class="gmail_msg" target="_blank">log.info</a>("info is %s", delayed: expensiveFunction())</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Unlike 'lambda' which returns a function (so the receiver must be lambda-aware), delayed execution blocks are for all purposes values. The first time the value (rather than location) is read, or any method on the delayed object is called, the expression is executed and the delayed expression is replaced with the result. (Thus, the delayed expression is only every evaluated once).</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Ideally:</div><div class="gmail_msg">a = delayed: 1+2</div><div class="gmail_msg">b = a</div><div class="gmail_msg">print(a) #adds 1 and 2, prints 3</div><div class="gmail_msg"># a and b are now both just 3</div><div class="gmail_msg">print(b) #just prints 3</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Mechanically, this would be similar to the following:</div><div class="gmail_msg"><pre style="background-color:rgb(43,43,43);color:rgb(169,183,198);font-family:"courier new";font-size:9pt" class="gmail_msg"><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">class </span><span style="font-weight:bold" class="gmail_msg">Delayed</span>():<br class="gmail_msg">    <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">def </span><span style="color:rgb(178,0,178)" class="gmail_msg">__init__</span>(<span style="color:rgb(148,85,141)" class="gmail_msg">self</span><span style="color:rgb(204,120,50)" class="gmail_msg">, </span>func):<br class="gmail_msg">        <span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__func = func<br class="gmail_msg">        <span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__executed = <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">False<br class="gmail_msg"></span><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">        </span><span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__value = <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">None<br class="gmail_msg"></span><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg"><br class="gmail_msg"></span><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">    def </span><span style="color:rgb(178,0,178)" class="gmail_msg">__str__</span>(<span style="color:rgb(148,85,141)" class="gmail_msg">self</span>):<br class="gmail_msg">        <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">if </span><span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__executed:<br class="gmail_msg">            <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">return </span><span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__value.<span style="color:rgb(178,0,178)" class="gmail_msg">__str__</span>()<br class="gmail_msg">        <span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__value = <span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__func()<br class="gmail_msg">        <span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__executed = <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">True<br class="gmail_msg"></span><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">        return </span><span style="color:rgb(148,85,141)" class="gmail_msg">self</span>.__value.<span style="color:rgb(178,0,178)" class="gmail_msg">__str__</span>()<br class="gmail_msg"><br class="gmail_msg"><br class="gmail_msg"><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">def </span><span style="font-weight:bold" class="gmail_msg">function_print</span>(value):<br class="gmail_msg">    <span style="color:rgb(136,136,198)" class="gmail_msg">print</span>(<span style="color:rgb(0,128,128)" class="gmail_msg">'function_print'</span>)<br class="gmail_msg">    <span style="color:rgb(136,136,198)" class="gmail_msg">print</span>(value)<br class="gmail_msg"><br class="gmail_msg"><span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">def </span><span style="font-weight:bold" class="gmail_msg">function_return_stuff</span>(value):<br class="gmail_msg">    <span style="color:rgb(136,136,198)" class="gmail_msg">print</span>(<span style="color:rgb(0,128,128)" class="gmail_msg">'function_return_stuff'</span>)<br class="gmail_msg">    <span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">return </span>value<br class="gmail_msg"><br class="gmail_msg">function_print(function_return_stuff(<span style="color:rgb(0,128,128)" class="gmail_msg">'no_delay'</span>))<br class="gmail_msg"><br class="gmail_msg">function_print(Delayed(<span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">lambda</span>: function_return_stuff(<span style="color:rgb(0,128,128)" class="gmail_msg">'delayed'</span>)))<br class="gmail_msg"><br class="gmail_msg">delayed = Delayed(<span style="color:rgb(204,120,50);font-weight:bold" class="gmail_msg">lambda</span>: function_return_stuff(<span style="color:rgb(0,128,128)" class="gmail_msg">'delayed_object'</span>))<br class="gmail_msg">function_print(delayed)<br class="gmail_msg">function_print(delayed)</pre></div><div class="gmail_msg">Unfortunately, due to <a href="https://docs.python.org/3/reference/datamodel.html#special-lookup" class="gmail_msg" target="_blank">https://docs.python.org/3/reference/datamodel.html#special-lookup</a> , this magic delayed class would need to implement many magic methods, as __getattribute__ is not _always_ called.</div></div>
<br class="gmail_msg">_______________________________________________<br class="gmail_msg">
Python-ideas mailing list<br class="gmail_msg">
<a href="mailto:Python-ideas@python.org" class="gmail_msg" target="_blank">Python-ideas@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br class="gmail_msg">
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" class="gmail_msg" target="_blank">http://python.org/psf/codeofconduct/</a><br class="gmail_msg"></blockquote></div><br class="gmail_msg"></div>
<br class="gmail_msg">_______________________________________________<br class="gmail_msg">
Python-ideas mailing list<br class="gmail_msg">
<a href="mailto:Python-ideas@python.org" class="gmail_msg" target="_blank">Python-ideas@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br class="gmail_msg">
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" class="gmail_msg" target="_blank">http://python.org/psf/codeofconduct/</a><br class="gmail_msg"></blockquote></div></div>
</div></div>
_______________________________________________<br class="gmail_msg">
Python-ideas mailing list<br class="gmail_msg">
<a href="mailto:Python-ideas@python.org" class="gmail_msg" target="_blank">Python-ideas@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br class="gmail_msg">
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" class="gmail_msg" target="_blank">http://python.org/psf/codeofconduct/</a></blockquote></div>