I recognize that I have so-far skirted the order-of-precedence concern. I believe I have used parens in my example everywhere there might be a question... But that's not a general description or rule.
I have a bunch of issues that I know I need to flesh out, many coming as suggestions in this thread, which I appreciate. I just wanted to provide something concrete to start the conversation.
FWIW, there is a bunch more at the link now than in my initial paste. But I want to clarify more before I copy a new version into the email thread.
I haven't used Twisted in a while, but it is certainly an important library, and I don't want to cause confusion. Any specific recommendation on language to use?
Thank you for your proposal David. At last we have a
counter-proposal to talk about. A few points:
(1) (As I pointed out in an earlier post) There is a flaw in using
the syntax of an expression PRECEDED by a SOFT keyword:
x = later -y
With your proposal, x is assigned a deferred-evaluation-object which
will be evaluated at some time later as "minus y", right?
Erm, no. This is already legal syntax for x being immediately
assigned a value of "later minus y".
If you put the soft keyword *after* the expression:
x = -y later
it may or may not read as well (subjective) but AFAICS would work.
Alternatively you could propose a hard keyword. Or a different
syntax altogether.
(2) Delayed evaluation may be useful for many purposes. But for the
specific purpose of providing late-bound function argument defaults,
having to write the extra line ("n = n" in your example) removes
much of the appeal. Two lines of boilerplate (using a sentinel)
replaced by one obscure one plus one keyword is not much if any of a
win, whereas PEP 671 would remove the boilerplate altogether apart
from one sigil. Under your proposal, I for one would probably stick
with the sentinel idiom which is explicit. I think "n=n" is
confusing to an inexperienced Python user.
You may not think this is important. My opinion is that late-bound
defaults are important. (We may have to agree to differ.) Apart
from anything else: Python fully supports early-bound defaults, why
discriminate against late-bound ones?
(3) You talk about "deferred objects" and in one place you actually
say "Evaluate the Deferred". A "deferred" is
an important object but a different concept in Twisted, I think
calling it something else would be better to avoid confusion.
Best wishes
Rob Cliffe
On 21/06/2022 21:53, David Mertz, Ph.D.
wrote:
Here is a very rough
draft of an idea I've floated often, but not with much
specification. Take this as "ideas" with little firm
commitment to details from me. PRs, or issues, or whatever,
can go to https://github.com/DavidMertz/peps/blob/master/pep-9999.rst
as well as mentioning them in this thread.
This PEP proposes introducing the soft keyword ``later`` to
express the concept
of deferred computation. When an expression is preceded by
the keyword, the
expression is not evaluated but rather creates a "thunk" or
"deferred object."
Reference to the deferred object later in program flow
causes the expression to
be executed at that point, and for both the value and type
of the object to
become the result of the evaluated expression.
Motivation
==========
"Lazy" or "deferred" evaluation is useful paradigm for
expressing relationships
among potentially expensive operations prior their actual
computation. Many
functional programming languages, such as Haskell, build
laziness into the
heart of their language. Within the Python ecosystem, the
popular scientific
library `dask-delayed <dask-delayed>`_ provides a
framework for lazy evaluation
that is very similar to that proposed in this PEP.
While the use of deferred computation is principally useful
when computations
are likely to be expensive, the simple examples shown do not
necessarily use
such expecially spendy computations. Most of these are
directly inspired by
examples used in the documentation of dask-delayed.
In dask-delayed, ``Delayed`` objects are create by
functions, and operations
create a *directed acyclic graph* rather than perform actual
computations. For
example::
>>> import dask
>>> @dask.delayed
... def later(x):
... return x
...
>>> output = []
>>> data = [23, 45, 62]
>>> for x in data:
... x = later(x)
... a = x * 3
... b = 2**x
... c = a + b
... output.append(c)
...
>>> total = sum(output)
>>> total
Delayed('add-8f4018dbf2d3c1d8e6349c3e0509d1a0')
>>> total.compute()
4611721202807865734
>>> total.visualize()
.. figure:: pep-9999-dag.png
:align: center
:width: 50%
:class: invert-in-dark-mode
Figure 1. Dask DAG created from simple operations.
Under this PEP, the soft keyword ``later`` would work in a
similar manner to
this dask.delayed code. But rather than requiring calling
``.compute()`` on a
``Delayed`` object to arrive at the result of a computation,
every reference to
a binding would perform the "compute" *unless* it was itself
a deferred
expression. So the equivalent code under this PEP would
be::
>>> output = []
>>> data = [23, 45, 62]
>>> for later x in data:
... a = later (x * 3)
... b = later (2**x)
... c = later (a + b)
... output.append(later c)
...
>>> total = later sum(output)
>>> type(total) # type() does not un-thunk
<class 'DeferredObject'>
>>> if value_needed:
... print(total) # Actual computation occurs here
4611721202807865734
In the example, we assume that the built-in function
`type()` is special in not
counting as a reference to the binding for purpose of
realizing a computation.
Alternately, some new special function like `isdeferred()`
might be used to
check for ``Deferred`` objects.
In general, however, every regular reference to a bound
object will force a
computation and re-binding on a ``Deferred``. This includes
access to simple
names, but also similarly to instance attributes, index
positions in lists or
tuples, or any other means by which an object may be
referenced.
Rejected Spellings
==================
A number of alternate spellings for creating a ``Deferred``
object are
possible. This PEP-author has little preference among
them. The words
``defer`` or ``delay``, or their past participles
``deferred`` and ``delayed``
are commonly used in discussions of lazy evaluation. All of
these would work
equally well as the suggested soft keyword ``later``. The
keyword ``lazy`` is
not completely implausible, but does not seem to read as
well.
No punctuation is immediately obvious for this purpose,
although surrounding
expressions with backticks is somewhat suggestive of quoting
in Lisp, and
perhaps slightly reminiscent of the ancient use of backtick
for shell commands
in Python 1.x. E.g.::
might_use = `math.gcd(a, math.factorial(b))`
Relationship to PEP-0671
========================
The concept of "late-bound function argument defaults" is
introduced in
:pep:`671`. Under that proposal, a special syntactic marker
would be permitted
in function signatures with default arguments to allow the
expressions
indicated as defaults to be evaluated at call time rather
than at runtime. In
current Python, we might write a toy function such as::
def func(items=[], n=None):
if n is None:
n = len(items)
items.append("Hello")
print(n)
func([1, 2, 3]) # prints: 3
Using the :pep:`671` approach this could be simplified
somewhat as::
def func(items=[], n=>len(items)):
# late-bound defaults act as if bound here
items.append("Hello")
print(n)
func([1, 2, 3]) # prints: 3
Under the current PEP, evaluation of a ``Deferred`` object
only occurs upon
reference. That is, for the current toy function, the
evaluation would not
occur until the ``print(n)`` line.::
This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
--
Keeping
medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of
the
uneducated; technology from the underdeveloped; and
putting
advocates of freedom in prisons. Intellectual property
is
to the 21st century what the slave trade was to the
16th.