Delayed evaluation of expressions [was Re: Time we switched to unicode?]

Steven D'Aprano steve+comp.lang.python at
Wed Mar 26 17:05:53 CET 2014

On Wed, 26 Mar 2014 00:30:21 -0400, Terry Reedy wrote:

> On 3/25/2014 8:12 PM, Steven D'Aprano wrote:
>> On Tue, 25 Mar 2014 19:55:39 -0400, Terry Reedy wrote:
>>> On 3/25/2014 11:18 AM, Steven D'Aprano wrote:
>>>> The thing is, we can't just create a ∑ function, because it doesn't
>>>> work the way the summation operator works. The problem is that we
>>>> would want syntactic support, so we could write something like this:
>>>>       p = 2
>>>>       ∑(n, 1, 10, n**p)
>>> Of course we can. If we do not insist on separating the dummy name
>>> from the expression that contains it. this works.
>>> def sigma(low, high, func):
>>>       sum = 0
>>>       for i in range(low, high+1):
>>>           sum += func(i)
>>>       return sum
>> There is no expression there. There is a function.
>> You cannot pass an expression to a function in Python,
> One passes an unquoted expression in code by quoting it with either
> lambda, paired quote marks (Lisp used a single '), 

Passing *strings* and *functions* is not the same as having compiler 
support for delayed evaluation. At best its a second-class work-around. 

    def if_else(true_function, condition, false_function):
        if condition:
            return true_function()
            return false_function()

    if_else(lambda: x/0, x != 0, lambda: float("inf"))

with this:

    x/0 if x != 0 else float("inf")

Aside from the difference between the function form and operator form, 
the second case is much more direct and natural than the first.

> or using it in a form
> that implicitly quotes it (that includes def statements). Unquoted
> expressions in statements ultimately get passed to an internal
> functions.

I think you are mistaken. Take the unquoted expression `x+1`. It doesn't 
get passed to anything, it gets compiled into byte code and evaluated:

py> from dis import dis
py> dis("x+1")
  1           0 LOAD_NAME                0 (x)
              3 LOAD_CONST               0 (1)
              6 BINARY_ADD
              7 RETURN_VALUE

Perhaps you are thinking of one or two special cases, such as list comps:

py> dis("[x+1 for x in spam]")
  1           0 LOAD_CONST               0 (<code object <listcomp> at
                                            0xb7af22a0, file "<dis>",
                                            line 1>)
              3 LOAD_CONST               1 ('<listcomp>')
              6 MAKE_FUNCTION            0
              9 LOAD_NAME                0 (spam)
             12 GET_ITER
             13 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             16 RETURN_VALUE

But that's an implementation detail, not a language requirement, and it 
doesn't apply to all delayed expressions:

py> dis("1/x if x else y")
  1           0 LOAD_NAME                0 (x)
              3 POP_JUMP_IF_FALSE       14
              6 LOAD_CONST               0 (1)
              9 LOAD_NAME                0 (x)
             12 BINARY_TRUE_DIVIDE
             13 RETURN_VALUE
        >>   14 LOAD_NAME                1 (y)
             17 RETURN_VALUE

Some Python implementations may use internal functions under the hood to 
delay the evaluation of an expression, but you still need support from 
the interpreter to compile the expression as an internal function rather 
than evaluating it.

>  > not in the sense I am talking about,
> well, if you eliminate all the possibilities ...

Obviously you can pass an expression to a function in the trivial sense 
that you can put an expression inside a function call. But that is not 
what I am talking about, since the expression is evaluated first, then 
the function is called:

py> dis("function(spam + eggs*cheese)")
  1           0 LOAD_NAME                0 (function)
              3 LOAD_NAME                1 (spam)
              6 LOAD_NAME                2 (eggs)
              9 LOAD_NAME                3 (cheese)
             12 BINARY_MULTIPLY
             13 BINARY_ADD
             14 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             17 RETURN_VALUE

I'm referring to delaying execution of the expression until later.

Coincidentally, the Sum function we were discussing is a notable example 
of Jensen's Device:

which is effectively what I'm referring to.

>  > because expressions are not first-class objects.
> The concept is not a class, and the Python stdlib does not have an
> expression class. But people have written classes to represent the
> concept. I used existing classes instead.
> Expressions are not (normally) mathematical objects either. (An
> exception is rewriting theory, or other theories, where strings
> representing expressions or WFFs (well-formed formulas) are the only
> objects.) Mathematicians quote expression strings by context or
> positiion. The sigma form is one of many examples.

Hmmm. I think you are misunderstanding me, possibly because I wrote 
"first-class object" when I should have called it a "first-class value". 

I'm not necessarily referring to creating something like an "arithmetic 
expression class" with methods and attributes. For example, sympy has 
things like that, so you can perform symbolic operations on the 
expression. That's not what I mean. 

What I mean is that you, the programmer, writes down an ordinary Python 
expression, using ordinary expression syntax, and the compiler treats it 
as a value in and of itself, rather than evaluating it to find out what 
value it has. In Python this will probably be some sort of object, but 
that's not the important part. The important part is that it is a *value*.

(Compare to languages where functions are not first-class values. You 
cannot pass a function to another function, or stick them in a list, or 
create them on the fly. You can only call them, evaluating them 

In Algol60, the compiler used thunks, which are a type of closure; in 
CPython, list comps use a hidden function object; the ternary if compiles 
byte code for a test and jump.

In case you haven't read the article on Jensen's Device above, in Algol60 
you can write a Sum function that behaves as a mathematician would 
expect. For example, to sum the entries of an array V for indexes 1 
through 100, a mathematician might write it something like this:

    ∑ V[i]

which we can rearrange to function-call syntax like this:

    Sum(i, 1, 100, V[i])

In Algol60, this function call would:

- pass the name "i" (not a string!) as the first argument;
- pass 1 as the second argument;
- pass 100 as the third argument;
- pass the expression "V[i]" (not a string!) as the fourth argument

and then *inside* the function Sum the expressions "i" and "V[i]" can be 
evaluated or assigned to as needed. Using Python syntax rather than Algol 

def Sum(name, lower, upper, expression):
    total = 0
    for name in range(lower, upper+1):
        total += expression
    return total

This doesn't work in Python! Python lacks call-by-name semantics, so the 
function call would:

- evaluate the expression i, and pass that value as the first argument;
- pass 1 as the second argument;
- pass 100 as the third argument;
- evaluate V[i] and pass that value as the fourth argument

and then inside the function Sum "name" refers to the local variable 
name, not the caller's variable i. Likewise "expression" refers to a 
local variable, and not the caller's expression V[i].

Python has no general way of doing this. There are a few ad-hoc special 
cases, like list comps, the ternary if operator, etc. which are hard-
coded in the compiler to delay execution of expressions. For the rest, 
you have a choice:

- give up on delayed evaluation, and redesign your API; or

- manually manage the delayed evaluation yourself, using
  some combination of functions, eval or exec.

Steven D'Aprano

More information about the Python-list mailing list