[Python-Dev] Extended Function syntax

[Guido van Rossum]

> > How about
> > 
> >   foo = property:
> >       ...
> 
> That looks quite nice to me, if it can be parsed...

That's the first bit of positive feedback I got on this proposal all
day. :-)

Fortunately the second bit came soon (although Raymond left in an
attribution to Greg Ewing that seemed to imply it was his idea :-).

And as another opening remark, I didn't mean this to replace the
[prop, prop, ...] syntax on function/method defs -- that seems a
useful addition in its own right.  I'm not so sure that it also
applies to classes -- there we already have metaclasses.

The form above can be parsed easily, if we say that "colon followed by
suite" is allowed only at the end of a simple assignment or expression
statement.  Then,

  v = e:
     S

would be equivalent to

  v = e(T)

where T is a thunk created from S.  (More about thunks below.)
Similarly,

  e:
    S

woul be equivalent to

  e(T)

Note that 'property' and 'synchronized' used as examples are not new
keywords!  They are just built-in objects that have the desired
semantics and know about thunks.  Syntactically, this would be valid
too:

  x = 42:
    print "Hello world"

but when executing it, you'd get a TypeError when calling the number
42 with a thunk as argument.

Other small stuff:

  v, w = e:
    S

is of course equivalent to

  v, w = e(T)

and similarly

  v = w = e:
    S

is equivalent to

  v = w = e(T)


Another exercise to see if this construct can be used flexibly might
be to see if we can use it for creating an interface syntax inspired
by Zope interfaces.  Let's see...

  I1 = interface(I2, I3):
     def m1(a, b, c):
       "docs for method m1"
     def m2(a, b, c):
       "docs for method m2"

This doesn't look very attractive, certainly not much better than
Zope's abuse of the class keyword.


BTW, I'd be interested in entertaining different syntactic proposals
that would make it less likely that a typo would accidentally turn
something into a thunk call.  It worries me that leaving out 'def' in
a simple function definition could turn it into code that gives a
TypeError when executed rather than a SyntaxError.  (Forgetting 'def'
is a mistake I frequently make.)


Samuele rightly questioned the semantics of the thunk with regard to
namespaces though.  In the example of a locking section:

  synchronized(aLock):
    BLOCK

you'd want the scope of BLOCK to be the same as that of the
surrounding code, just as with an if or try statement.

Note that synchronized() must be a function of one argument that
*returns a function*.  Something like this:

  def synchronized(lock):
    def f(thunk):
      lock.acquire()
      try:
        thunk()
      finally:
        lock.release()
    return f

On the other hand, when using the same syntax to define a property:

  foo = property:
    "docs"
    def get(self):
      return self.__foo
    def set(self, value):
      self.__foo = value

we would like the thunk to be executed in a new scope.

In this case, the property constructor must (1) detect that it is
being called with a thunk argument, and (2) in that case, *call* the
thunk (so the def statements inside the thunk are executed) and
somehow extract them from the thunk's namespace.

I don't mind if a function that takes a thunk has to work a little
harder (although there should be a simple case that's roughly
equivalent to Ruby blocks).  I also don't mind making a thunk an
object with various pieces of metadata and perhaps different methods
so that synchronized and property can pick it apart and call it in
different ways.

If we didn't have local variable optimizations or nested scopes, and
all namespaces were simply dictionaries, then it might be sufficient
if a thunk were a code object.  synchronized() could use this to
execute its thunk in its caller's scope:

    exec thunk in sys._getframe(1).f_locals

and property could execute it in a freshly created namespace:

    d = {}
    exec thunk in d

but alas, life isn't that simple any more.  A thunk may:

  - use variables defined in the containing scope (but which may have
    no value when the thunk is executed)

  - set variables that are defined in the containing scope

  - set variables that don't appear in the containing scope

  - and more (e.g. think about globals, outer scopes, and so on)

Worse, you may have multiple thunks, and they may set variables with
the same name (e.g. all properties will tend to define get and set).

Also, if a global is used in the scope containing a thunk and set in
the thunk, does that make it a local variable shared between the
containing scope and the thunk?  Or does it make it a local variable
in the thunk that shadows the global only in the thunk's scope?

The compiler could generate code for thunks that leave the decision on
all these issues up to the code (e.g. the implementation of property
or synchronized) that calls the thunk.  Maybe the cells used to
implement nested scopes can help.  But it would be pretty hairy!

--Guido van Rossum (home page: http://www.python.org/~guido/)