[Python-Dev] On Syntax Extensibility (again)

[Samuele Pedroni]

I thought python-dev archives would be a better place for this than my hd. 
Warning: terse.

I have come to the conclusion that I should agree with Guido:

- generators do not let really abstract over things like exception handling 
etc.
- some kind of very structured and Smalltalk-flavored syntax extensibility 
would be nice to have (not macros)
- people who affirm that Ruby is better because is OO from the ground up 
(???) and has blocks are annoying but have some point :):), although block 
semantics in Ruby are kind of idiosyncratic (modern smalltalks are much saner)
- while blocks are really not orthogonal with the rest of Python, the 
minimal orthogonal additions to the language, I managed to imagine, would 
absolutely not be natural, unless you find something like this natural:

def sum(upto):
   s = 0
   repeat(upto,_) where:
     _ = effect (inc): s+=inc
   return s

'where' should be obvious, 'effect' is to assigments what 'lambda' is to 
expressions, so s is not local to the effect but is 's' from the outer 
scope and that 's' will be rebound.

- * -

All keywords&syntax is tentative. Some details are sketchy. This stuff is 
likely implementable but not all of this is probably a good idea, my goal 
was more to see how far one can go once we have first-class blocks and we 
re-apply some other Python's "semantics" patterns together with them.

[!!! thought after writing all the rest: the notion of first-class blocks 
would naturally demand a scope-global __Return__ handler and one 
__Break__,__Continue__ handler for each 'while','for',  in all function 
scopes with 'on' definitions': I doubt this would be good style:

def f():
   on worker(i):
     if i==20: break
     print i
   for i in in range(40):
     worker(i)

but should probably be supported, at least the construct as such is orthogonal.
]

First, blocks as first-class citizen, 'on' definitions:

on <name>[(args)]:
   <suite>

'on' defs don't have their own locals, do not introduce a new scope but 
close over "outer" scope locals and even assigned to "locals" are free, 
[behavior of virtual 'BLOCK' definition], except for args (?).

Scopes with at least one 'on' inside define locally:

class __Return__(Exception?): pass
maybe:
class __Break__(Exception?): pass
class __Continue__(Exception?): pass

Then

on fcb(args):
   <suite>

is equivalent to:

BLOCK fcb(args): # BLOCK pseudo-def, see above for scoping rules
   <suite>
fcb.__Return__ = __Return__
# maybe
fcb.__Continue__ = __Continue__
fcb.__Break__ = __Break__


<suite> is compiled using following code transformations:
  value expr   ==> return expr # needs new keyword!, produce value

  return expr  ==> raise __Return__,expr # i.e. non-local return

  yield : not supported!!! (simple generators and first-class blocks do not 
mix)

maybe:
  break        ==> raise __Break__ # i.e. non-local break
  continue     ==> raise __Continue__ # i.e. non-local continue

- * -

'do' statement:

[x... ?=] do expr: [(args)]: # ?= stands for all possible assignment forms
   <suite>

equivalent to:

on _anonym[(args)]:
   <suite>
try:
   [x... ?=] expr(_anonym)
except __Return__,_ret: # if meaningful; _ret should be user-invisible
   return _ret.args[0]
except __Continue__: # if meaningful
   continue
except __Break: # if meaningful
   break

Example:

def list10(pfx):
  do repeat(10): (i):
    print pfx,i

<===>

def list10(pfx):
   class __Break__(Exception?): pass
   class __Return__(Exception?): pass
   class __Continue__(Exception?): pass
   BLOCK _anonym(i):
    print pfx,i
   _anonym.__Break__ = __Break__
   _anonym.__Continue__ = __Continue__
   _anonym.__Return__ = __Return__
   try:
     repeat(10)(_anonym)
   except __Return__,_ret:
     return _ret.args[0]

Some global built-in __Return__,etc. could do, but using locally defined 
ones, let people call _anonym inside a 'do' inside expr resulted callable 
getting the "expected" behavior.

Notice: a user accessible 'on' construct is not necessary for 'do' but see 
next.

- * -

Further:

~First-class explicit blocks are to complex user-defined statements what 
methods are to classes~

'do**' statement:

[x... ?=] do** expr: [(args)]:
   <suite>

semantics:

build dict _suite_dict out of <suite> (same as 'class' statement):

try:
   [x... ?=] expr(**_suite_dict)
except __Return__,_ret: # if meaningful, _ret should be user-invisible
   return _ret.args[0]
except __Continue__: # if meaningful
   continue
except __Break: # if meaningful
   break

expr resulted callable should use _suite_dict only in a read-only way (let 
the door open to some optimizations?)

examples:

do** tryfin: # contrived
   on try_:
     <suite>
   on finally_:
     <suite>


class C(object):
   v = do** property: # with suitable property def
     def get...: ...
     def set...: ...

Yes, do** is kind of an awful keyword.

- * -

Needed keywords: one for each of 'do','do**','on','value'

- * -

Possible further extensions, tentative syntax:

inside do** <suite>s allow for assigment/defs statements of the form:
   def [expr](...): ... # no here [] mean really '['']' not optional
   on [expr]...
   [expr] ?= ... # assignment forms

which would have more-or-less semantics:

   [expr] ?= ... <===> locals()[expr] ?= ...

although a special bytecode would be probably used. And then expr result 
would have to be called as in

expr(_suite_dict)

because non-string keywords are not supported and not simply ignored [at 
least now].

then:

do** except: # contrived
   on try_:
     ...
   on [ExcClass1]:
     ...
   on [(ExcClass2,ExcClass3)] (e):
     ...

If exec would work properly with subclasses of dict [that's not the case 
now] do** expr could have a __make_ns__ attribute so that a specialized 
kind of dict could be used (multi-key or remembering the order by which 
items were set, etc.)

_exv = expr
if hasattr(_exv,'__make_ns__'):
   _ns = _exv.__make_ns__()
else:
   _ns = {}
exec <suite-code> in _ns
try:
   _exv(_ns)
   ...

That's all.