[Python-Dev] anonymous blocks as scope-collapse: detailed proposal

Jim Jewett jimjjewett at gmail.com
Thu Apr 28 23:53:36 CEST 2005

Based on Guido's opinion that caller and callee should both be
marked, I have used keywords 'include' and 'chunk'.  I therefore
call them "Chunks" and "Includers".

Examples are based on

(1)  The common case of a simple resource manager.  e.g.

(2)  Robert Brewer's Object Relational Mapper
which uses several communicating Chunks in the same Includer, and
benefits from Includer inheritance.

Note that several cooperating Chunks may use the same name
(e.g. old_children) to refer to the same object, even though
that object is never mentioned by the Includer.

It is possible for the same code object to be both a Chunk and
an Includer.  Its own included sub-Chunks also share the top
Includer's namespace.

Chunks and Includers must both be written in pure python,
because C frames cannot be easily manipulated.  They can
of course call or be called (as a unit) by extension modules.

I have assumed that Chunks should not take arguments.  While
arguments are useful ("Which pattern should I match against
on this inclusion?"), the same functionality *can* be had by
binding a known name in the Includer.  When that starts to get
awkward, it is a sign that you should be using separate
namespaces (and callbacks, or value objects).

"self" and "cls" are just random names to a Chunk, though
using them for any but the conventional meaning will be as
foolhardy as it is in a method.

Chunks are limited to statement context, as they do not return
a value.  

Includers must provide a namespace.  Therefore a single inclusion
will turn the entire nearest enclosing namespace into an Includer.
    ?  Should this be limited to nearest enclosing function or
       method?  I can't think of a good use case for including
       directly from class definition or module toplevel, except
       registration.  And even then, a metaclass might be better.

Includers may only be used in a statement context, as the Chunks
must be specified in a following suite.  (It would be possible to
skip the suite if all Chunk names are already bound, but I'm not
sure that is a good habit to encourage -- so initially forbid it.)

Chunks are defined without a (), in analogy to parentless classes.
They are included (called) with a (), so that they can remain first
class objects.

Example Usage

def withfile(filename, mode='r'):
    """Close the file as soon we're done.

    This frees up file handles sooner.  This is particularly important
    under Jython, or if you are using files in cyclic structures."""
    openfile = open(filename, mode)
        include fileproc()  # keyword 'include' prevents XXX_FAST optimization

chunk nullreader:           # callee Chunk defined for reuse
    for line in openfile:

withfile("testr.txt"):      # Is this creation of a new block-starter a problem?
    fileproc=nullreader     # Using an external Chunk object

withfile("testw.txt", "w"):
    chunk fileproc:         # Providing an "inline" Chunk
        openfile.write("Line 1")

#   If callers must be supported in expression context
#withfile("tests.txt")      # Resolve Chunk name from caller's default
                            # binding, which in this case defaults back
                            # to the current globals.
                            # Is this just asking for trouble?

class ORM(object):

    chunk nullchunk:                # The extra processing is not always needed.
    begin=pre=post=end=nullchunk    # Default to no extra processing

    def __set__(self, unit, value):
        include self.begin()
        if self.coerce:
            value = self.coerce(unit, value)
        oldvalue = unit._properties[self.key]
        if oldvalue != value:
            include self.pre()
            unit._properties[self.key] = value
            include self.post()
        include self.end()

class TriggerORM(ORM):
    chunk pre:
        include super(self,TriggerORM).pre()    # self was bound by __set__
        old_children = self.children()          # inject new variable
    chunk post:
        include super(self,TriggerORM).post()
        for child in self.children():
            if child not in old_children:       # will see pre's binding
                notify_somebody("New child %s" % child)

As Robert Brewer said, 

> The above is quite ugly written with callbacks (due to
> excessive argument passing), and is currently fragile
> when overriding __set__ (due to duplicated code).

How to Implement

The Includer cannot know which variables a Chunk will use (or
inject), so the namespace must remain a dictionary.  This precludes
use of the XXX_FAST bytecodes.  But as Robert pointed out, avoiding
another frame creation/destruction will compensate somewhat.

Two new bytecodes will be needed to handle the jump and return to
a different bytecode string without setting up or tearing down a
new frame.  Position in the Includer bytecode will need to be kept
in a stack, though it might make sense to use a frame variable
instead of the execution stack.

With those two exceptions, the Includer and Chunk are both
composed entirely of valid statements that can already be
compiled to ordinary bytecode.


More information about the Python-Dev mailing list