Based on Guido's opinion that caller and callee should both be marked, I have used keywords 'include' and 'chunk'. I therefore call them "Chunks" and "Includers".
Examples are based on
(1) The common case of a simple resource manager. e.g. http://mail.python.org/pipermail/python-dev/2005-April/052751.html
(2) Robert Brewer's Object Relational Mapper http://mail.python.org/pipermail/python-dev/2005-April/052924.html which uses several communicating Chunks in the same Includer, and benefits from Includer inheritance.
Note that several cooperating Chunks may use the same name (e.g. old_children) to refer to the same object, even though that object is never mentioned by the Includer.
It is possible for the same code object to be both a Chunk and an Includer. Its own included sub-Chunks also share the top Includer's namespace.
Chunks and Includers must both be written in pure python, because C frames cannot be easily manipulated. They can of course call or be called (as a unit) by extension modules.
I have assumed that Chunks should not take arguments. While arguments are useful ("Which pattern should I match against on this inclusion?"), the same functionality *can* be had by binding a known name in the Includer. When that starts to get awkward, it is a sign that you should be using separate namespaces (and callbacks, or value objects).
"self" and "cls" are just random names to a Chunk, though using them for any but the conventional meaning will be as foolhardy as it is in a method.
Chunks are limited to statement context, as they do not return a value.
Includers must provide a namespace. Therefore a single inclusion will turn the entire nearest enclosing namespace into an Includer. ? Should this be limited to nearest enclosing function or method? I can't think of a good use case for including directly from class definition or module toplevel, except registration. And even then, a metaclass might be better.
Includers may only be used in a statement context, as the Chunks must be specified in a following suite. (It would be possible to skip the suite if all Chunk names are already bound, but I'm not sure that is a good habit to encourage -- so initially forbid it.)
Chunks are defined without a (), in analogy to parentless classes. They are included (called) with a (), so that they can remain first class objects.
Example Usage =============
def withfile(filename, mode='r'): """Close the file as soon we're done.
This frees up file handles sooner. This is particularly important under Jython, or if you are using files in cyclic structures.""" openfile = open(filename, mode) try: include fileproc() # keyword 'include' prevents XXX_FAST optimization finally: openfile.close()
chunk nullreader: # callee Chunk defined for reuse for line in openfile: pass
withfile("testr.txt"): # Is this creation of a new block-starter a problem? fileproc=nullreader # Using an external Chunk object
withfile("testw.txt", "w"): chunk fileproc: # Providing an "inline" Chunk openfile.write("Line 1")
# If callers must be supported in expression context #fileproc=nullreader #withfile("tests.txt") # Resolve Chunk name from caller's default # binding, which in this case defaults back # to the current globals. # Is this just asking for trouble?
chunk nullchunk: # The extra processing is not always needed. pass begin=pre=post=end=nullchunk # Default to no extra processing
def __set__(self, unit, value): include self.begin() if self.coerce: value = self.coerce(unit, value) oldvalue = unit._properties[self.key] if oldvalue != value: include self.pre() unit._properties[self.key] = value include self.post() include self.end()
class TriggerORM(ORM): chunk pre: include super(self,TriggerORM).pre() # self was bound by __set__ old_children = self.children() # inject new variable
chunk post: include super(self,TriggerORM).post() for child in self.children(): if child not in old_children: # will see pre's binding notify_somebody("New child %s" % child)
As Robert Brewer said,
The above is quite ugly written with callbacks (due to excessive argument passing), and is currently fragile when overriding __set__ (due to duplicated code).
How to Implement ----------------
The Includer cannot know which variables a Chunk will use (or inject), so the namespace must remain a dictionary. This precludes use of the XXX_FAST bytecodes. But as Robert pointed out, avoiding another frame creation/destruction will compensate somewhat.
Two new bytecodes will be needed to handle the jump and return to a different bytecode string without setting up or tearing down a new frame. Position in the Includer bytecode will need to be kept in a stack, though it might make sense to use a frame variable instead of the execution stack.
With those two exceptions, the Includer and Chunk are both composed entirely of valid statements that can already be compiled to ordinary bytecode.