[Python-Dev] block-based decorators and other block issues

[Martin Zarate]

Our discussion has taken an interesting turn, leading to this concept 
of "super-decorators" that are more versatile than normal decorators, because 
they can accept multiple objects, and because they can play with namespaces.

First of all, it is important not to get too far ahead of ourselves - 
decorators should not be full "custom blocks" like having a "threadsafe" 
decorator block and a "fork function" block wherein the decoratotor is 
changing the fundamental behaviour of a block of code.  Concepts that we see 
in Ruby (I highly, highly recommend that people interested in this subject 
read over the Ruby tutorial, it is very insightful on the concept of versatile 
block constructs) could be brought to Python, but it would have to be done 
properly through a good general-purpose mechanism.

Decorators are simply a filtering mechanism - they take an object defined with 
the decorator, and then either a) return a substitute to the parent scope or, 
possibly b) throw it out.  They are not blocking concepts.

The reason is simple - speed.  A decorator shouldn't have to perform 
fasttolocals and vice versa.  It doesn't need to be its own variable 
namespace - it can sit within the external namespace.  To do otherwise is 
suboptimal.  The decorator should not have its own scope.  Instead, it should 
provide a simple filter.

Now, it would be nice if Python had another versatile block structure usable 
for custom stuff that does involve having its own full namespace (hell, just 
a "block" keyword would do) but I'll get to that later.

First, I need a keyword.   I will use "turn", just to keep things different, 
but anything will do - has has been remarked elsewhere, a naked tuple with 
a ":" suffix would be enough - it would differentiate the conversion block 
from standard comma-delimited lists, while not looking like any other block.

At any rate, people advocating the "decorator block" concept that I suggested  
keep assuming that the descriptor functions are functions that take a whole 
namespace in and put a whole other namespace out, which is then imported into 
the parent namespace - horribly inefficient.  Use that later for custom blocks.

The call signature of a descriptor could simply be

descriptor(object, name, holder, doc, args)

where object is the object being described, name is the string with its name 
(immutable, as always) and holder is the current contents of the holder.  The 
return value of the descriptor function is assigned into the holder at the end 
of the operation - if no change is desired and the nested objects are to be 
thrown out, then simply return the holder parameter.  The name parameter 
exists for the sake of naming objects.  With the holder parameter, inspection 
of the existing object is allowed, so that the "property_get" descriptor 
(which actually alters an existing property or creates a new one if none 
exists) can work.  The reason for the longer function signature is so that 
more complicated syntax is available to the decorator function - such as a 
docstring that can be prepended to the nested docstrings.  This seems silly 
for functions, but would be appropriate when decorating non-function objects 
(such as data members).

Creating a new scope block for decorators would be too slow, but still would 
be very useful for more bizarre custom linguistic constructs.  A general 
purpose "scope" command with access to external and internal namespaces would 
be extremely useful.

Imagine the general purpose "scope" statement:

scope handler1(args), handler2(args), handler3(args):

where the contained code within the block would be run, then its local 
namespace harvested, as well as the external namespace, both passed in to

handlerN(inner, outer, args)

this would pretty much let you do whatever you want within a random scope: 
block, allowing for some bizzarro macroesque stuff that would make descriptors 
look simple by comparison.  Personally I think such a structure would be silly 
and dangerous, but its what many coders sound like they want.

A better, more general-purpose structure would be an "object" block like

object name handler(args):

wherein the contained code is run on the spot and the local namespace is 
converted into a dictionary, which is then called with the signature

handler(namespace, name, args)

where namespace is the namespace is the local namespace, name is the string of 
the name, and args is the argument tuple.  You could use this to code your own 
analogs to the class structure of Python.  Say you want a forced single-
inheritence-tree class system?  You could have one with such custom blocks, or 
any other alternate-class-structure.  Hell, with such a system the class 
itself wouldn't need to be a true keyword as it is now, but instead could 
behave as simply just one more "object" expression handler - as I understand 
it, all it does is take that namespace and the argument tuple and do some fun 
stuff with it and then plonk that info (plus the name of the class) into 
the "type" constructor.  That doesn't need to be interpreter level.

So instead of 

class foo(bar baz):

we get

object foo class(bar baz):

which could be substituted with any wierd thing you want instead of "class" - 
like a quick module object creator with none of the overhead of a class, for 
example.  Or you could do the much coveted "instance" block that many people 
have clamored for, in which you subclass an instance instead of subclassing a 
class.  Or lets say you're making a game engine and you need to make a 
separate class system for engine objects (the situation I was in, actually)and 
you want to be able to work with them as if they were classes, but also want 
it to inherit some invisible engine settings not apparent to the python 
interpreter from other engine objects, as well as instead of making some kinds 
of functions visible to the interpreter they're loaded into the event handler 
automatically (this is personal experience here - I wanted these features very 
badly).

Or you could just make a quick dict maker

def dictblock(namespace, name, args):
    return namespace;

Like sort of the reverse of a "with" block.

Take note that this system would be by far the best way to handle the property 
object, rather than mucking about with descriptors.

Works as such...

object foo propertyblock:
    def fget(self):
         return "bar"
    def fset(self):
         pass
    def fdel(self):
         del self.foo
    doc = "baz"

(alternately, docstrings could be implemented like functions, filling the 
__doc__ entry of the dictionary)

and implemented like...

propertyblock(space, name, args):
    return property(space["fget"], space["fset"], space["fdel"], space["doc"])

You could do a nice shortcut involving 
passing the namespace dict into the keywords dict while calling "property", 
but that's the more legible way.

The point is that this would be distinct from the descriptor block concept, as 
descriptors don't need to play with the namespaces beyond one object and one 
name at a time.

But none of this explains why I like using a block for descriptors - I'm just 
trying to explain to those who thought of the possibilites of a block where 
you could play with the local and surrounding namespace how yes, that stuff 
would be useful, but would have a lot more overhead than simply a descriptor 
block.

There are places a multi-line descriptor block is ugly, but there are also 
places where it is goregeous.  Public and Private are good examples that C++ 
devs would love, rather than saying public and private for each function its

turn private:
    foo = 1
    bar = 2
    baz = 3

Hell, if you really don't like the keyword, we could get even wierder and play 
with a new syntax: use the @ with a bracket and nest the block within Guido's 
syntax...

@[static,
private,
functor]:
    def foo:
         pass

Wow, I covered way, way too much ground with that post.  Seriously people - 
read the Ruby tutorial "Programming Ruby" http://www.rubycentral.com/book/ to 
see the gorgeous way that Ruby handles the whole concept of blocks.  
Unfortunately the rest of the language is thoroughly illegible, but the block 
concept is cool.