block-based decorators and other block issues

Our discussion has taken an interesting turn, leading to this concept of "super-decorators" that are more versatile than normal decorators, because they can accept multiple objects, and because they can play with namespaces. First of all, it is important not to get too far ahead of ourselves - decorators should not be full "custom blocks" like having a "threadsafe" decorator block and a "fork function" block wherein the decoratotor is changing the fundamental behaviour of a block of code. Concepts that we see in Ruby (I highly, highly recommend that people interested in this subject read over the Ruby tutorial, it is very insightful on the concept of versatile block constructs) could be brought to Python, but it would have to be done properly through a good general-purpose mechanism. Decorators are simply a filtering mechanism - they take an object defined with the decorator, and then either a) return a substitute to the parent scope or, possibly b) throw it out. They are not blocking concepts. The reason is simple - speed. A decorator shouldn't have to perform fasttolocals and vice versa. It doesn't need to be its own variable namespace - it can sit within the external namespace. To do otherwise is suboptimal. The decorator should not have its own scope. Instead, it should provide a simple filter. Now, it would be nice if Python had another versatile block structure usable for custom stuff that does involve having its own full namespace (hell, just a "block" keyword would do) but I'll get to that later. First, I need a keyword. I will use "turn", just to keep things different, but anything will do - has has been remarked elsewhere, a naked tuple with a ":" suffix would be enough - it would differentiate the conversion block from standard comma-delimited lists, while not looking like any other block. At any rate, people advocating the "decorator block" concept that I suggested keep assuming that the descriptor functions are functions that take a whole namespace in and put a whole other namespace out, which is then imported into the parent namespace - horribly inefficient. Use that later for custom blocks. The call signature of a descriptor could simply be descriptor(object, name, holder, doc, args) where object is the object being described, name is the string with its name (immutable, as always) and holder is the current contents of the holder. The return value of the descriptor function is assigned into the holder at the end of the operation - if no change is desired and the nested objects are to be thrown out, then simply return the holder parameter. The name parameter exists for the sake of naming objects. With the holder parameter, inspection of the existing object is allowed, so that the "property_get" descriptor (which actually alters an existing property or creates a new one if none exists) can work. The reason for the longer function signature is so that more complicated syntax is available to the decorator function - such as a docstring that can be prepended to the nested docstrings. This seems silly for functions, but would be appropriate when decorating non-function objects (such as data members). Creating a new scope block for decorators would be too slow, but still would be very useful for more bizarre custom linguistic constructs. A general purpose "scope" command with access to external and internal namespaces would be extremely useful. Imagine the general purpose "scope" statement: scope handler1(args), handler2(args), handler3(args): where the contained code within the block would be run, then its local namespace harvested, as well as the external namespace, both passed in to handlerN(inner, outer, args) this would pretty much let you do whatever you want within a random scope: block, allowing for some bizzarro macroesque stuff that would make descriptors look simple by comparison. Personally I think such a structure would be silly and dangerous, but its what many coders sound like they want. A better, more general-purpose structure would be an "object" block like object name handler(args): wherein the contained code is run on the spot and the local namespace is converted into a dictionary, which is then called with the signature handler(namespace, name, args) where namespace is the namespace is the local namespace, name is the string of the name, and args is the argument tuple. You could use this to code your own analogs to the class structure of Python. Say you want a forced single- inheritence-tree class system? You could have one with such custom blocks, or any other alternate-class-structure. Hell, with such a system the class itself wouldn't need to be a true keyword as it is now, but instead could behave as simply just one more "object" expression handler - as I understand it, all it does is take that namespace and the argument tuple and do some fun stuff with it and then plonk that info (plus the name of the class) into the "type" constructor. That doesn't need to be interpreter level. So instead of class foo(bar baz): we get object foo class(bar baz): which could be substituted with any wierd thing you want instead of "class" - like a quick module object creator with none of the overhead of a class, for example. Or you could do the much coveted "instance" block that many people have clamored for, in which you subclass an instance instead of subclassing a class. Or lets say you're making a game engine and you need to make a separate class system for engine objects (the situation I was in, actually)and you want to be able to work with them as if they were classes, but also want it to inherit some invisible engine settings not apparent to the python interpreter from other engine objects, as well as instead of making some kinds of functions visible to the interpreter they're loaded into the event handler automatically (this is personal experience here - I wanted these features very badly). Or you could just make a quick dict maker def dictblock(namespace, name, args): return namespace; Like sort of the reverse of a "with" block. Take note that this system would be by far the best way to handle the property object, rather than mucking about with descriptors. Works as such... object foo propertyblock: def fget(self): return "bar" def fset(self): pass def fdel(self): del self.foo doc = "baz" (alternately, docstrings could be implemented like functions, filling the __doc__ entry of the dictionary) and implemented like... propertyblock(space, name, args): return property(space["fget"], space["fset"], space["fdel"], space["doc"]) You could do a nice shortcut involving passing the namespace dict into the keywords dict while calling "property", but that's the more legible way. The point is that this would be distinct from the descriptor block concept, as descriptors don't need to play with the namespaces beyond one object and one name at a time. But none of this explains why I like using a block for descriptors - I'm just trying to explain to those who thought of the possibilites of a block where you could play with the local and surrounding namespace how yes, that stuff would be useful, but would have a lot more overhead than simply a descriptor block. There are places a multi-line descriptor block is ugly, but there are also places where it is goregeous. Public and Private are good examples that C++ devs would love, rather than saying public and private for each function its turn private: foo = 1 bar = 2 baz = 3 Hell, if you really don't like the keyword, we could get even wierder and play with a new syntax: use the @ with a bracket and nest the block within Guido's syntax... @[static, private, functor]: def foo: pass Wow, I covered way, way too much ground with that post. Seriously people - read the Ruby tutorial "Programming Ruby" http://www.rubycentral.com/book/ to see the gorgeous way that Ruby handles the whole concept of blocks. Unfortunately the rest of the language is thoroughly illegible, but the block concept is cool.

On Aug 13, 2004, at 8:20 AM, Martin Zarate wrote:
Our discussion has taken an interesting turn, leading to this concept of "super-decorators" that are more versatile than normal decorators, because they can accept multiple objects, and because they can play with namespaces.
The reason is simple - speed. A decorator shouldn't have to perform fasttolocals and vice versa. It doesn't need to be its own variable namespace - it can sit within the external namespace. To do otherwise is suboptimal. The decorator should not have its own scope. Instead, it should provide a simple filter.
Ok, I agree with you throughout this post except for this "speed" business. Decorators are largely considered a compile-time feature and will almost exclusively be executed as module-level code. Each 'decorator statement' will likely only be executed once in its entire lifetime upon module import. I don't think speed is too much of a concern at this point and worrying about it definitely seems like one heck of a premature optimization. -bob
participants (2)
-
Bob Ippolito
-
Martin Zarate