After some interesting conversations at PyCodeConf, I'm killing PEP 3150 (Statement Local Namespaces). It's too big, too unwieldy, too confusing and too hard to implement to ever be a good idea. PEP 403 is a far simpler idea, that looks to decorators (and Ruby blocks) for inspiration. It's still a far from perfect idea, but it has a lot more going for it than PEP 3150 ever did. The basic question to ask yourself is this: What if we had a syntax that allowed us to replace the final "name = obj" step that is implicit in function and class definitions with an alternative statement, and had a symbol that allowed us to refer to the function or class in that statement without having to repeat the name? The new PEP is included below and is also available online: http://www.python.org/dev/peps/pep-0403/ I would *love* for people to dig through their callback based code (and any other examples of "single use" functions and classes) to see if this idea would help them. Cheers, Nick. PEP: 403 Title: Statement local classes and functions Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan Status: Deferred Type: Standards Track Content-Type: text/x-rst Created: 2011-10-13 Python-Version: 3.x Post-History: 2011-10-13 Resolution: TBD Abstract ======== This PEP proposes the addition of ':' as a new class and function prefix syntax (analogous to decorators) that permits a statement local function or class definition to be appended to any Python statement that currently does not have an associated suite. In addition, the new syntax would allow the '@' symbol to be used to refer to the statement local function or class without needing to repeat the name. When the ':' prefix syntax is used, the associated statement would be executed *instead of* the normal local name binding currently implicit in function and class definitions. This PEP is based heavily on many of the ideas in PEP 3150 (Statement Local Namespaces) so some elements of the rationale will be familiar to readers of that PEP. That PEP has now been withdrawn in favour of this one. PEP Deferral ============ Like PEP 3150, this PEP currently exists in a deferred state. Unlike PEP 3150, this isn't because I suspect it might be a terrible idea or see nasty problems lurking in the implementation (aside from one potential parsing issue). Instead, it's because I think fleshing out the concept, exploring syntax variants, creating a reference implementation and generally championing the idea is going to require more time than I can give it in the 3.3 time frame. So, it's deferred. If anyone wants to step forward to drive the PEP for 3.3, let me know and I can add you as co-author and move it to Draft status. Basic Examples ============== Before diving into the long history of this problem and the detailed rationale for this specific proposed solution, here are a few simple examples of the kind of code it is designed to simplify. As a trivial example, weakref callbacks could be defined as follows:: :x = weakref.ref(obj, @) def report_destruction(obj): print("{} is being destroyed".format(obj)) This contrasts with the current repetitive "out of order" syntax for this operation:: def report_destruction(obj): print("{} is being destroyed".format(obj)) x = weakref.ref(obj, report_destruction) That structure is OK when you're using the callable multiple times, but it's irritating to be forced into it for one-off operations. Similarly, singleton classes could now be defined as:: :instance = @() class OnlyOneInstance: pass Rather than:: class OnlyOneInstance: pass instance = OnlyOneInstance() And the infamous accumulator example could become:: def counter(): x = 0 :return @ def increment(): nonlocal x x += 1 return x Proposal ======== This PEP proposes the addition of an optional block prefix clause to the syntax for function and class definitions. This block prefix would be introduced by a leading ``:`` and would be allowed to contain any simple statement (including those that don't make any sense in that context - while such code would be legal, there wouldn't be any point in writing it). The decorator symbol ``@`` would be repurposed inside the block prefix to refer to the function or class being defined. When a block prefix is provided, it *replaces* the standard local name binding otherwise implicit in a class or function definition. Background ========== The question of "multi-line lambdas" has been a vexing one for many Python users for a very long time, and it took an exploration of Ruby's block functionality for me to finally understand why this bugs people so much: Python's demand that the function be named and introduced before the operation that needs it breaks the developer's flow of thought. They get to a point where they go "I need a one-shot operation that does <X>", and instead of being able to just *say* that, they instead have to back up, name a function to do <X>, then call that function from the operation they actually wanted to do in the first place. Lambda expressions can help sometimes, but they're no substitute for being able to use a full suite. Ruby's block syntax also heavily inspired the style of the solution in this PEP, by making it clear that even when limited to *one* anonymous function per statement, anonymous functions could still be incredibly useful. Consider how many constructs Python has where one expression is responsible for the bulk of the heavy lifting: * comprehensions, generator expressions, map(), filter() * key arguments to sorted(), min(), max() * partial function application * provision of callbacks (e.g. for weak references) * array broadcast operations in NumPy However, adopting Ruby's block syntax directly won't work for Python, since the effectiveness of Ruby's blocks relies heavily on various conventions in the way functions are *defined* (specifically, Ruby's ``yield`` syntax to call blocks directly and the ``&arg`` mechanism to accept a block as a functions final argument. Since Python has relied on named functions for so long, the signatures of APIs that accept callbacks are far more diverse, thus requiring a solution that allows anonymous functions to be slotted in at the appropriate location. Relation to PEP 3150 ==================== PEP 3150 (Statement Local Namespaces) described its primary motivation as being to elevate ordinary assignment statements to be on par with ``class`` and ``def`` statements where the name of the item to be defined is presented to the reader in advance of the details of how the value of that item is calculated. This PEP achieves the same goal in a different way, by allowing the simple name binding of a standard function definition to be replaced with something else (like assigning the result of the function to a value). This PEP also achieves most of the other effects described in PEP 3150 without introducing a new brainbending kind of scope. All of the complex scoping rules in PEP 3150 are replaced in this PEP with the simple ``@`` reference to the statement local function or class definition. Symbol Choice ============== The ':' symbol was chosen due to its existing presence in Python and its association with 'functions in expressions' via ``lambda`` expressions. The past Simple Implicit Lambda proposal (PEP ???) was also a factor. The proposal definitely requires *some* kind of prefix to avoid parsing ambiguity and backwards compatibility problems and ':' at least has the virtue of brevity. There's no obious alternative symbol that offers a clear improvement. Introducing a new keyword is another possibility, but I haven't come up with one that really has anything to offer over the leading colon. Syntax Change ============= Current:: atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False') Changed:: atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False' | '@') New:: blockprefix: ':' simple_stmt block: blockprefix (decorated | classdef | funcdef) The above is the general idea, but I suspect that change to the 'atom' definition would cause an ambiguity problem in the parser when it comes to detecting decorator lines. So the actual implementation would be more complex than that. Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar Possible Implementation Strategy ================================ This proposal has one titanic advantage over PEP 3150: implementation should be relatively straightforward. Both the class and function definition statements emit code to perform the local name binding for their defined name. Implementing this PEP should just require intercepting that code generation and replacing it with the code in the block prefix. The one potentially tricky part is working out how to allow the dual use of '@' without rewriting half the grammar definition. More Examples ============= Calculating attributes without polluting the local namespace (from os.py):: # Current Python (manual namespace cleanup) def _createenviron(): ... # 27 line function environ = _createenviron() del _createenviron # Becomes: :environ = @() def _createenviron(): ... # 27 line function Loop early binding:: # Current Python (default argument hack) funcs = [(lambda x, i=i: x + i) for i in range(10)] # Becomes: :funcs = [@(i) for i in range(10)] def make_incrementor(i): return lambda x: x + i # Or even: :funcs = [@(i) for i in range(10)] def make_incrementor(i): :return @ def incrementor(x): return x + i Reference Implementation ======================== None as yet. TO DO ===== Sort out links and references to everything :) Acknowledgements ================ Huge thanks to Gary Bernhardt for being blunt in pointing out that I had no idea what I was talking about in criticising Ruby's blocks, kicking off a rather enlightening process of investigation. References ========== TBD Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia