Mailman 3 October 2011 - Python-ideas

The way decorators are parsng
by Tarek Ziadé 19 Oct '11

19 Oct '11

Hello Today I've tried to write a one-liner for a decorator, The decorator is a method in a class. I wanted to do something like this: @Class().decorator() def function(): ... That threw a syntax error to my surprise. But the semantic is correct, since I am currently writing: obj = Class() @obj.decorator() def function(): ... And I can also write dec = Class().decorator @dec() def function(): ... Is there something obvious I am missing, or is there a weird thing in the way decoratirs are parsed ? Demo: >>> class Some(object): ... def stuff(self, func): ... return func ... >>> s = Some() >>> @s.stuff ... def ok(): ... print 'ok' ... >>> ok() ok >>> s = Some().stuff >>> @s ... def ok(): ... print 'ok' ... >>> ok() ok >>> @Some().stuff File "<stdin>", line 1 @Some().stuff ^ SyntaxError: invalid syntax Cheers Tarek -- Tarek Ziadé | http://ziade.org

5 7

Statement local functions and classes (aka PEP 3150 is dead, say 'Hi!' to PEP 403)
by Nick Coghlan 19 Oct '11

19 Oct '11

After some interesting conversations at PyCodeConf, I'm killing PEP 3150 (Statement Local Namespaces). It's too big, too unwieldy, too confusing and too hard to implement to ever be a good idea. PEP 403 is a far simpler idea, that looks to decorators (and Ruby blocks) for inspiration. It's still a far from perfect idea, but it has a lot more going for it than PEP 3150 ever did. The basic question to ask yourself is this: What if we had a syntax that allowed us to replace the final "name = obj" step that is implicit in function and class definitions with an alternative statement, and had a symbol that allowed us to refer to the function or class in that statement without having to repeat the name? The new PEP is included below and is also available online: http://www.python.org/dev/peps/pep-0403/ I would *love* for people to dig through their callback based code (and any other examples of "single use" functions and classes) to see if this idea would help them. Cheers, Nick. PEP: 403 Title: Statement local classes and functions Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan(a)gmail.com> Status: Deferred Type: Standards Track Content-Type: text/x-rst Created: 2011-10-13 Python-Version: 3.x Post-History: 2011-10-13 Resolution: TBD Abstract ======== This PEP proposes the addition of ':' as a new class and function prefix syntax (analogous to decorators) that permits a statement local function or class definition to be appended to any Python statement that currently does not have an associated suite. In addition, the new syntax would allow the '@' symbol to be used to refer to the statement local function or class without needing to repeat the name. When the ':' prefix syntax is used, the associated statement would be executed *instead of* the normal local name binding currently implicit in function and class definitions. This PEP is based heavily on many of the ideas in PEP 3150 (Statement Local Namespaces) so some elements of the rationale will be familiar to readers of that PEP. That PEP has now been withdrawn in favour of this one. PEP Deferral ============ Like PEP 3150, this PEP currently exists in a deferred state. Unlike PEP 3150, this isn't because I suspect it might be a terrible idea or see nasty problems lurking in the implementation (aside from one potential parsing issue). Instead, it's because I think fleshing out the concept, exploring syntax variants, creating a reference implementation and generally championing the idea is going to require more time than I can give it in the 3.3 time frame. So, it's deferred. If anyone wants to step forward to drive the PEP for 3.3, let me know and I can add you as co-author and move it to Draft status. Basic Examples ============== Before diving into the long history of this problem and the detailed rationale for this specific proposed solution, here are a few simple examples of the kind of code it is designed to simplify. As a trivial example, weakref callbacks could be defined as follows:: :x = weakref.ref(obj, @) def report_destruction(obj): print("{} is being destroyed".format(obj)) This contrasts with the current repetitive "out of order" syntax for this operation:: def report_destruction(obj): print("{} is being destroyed".format(obj)) x = weakref.ref(obj, report_destruction) That structure is OK when you're using the callable multiple times, but it's irritating to be forced into it for one-off operations. Similarly, singleton classes could now be defined as:: :instance = @() class OnlyOneInstance: pass Rather than:: class OnlyOneInstance: pass instance = OnlyOneInstance() And the infamous accumulator example could become:: def counter(): x = 0 :return @ def increment(): nonlocal x x += 1 return x Proposal ======== This PEP proposes the addition of an optional block prefix clause to the syntax for function and class definitions. This block prefix would be introduced by a leading ``:`` and would be allowed to contain any simple statement (including those that don't make any sense in that context - while such code would be legal, there wouldn't be any point in writing it). The decorator symbol ``@`` would be repurposed inside the block prefix to refer to the function or class being defined. When a block prefix is provided, it *replaces* the standard local name binding otherwise implicit in a class or function definition. Background ========== The question of "multi-line lambdas" has been a vexing one for many Python users for a very long time, and it took an exploration of Ruby's block functionality for me to finally understand why this bugs people so much: Python's demand that the function be named and introduced before the operation that needs it breaks the developer's flow of thought. They get to a point where they go "I need a one-shot operation that does <X>", and instead of being able to just *say* that, they instead have to back up, name a function to do <X>, then call that function from the operation they actually wanted to do in the first place. Lambda expressions can help sometimes, but they're no substitute for being able to use a full suite. Ruby's block syntax also heavily inspired the style of the solution in this PEP, by making it clear that even when limited to *one* anonymous function per statement, anonymous functions could still be incredibly useful. Consider how many constructs Python has where one expression is responsible for the bulk of the heavy lifting: * comprehensions, generator expressions, map(), filter() * key arguments to sorted(), min(), max() * partial function application * provision of callbacks (e.g. for weak references) * array broadcast operations in NumPy However, adopting Ruby's block syntax directly won't work for Python, since the effectiveness of Ruby's blocks relies heavily on various conventions in the way functions are *defined* (specifically, Ruby's ``yield`` syntax to call blocks directly and the ``&arg`` mechanism to accept a block as a functions final argument. Since Python has relied on named functions for so long, the signatures of APIs that accept callbacks are far more diverse, thus requiring a solution that allows anonymous functions to be slotted in at the appropriate location. Relation to PEP 3150 ==================== PEP 3150 (Statement Local Namespaces) described its primary motivation as being to elevate ordinary assignment statements to be on par with ``class`` and ``def`` statements where the name of the item to be defined is presented to the reader in advance of the details of how the value of that item is calculated. This PEP achieves the same goal in a different way, by allowing the simple name binding of a standard function definition to be replaced with something else (like assigning the result of the function to a value). This PEP also achieves most of the other effects described in PEP 3150 without introducing a new brainbending kind of scope. All of the complex scoping rules in PEP 3150 are replaced in this PEP with the simple ``@`` reference to the statement local function or class definition. Symbol Choice ============== The ':' symbol was chosen due to its existing presence in Python and its association with 'functions in expressions' via ``lambda`` expressions. The past Simple Implicit Lambda proposal (PEP ???) was also a factor. The proposal definitely requires *some* kind of prefix to avoid parsing ambiguity and backwards compatibility problems and ':' at least has the virtue of brevity. There's no obious alternative symbol that offers a clear improvement. Introducing a new keyword is another possibility, but I haven't come up with one that really has anything to offer over the leading colon. Syntax Change ============= Current:: atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False') Changed:: atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False' | '@') New:: blockprefix: ':' simple_stmt block: blockprefix (decorated | classdef | funcdef) The above is the general idea, but I suspect that change to the 'atom' definition would cause an ambiguity problem in the parser when it comes to detecting decorator lines. So the actual implementation would be more complex than that. Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar Possible Implementation Strategy ================================ This proposal has one titanic advantage over PEP 3150: implementation should be relatively straightforward. Both the class and function definition statements emit code to perform the local name binding for their defined name. Implementing this PEP should just require intercepting that code generation and replacing it with the code in the block prefix. The one potentially tricky part is working out how to allow the dual use of '@' without rewriting half the grammar definition. More Examples ============= Calculating attributes without polluting the local namespace (from os.py):: # Current Python (manual namespace cleanup) def _createenviron(): ... # 27 line function environ = _createenviron() del _createenviron # Becomes: :environ = @() def _createenviron(): ... # 27 line function Loop early binding:: # Current Python (default argument hack) funcs = [(lambda x, i=i: x + i) for i in range(10)] # Becomes: :funcs = [@(i) for i in range(10)] def make_incrementor(i): return lambda x: x + i # Or even: :funcs = [@(i) for i in range(10)] def make_incrementor(i): :return @ def incrementor(x): return x + i Reference Implementation ======================== None as yet. TO DO ===== Sort out links and references to everything :) Acknowledgements ================ Huge thanks to Gary Bernhardt for being blunt in pointing out that I had no idea what I was talking about in criticising Ruby's blocks, kicking off a rather enlightening process of investigation. References ========== TBD Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

18 75

Long Live PEP 3150 (was: Re: Statement local functions and classes (aka PEP 3150 is dead, say 'Hi!' to PEP 403))
by Eric Snow 18 Oct '11

18 Oct '11

On Fri, Oct 14, 2011 at 12:28 PM, Eric Snow <ericsnowcurrently(a)gmail.com> wrote: > I know for a fact that Nick knows a lot more than me (and has been at > this a lot longer), so I assume that I'm missing something here. The > big advantage of the post-order given statement, that I see, is that > you can do a one-liner: > > x = [given.len(i) for i in somebiglist] given: len = len > > vs. > > given: len = len > x = [given.len(i) for i in somebiglist] After Nick's update to PEP 3150 I saw the post-order light (sort of). If you restrict the given clause to just simple statements, as the PEP does, the post-order variant actually makes more sense. The given clause for simple statements is like giving a suite to all the statements that don't have one[1]. The original statement is then the header for the subsequent block. I like that. If the new syntax were exclusive to simple statements then that's a good fit. I still prefer the in-order variant for compound statements though (they already have their own suite). If PEP 3150 were to march ahead with post-order, we probably couldn't add in-order given clauses for compound statements later, could we? Does it matter? -eric [1] http://mail.python.org/pipermail/python-ideas/2011-April/009891.html

7 9

Tuple Comprehensions
by Karthick Sankarachary 17 Oct '11

17 Oct '11

Hello Python Ideas, To quote the tutorial, "List comprehensions provide a concise way to create lists without resorting to use of map(), filter() and/or lambda.". There are a couple of constraints in the above definition that I feel could be overcome. For one thing, list comprehensions don't let you create immutable sequences (such as a tuple) by design. For another, it does not provide a concise alternative to reduce() function. To address both of the above, I'd like to introduce tuple comprehensions, which would work like so: >>> freshfruit = [' banana', ' loganberry ', 'passion fruit '] >>> (weapon.strip() for weapon in freshfruit) ('banana', 'loganberry', 'passion fruit') >>> import operator >>> (operator.concat(_, weapon.strip()) for weapon in freshfruit) 'bananaloganberrypassion fruit' As you can see, we use parenthesis instead of square brackets around the comprehension. In the first tuple comprehension, we create a true tuple with multiple items. This might a tad more efficient, not to mention less verbose, than applying the "tuple" function on top of a list comprehension. In the second tuple comprehension, we use a reduce() function (specifically operator.concat) to concatenate all of the fruit names. In particular, we use the "_" variable (for lack of a better name) to track the running outcome of the reduce() function. Best Regards, Karthick Sankarachary

12 17

Re: [Python-ideas] Implement comparison operators for range objects
by David Townshend 17 Oct '11

17 Oct '11

On Oct 12, 2011 9:37 PM, "Sven Marnach" <sven(a)marnach.net> wrote: > > Steven D'Aprano schrieb am Do, 13. Okt 2011, um 04:33:49 +1100: > > >When implementing '==' and '!=' for range objects, it would be natural > > >to implement the other comparison operators, too (lexicographically, > > >as for all other sequence types). > > > > I don't agree. Equality makes sense for ranges: two ranges are equal > > if they have the same start, stop and step values. > > No, two ranges should be equal if they represent the same sequence, > i.e. if they compare equal when converted to a list: > > range(0) == range(4, 4, 4) > range(5, 10, 3) == range(5, 11, 3) > range(3, 6, 3) == range(3, 4) > > > But order > > comparisons don't have any sensible meaning: range objects are > > numeric ranges, integer-valued intervals, not generic lists, and it > > is meaningless to say that one range is less than or greater than > > another. > > Well, it's meaningless unless you define what it means. Range objects > are equal if they compare equal after converting to a list. You could > define '<' or '>' the same way. All built-in sequence types support > lexicographical comparison, so I thought it would be natural to bring > the only one that behaves differently in line. (Special cases aren't > special enough...) > > This is just to explain my thoughts, I don't have a strong opinion on > this one. > > I'll try and prepare a patch for '==' and '!=' and add it to the issue > tracker. > > Cheers, > Sven > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > http://mail.python.org/mailman/listinfo/python-ideas If you consider a range to represent a special type of set, which it is since it always contains unique values, then comparison operators do make sense. E.g. range(4,8) < range(2,9) is a subset comparison. +1 for equality checks David

14 27

Statement local namespaces - a different perspective
by Mike Meyer 16 Oct '11

16 Oct '11

Last night, I realized something interesting about statement local namespaces: Pretty much all the arguments, both for and against, apply to allowing suites in compound statements. I take it as given that any suite in a statement can be refactored into a function definition and an appropriate invocation of that function, even though it may require returning a tuple into a tuple assignment. This is one of the key features of programming languages in general. So we could, in theory, restrict all compound statements suites to a single statement. If you needed more than one statement in them, you'd have to wrap the suite in a named function and invoke it appropriately. The benefits from this restriction would be the same as they are for disallowing statement-local namespaces: it makes the suite available for independent testing and reuse, forces the programmer to come up with a (hopefully) descriptive name, and forces parts to be defined before they are used. The downsides are the same problems that the statement-local namespace proposals are addressing: having to come up with a good name, polluting the namespace, and giving the parts more prominent placement than the whole. Python doesn't do this for compound statements because it's impractical. Not every arbitrary collection of statements that needs to be grouped in a suite has a descriptive name or needs to be reused. Putting them in a named function just means you should test the function as well as the statement. In such cases, it's better to just embed the suite into the statement, and Python allows that. I believe the same thing applies to values in an expression. Not every group of sub-expressions has a good name or needs to be reused. Putting them in a named function just means you should test the function as well as the statement. In such cases, the code would be better if you could just embed the suite into the statement. I think it's time Python allowed that. <mike -- Mike Meyer <mwm(a)mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

1 0

Implement comparison operators for range objects
by Sven Marnach 14 Oct '11

14 Oct '11

There are circumstances, for example in unit testing, when it might be useful to check if two range objects describe the same range. Currently, this can't be done using the '==' operator: >>> range(5) == range(5) False To get a useful comparison, you would either need to realise both range objects as lists or use a function like def ranges_equal(r0, r1): if not r0: return not r1 return len(r0) == len(r1) and r0[0] == r1[0] and r0[-1] == r1 [-1] All other built-in sequence types (that is bytearray, bytes, list, str, and tuple) define equality by "all items of the sequence are equal". I think it would be both more consistent and more useful if range objects would pick up the same semantics. When implementing '==' and '!=' for range objects, it would be natural to implement the other comparison operators, too (lexicographically, as for all other sequence types). This change would be backwards incompatible, but I very much doubt there is much code out there relying on the current behaviour of considering two ranges as unequal just because they aren't the same object (and this code could be easily fixed by using 'is' instead of '=='). Opinions? -- Sven

20 33

If I had the time machine for comparisons
by Jim Jewett 13 Oct '11

13 Oct '11

On Wed, Oct 12, 2011 at 11:18 PM, Guido van Rossum <guido(a)python.org> wrote: > (I'm also -1 on adding ordering comparisons; there's little > disagreement on that issue.) If I had a time machine, I would allow comparisons to return "unordered" as well. Right now, objects are comparable or not based strictly on the type, even though comparison is inherently about the values. I think that range(3) < range(10) is obviously true, even though it isn't clear whether or not range(3, 15, 2) < range(7, -8, -1) is true. Here we err by not allowing the first comparison; other objects (like dicts) we err by forcing an arbitrary ordering. -jJ

3 2

Another idea for a switch statement
by Lucas Malor 11 Oct '11

11 Oct '11

Hello all. I read PEP 275 but I don't like the syntax of examples. What do you think about something like this? if x case value1 : [...] case value2 : [...] case value3 : [...] else : [...]

4 3

Help mode improvement.
by Stayvoid 11 Oct '11

11 Oct '11

Hi there! I want to make an improvement connected with the interactive help mode. Example: You want to check some keys in the dictionary, but you can't remember the syntax of the command. If you type something like this: help(key), the interpreter will output an error. Because help(key) is just a plain expression, and it tries to evaluate key first before even calling help(). Maybe help(*key*) could make it work? In my opinion it will be very helpful for newcomers if the interpreter could search for similar commands and output them all. Code listing: help(*key*) - [1] D.has_key() [2] D.keys() … To learn more you should type the number. I think the game is worth the candle if the suggested feature could work as fast as the current helper. Kind regards.

8 8