Long Live PEP 3150 (was: Re: Statement local functions and classes (aka PEP 3150 is dead, say 'Hi!' to PEP 403))
On Fri, Oct 14, 2011 at 12:28 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
After Nick's update to PEP 3150 I saw the post-order light (sort of). If you restrict the given clause to just simple statements, as the PEP does, the post-order variant actually makes more sense. The given clause for simple statements is like giving a suite to all the statements that don't have one[1]. The original statement is then the header for the subsequent block. I like that. If the new syntax were exclusive to simple statements then that's a good fit. I still prefer the in-order variant for compound statements though (they already have their own suite). If PEP 3150 were to march ahead with post-order, we probably couldn't add in-order given clauses for compound statements later, could we? Does it matter? -eric [1] http://mail.python.org/pipermail/python-ideas/2011-April/009891.html
Since I didn't get around to posting my own announcement email, I'll just note this here: As suggested by Eric's change to the subject line, I've now withdrawn the short-lived PEP 403, revived PEP 3150 (statement local namespaces) and updated it based on the feedback received in relation to PEP 403. The current draft of PEP 3150 is available on python.org: http://www.python.org/dev/peps/pep-3150/ On Sun, Oct 16, 2011 at 3:20 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
Not really, because you can embed arbitrary compound statements inside a PEP 3150 style "given" clause if you really want to. If we ever added "given" clauses to compound statements, I'd actually suggest we do it selectively in the respective header lines, assigning scoping semantics that are appropriate for the affected statements. For example: # Embedded assignments in if statements if match is not None given match=re.search(pattern, text): # process match else: # handle case where regex is not None # Embedded assignments in while loops while match is not None given match=re.search(pattern, text): # process match else: # handle case where regex is not None # Shared state for functions def accumulator(x) given tally=0: tally += x return x We may also decide to eliminate the "new scope" implications for 'given' statements entirely and focus solely on the "out of order execution" aspect. That would not only simplify the implementation, but also make for a cleaner extension to the compound statement headers. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Oct 16, 2011, at 12:16 AM, Nick Coghlan wrote:
The current draft of PEP 3150 is available on python.org: http://www.python.org/dev/peps/pep-3150/
FWIW, I think the word "declarative" is being misused. In the context of programming languages, "declarative" is usually contrasted to "imperative" -- describing what you want done versus specifying how to do it. http://en.wikipedia.org/wiki/Declarative_programming I think what you probably meant to describe was something akin to top-down programming http://en.wikipedia.org/wiki/Top%E2%80%93down_and_bottom%E2%80%93up_design#T... using forward declarations: http://en.wikipedia.org/wiki/Forward_declaration . Looking at the substance of the proposal, I'm concerned that style gets in the way of fluid code development. Using the PEPs example as a starting point: sorted_data = sorted(data, key=sort_key) given: def sort_key(item): return item.attr1, item.attr2 What if I then wanted to use itertools.groupby with the same key function? I would first have to undo the given-clause. AFAICT, anything in the given statement block becomes hard to re-use or to apply to more than one statement. My guess is that code written using "given" would frequently have to be undone to allow code re-use. Also, it looks like there is a typo in the attrgetter example (the "v." is wrong). It should read: sorted_list = sorted(original, key=attrgetter('attr1', 'attr2') When used with real field names, that is perfectly readable: sorted(employees, key=attrgetter('lastname', 'firstname') That isn't much harder on the eyes than: SELECT * FROM employees ORDER BY lastname, firstname; Raymond
On Mon, Oct 17, 2011 at 5:53 PM, Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
Agreed, top-down vs bottom-up would be better terminology than declarative vs imperative (although I *do* believe having the given statement available would make it easier to write declarative APIs in Python without abusing decorators or needing to delve into metaclass programming).
Agreed, but is that really so different from lifting *any* inline code out into a separate function for later reuse? Consider that the standard recommendation for "multi-line lambdas" is to define a function immediately before the statement where you want to use it. If that statement is in module level code, fine, that function is now available for reuse elsewhere (perhaps more places than you really want, since it is now potentially part of the module namespace). But in a class scope it's almost certainly more exposed than you want (unless you delete it after use) and in a function scope it isn't exposed any more than it would be in a given statement, and will need to be moved before you can reuse it in a different function. There's a reason the PEP's suggested additions to PEP 8 mention the idea that given statements are a stepping stone towards splitting an operation out into its own function - it's a discrete calculation or other operation, so it could be separated out, but you don't actually want to reuse it anywhere else (or can't come up with a good name), so it makes more sense to leave the code in its current location. It's definitely a concern (and is one of the reasons why I think this PEP needs to bake for a *long* time before it can be considered remotely justifiable), but I still think allowing people to express top down thought processes clearly, and have the code be able to execute in that form is potentially valuable, even if other software engineering concerns soon result in the code being restructured when it comes to application programs.
Oops, you're right - I'll fix that in the next update.
Again, agreed, but I don't think SQL rises to the bar of executable pseudocode either ;) I've actually been trying to think of an example where you'd want to be normalising data on the fly, catching exceptions as you go along, so that lambda expressions and the operator module can't help, with bottom-up programming being the only alternative to the PEP's direct reflection of a top-down thought process. No luck on that front so far, though. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 16 October 2011 08:16, Nick Coghlan <ncoghlan@gmail.com> wrote:
FWIW, I did a quick review of some of my own code where I expected the given clause to be helpful (relatively mathematical code computing continued fractions - lots of complex expressions with subexpressions that can be factored out using throwaway names, and very few of the subexpressions are meaningful enough by themselves to make coming up with helpful names really viable). I was surprised to find that the in-line code, using a lot of x's and y's, was readable enough that I could find little use for the given clause. If the "new scope" semantics is included, I might have got some small benefits from being able to reuse x and y all over the place safe in the assurance that a typo wouldn't cause me to silently pick up the wrong value. But even that is stretching for benefits. I still like the idea in principle, but I'm no longer sure it's as useful in practice as I'd expected. A really good use case would help a lot (Nick's argparse example is a start, but I'm nervous about the idea of writing an API that relies on out of order evaluation to be readable). Paul.
On 17 October 2011 12:56, Paul Moore <p.f.moore@gmail.com> wrote:
Note that, if the "given" keyword creates a new scope, it provides a way to give static variables to functions - a feature for which various ad-hoc syntaxes were recently discussed on this list: counter = counter given: acc = 0 def counter(i): nonlocal acc acc += i return acc -- Arnaud
On Sun, 16 Oct 2011 17:16:40 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Urk. What is the use case for these? Save one line of code but actually type *more* characters? "given" doesn't look like a very pretty notation to me, honestly. Apparently you're looking for a way to allow assignments in other statements but without admitting it (because we don't want to lose face?...). Also: sorted_data = sorted(data, key=sort_key) given: def sort_key(item): return item.attr1, item.attr2 isn't light-weight compared to anonymous functions that other languages have. The whole point of anonymous functions in e.g. Javascript is that embedding them in another statement or expression makes it a very natural way of writing code. The "given" syntax doesn't achieve this IMO; it forces you to write two additional keywords ("given" and "def") and also write twice a function name that's totally useless - since you can't reuse it anyway, as pointed out by Raymond. I'm -1 on that syntax. Regards Antoine.
On Mon, Oct 17, 2011 at 10:30 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
They're throwaway ideas - there's a reason the PEP itself is restricted to simple statements.
If we accept the premise that full featured anonymous functions have their place in life (and, over the years, I've been persuaded that they do), then Python's sharp statement/expression dichotomy and significant leading whitespace at the statement level severely limit our options: 1. Adopt a syntax that still names the functions, but uses those names to allow forward references to functions that are defined later in a private indented suite (this is the route I've taken in PEP 3150). This is based on the premise that the real benefit of anonymous functions lies in their support for top down thought processes, and that linking them up to a later definition with a throwaway name will be less jarring than having to go back to the previous line in order to fill them in while writing code, and then skip over them while reading code to get to the line that matters, before scanning back up to loop at the operation details. 2. Adopt a symbolic syntax to allow a forward reference to a trailing suite that is an implicit function definition somewhat along the lines of Ruby block, only with Python-style namespace semantics (this approach was soundly demolished in the overwhelmingly negative reactions to PEP 403 - the assorted reactions to PEP 3150 have been positively welcoming by comparison) 3. Continue the trend of giving every Python statement an equivalent expression form so that lambda expressions become just as powerful as named functions (we've gone a long way down that road already, but exception handling and name binding are sticking points. One of my goals with PEP 3150 is actually to *halt* that trend by providing a "private suite" that allows the existing significant whitespace syntax to be embedded inside a variety of statements) 4. Add a non-whitespace delimited syntax that allows suites to be embedded inside expressions at arbitrary locations (I believe "from __future__ import braces" answers that one) 5. Just accept that there are some styles of thought that cannot be expressed clearly in Python. Developers that don't like that can either suck it up and learn to live with writing in a style that Python supports (which is, admittedly, not a problem most of the time) or else find another language that fits their brains better. That's a perfectly reasonable choice for us to make, but we should do it with a clear understanding of the patterns of thought that we are officially declaring to be unsupported. That last option, of course, is the status quo that currently wins by default. If anyone can think of additional alternatives outside those 5 options, I'd love to see a PEP :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, 17 Oct 2011 23:18:53 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
They are nice to have. But so are switch statements, multi-line comments, syntactic support for concurrency and other constructs that Python doesn't have.
The "postdef" keyword is arguably inelegant. You haven't answered to my lambda-based proposal on this PEP. What do you think of it?
:-} Regards Antoine.
On 10/17/2011 9:18 AM, Nick Coghlan wrote:
It may be that I have spent so much time using Twisted, but this seems like the only viable solution given my need to define callbacks simultaneously with an errback. That is: d = getDeferredFromSomewhere() d.addCallbacks(callback, errback) Being only able to name a single anonymous function is significantly less useful (perhaps to the point of useless) to anyone writing Twisted Deferred-style code. (BTW, you cannot split that into two statements to weasel around the "given" limitation). Personally, I never noticed that this "style" was awkward. If I was writing C, then I would have to put those functions into the file-scope and, while there is no ordering restriction there, I always put them above all of the functions I used them in. So when I came to Python, I just kept with that style except that I don't have to elevate them to the module-scope if a closure is useful (often it is!). Usually my callbacks/errbacks are non-trivial enough that I would not want to put them inline with executable statements -- I usually pull them to the top of whatever scope I am defining them and live with the out-of-order-ness. However, I could see how hanging them off the bottom in an indented scope might be more readable. -- Scott Dial scott@scottdial.com
participants (7)
-
Antoine Pitrou
-
Arnaud Delobelle
-
Eric Snow
-
Nick Coghlan
-
Paul Moore
-
Raymond Hettinger
-
Scott Dial