Mailman 3 December 2011 - Python-ideas

An Alternate Suite Delineation Syntax For Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
by Nick Coghlan 15 Dec '11

15 Dec '11

A more appropriate way to phrase this suggestion would be one that clearly states the problems with the status quo and explores them, rather than assuming an answer, and indulging in a long rambling diatribe against those that will shout you down for adding nothing new to the discussion. Step 1: post to the right mailing list (i.e. python-ideas, not python-dev) Step 2: don't assume the conclusion in the subject line. Ask a question, don't state an imperative. For example: "An Alternate Suite Delineation Syntax For Python?" Step 3: respect the time of others, by trimming your message to its core essence For example: An Alternate Suite Delineation Syntax For Python? ================================================= Python's whitespace based delineation of suites is one of its greatest strengths. It aligns what the human reader perceives with what the computer is actually executing, reducing the frequency of semantic errors due to mismatches between the use of separate block delimiters and the human readable indentation. However, this benefit comes at quite a high price: it is effectively impossible to embed arbitrary Python statements into any environment where leading whitespace is *not* significant, including Python's own expression syntax. It can be argued that this restriction has led directly to the introduction of "expression friendly" variants of several Python top level constructs (for example, lambda expressions, conditional expressions and as a contributing factor in creating the various forms of comprehension). It is also one of the reasons Python-based templating languages almost always create their own custom syntax - embedding Python's own whitespace sensitive statement syntax into environments where leading whitespace is either ignored or forms a significant part of the template output is a formidable challenge. In other languages, this kind of issue is handled by using explicit suite and statement delimiters (often braces and semi-colons, respectively) to allow full suites to be used as expressions. While Python uses braces for another purpose (dictionary and set definitions), it is already the case that semi-colons (';') can be used as statement terminators, both optionally at the end of any simple statement, and also to combine multiple simple statements into a single larger statement (e.g. "x += y; print(x)"). It seems that this existing feature could be combined with a brace-based notation to create an unambiguous "suite expression" syntax that would enjoy the same semantics as ordinary Python suites (i.e. doesn't create a new scope, doesn't directly affect control flow), but allows *all* Python statements to be embedded inside expressions. Currently, the character sequence "{:" is a Syntax Error: you are attempting to end a compound statement header line while an opening brace remains unmatched, or else trying to build a dictionary without specifying the key value. This creates an opportunity to re-use braces for a suite expression syntax without conflicting with their use for set and dictionary construction. Specifically, it should be possible to create a variant of the top-level Python syntax that: 1. Explicitly delimits suites using the notation "{:" to open the suite and "}" to end it 2. Requires the use of ";" to separate simple statements (i.e. newline characters would not end a statement, since we would be inside an expression) 3. Requires that all subordinate suites also be suite expressions (i.e. leading whitespace would not be significant, since we would be inside an expression) 4. Returns the value of the last statement in the suite as the result of the suite expression (since return statements would affect the containing scope) 5. Anonymous class and function definitions would be permitted in a suite expression (but accepting only a suite expression instead of an ordinary suite) (Ruby block notation and C's comma expressions are pretty much direct inspiration for the above list) Some examples: Raise expressions: x = y if y is not None else {: raise ValueError("y must not be None!")} Try expressions: x = {: try {: y.hello} except AttributeError {: "world!"}} With expressions: data = {: with open(fname) as f {: f.read()}} Embedded assignments: if {: m = pat.search(data); m is not None}: # do something with m else: # No match! In-order conditional expressions: {: if a {:b} else {:c}} One-liner accumulator function: def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return s}}} Cheers, Nick. P.S. I actually think the above idea is kinda cryptic and ugly. It is, however, extraordinarily powerful, especially when it comes to embedding Python code inside other environments that may not be friendly to significant leading whitespace. Applying the excellent set of criteria suggested by Mike Meyer: 1. What's the use case? Cleanly embedding arbitrary Python statements inside environments that are not generally sensitive to leading whitespace, such as templating languages, Python expressions and Python strings. 2. Does it make such code more readable? Yes, I believe it does, by providing "one obvious way to do it". Currently there are a mish-mash of alternatives out there (or else people give up and use something else entirely, like Javascript). 3. Does it encourage writing unreadable code? Perhaps. You certainly end up with a lot of braces, colons and semi-colons floating around. However, that's also something that can be addressed by style guides - if people are using the embedded syntax when the indented syntax would work fine, the problem isn't really due to the embedded syntax, it's due to people not caring about maintainability. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

19 52

Re: [Python-ideas] Inconsistent script/console behaviour
by Nick Coghlan 15 Dec '11

15 Dec '11

(moved thread to python-ideas, where it belongs. Messing with 20 years of precedent is *not* a topic for python-dev) On Thu, Dec 15, 2011 at 6:58 PM, anatoly techtonik <techtonik(a)gmail.com> wrote: >> You can't fix this without completely changing the way the interactive >> console treats blank lines. > > But the fix doesn't require changing the way interactive console treats > blank lines at all. It only requires to finish current block when a dedented > line is encountered and not throwing obviously confusing SyntaxError. At the > very least it should not say it is SyntaxError, because the code is pretty > valid Python code. If it appears to be invalid "Python Console code" - the > error message should say that explicitly. That would be a correct > user-friendly fix for this UX issue, but I'd still like the behavior to be > fixed - i.e. "allow dedented lines end current block in console without > SyntaxError". Right now I don't see the reasons why it is not possible. > > Please speak code when replying about use cases/examples that will be broken > - I didn't quite get the problem with "global scope if" statements. There are two steps to what the interactive console does: Step 1: accept a block of code. Step 2: compile and execute it in single statement mode (in order to get the result of any expressions for display) It is the *first* step that is terminated by a blank line. That step understands almost nothing about Python syntax - just enough to figure out whether or not it should be asking for a continuation line. (IIRC, it pretty much just does bracket matching and "if I have seen a ':' character, ask for continuation lines until one of them is blank". So yes, fixing example 1 *does* require significant changes to the way the interactive interpreter works. The interactive interpreter currently operates on individual statements. For compound statements, those statements are terminated by a blank line. If you don't terminate them correctly, then it will attempt to interpret two statements as one, and hence you will get UnexpectedDedentError. To fix example 1, the interactive interpreter would have to be redesigned to use exec mode instead, so it can accept multiple statements at once. That's not impossible, but is a lot of effort to address a relatively minor inconvenience (the inability to tolerate blank lines *within* a suite significantly more annoying in my experience, but even harder to fix than example 1 would be). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1

Fwd: Transportable indent level markers. >>>===<<<
by Andre Roberge 14 Dec '11

14 Dec '11

On Wed, Dec 14, 2011 at 9:08 PM, Ron Adam <ron3200(a)gmail.com> wrote: > On Wed, 2011-12-14 at 18:53 -0500, Terry Reedy wrote: > > On 12/13/2011 10:53 AM, Guido van Rossum wrote: > > > On Mon, Dec 12, 2011 at 9:24 PM, Terry Reedy<tjreedy(a)udel.edu> wrote: > > >> On 12/12/2011 5:07 PM, Ron Adam wrote: > > >> > > >>> To make that work, we need to use unique character sequences that > aren't > > >>> already used. After some thought, I came up with these > indent-markers. > > >>> > > >>> /// # indent level +1 > > >>> ;;; # indent level +0 > > >>> \\\ # indent level -1 > Why not use INDENT, ;, and DEDENT (or <INDENT> <;> and <DEDENT>) ? ;-) André

1 0

Re: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section to the tutorial
by Richard Prosser 12 Dec '11

12 Dec '11

Ned, I accept your comment about the tutorial - I wasn't sure about that. Thanks. However I still feel that there are some aspects of the language which are not in the true spirit of Python (i.e. 'intuitive'). The discussion about default mutable types is one of these. It seems to me that the essential problem is that of assignment in general, which (I believe) creates a reference on the LHS to the object on the RHS, rather than having a copy operation to make the two objects completely separate. That can be confusing in other contexts, not just with default parameters. If I am to write a 'gotchas' FAQ or whatever then I would like to understand the reasoning behind such design decisions but I can't find any 'deep' explanations at present - just several posts about people being puzzled! A similar comment applies to the lack of type declarations. So if you or anyone else can explain exactly why such odditties are implemented I would be grateful. Unfortunately it is almost certainly too late to propose fixes (if appropriate) for such quirks in Python 3 but at least I should be able provide arguments as to why things are done the way they are. Richard On 11 December 2011 01:21, <python-ideas-request(a)python.org> wrote: > Message: 2 > Date: Sat, 10 Dec 2011 13:34:15 -0500 > From: Ned Batchelder <ned(a)nedbatchelder.com> > To: Richard Prosser <richard.prosser(a)mail.com> > Cc: python-ideas(a)python.org > Subject: Re: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' > section to the tutorial > Message-ID: <4EE3A627.8010002(a)nedbatchelder.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 12/10/2011 9:16 AM, Richard Prosser wrote: > > Although I love Python there are some aspects of the language design > > which are disappointing and which can even lead to problems in some > cases. > > > > A classic example is a mutable default argument having the potential > > to produce unexpected side-effects, as a consequence of the > > non-intuitive scoping rules. > > > > Another awkward 'feature' is the requirement for a trailing comma in > > singleton tuples, due I believe to the use of expression parentheses > > rather than (say) the use of special brackets like chevrons. > > > > Something that I personally wish for is the ability to declare > > variable types 'up front' but that facility is missing from Python. > > > One of these things is not like the others. Mutable default arguments, > and singleton tuples are surprises that make sense once you understand > things on a deeper level. It makes sense to call them out as a > "gotcha": a common stumbling block for learners. But "no type > declarations" is not a wart, it's a fundamental feature of the language > that is immediately apparent from the first lesson. > > --Ned. >

7 14

Avoiding nested for try..finally: atexit for functions?
by Nikolaus Rath 12 Dec '11

12 Dec '11

Hello, I often have code of the form: def my_fun(): allocate_res1() try: # do stuff allocate_res2() try: # do stuff allocate_res3() try: # do stuff finally: cleanup_res3() finally: cleanup_res2() finally: cleanup_res1() return With increasing number of managed resources, the indentation becomes really annoying, there is lots of line noise, and I don't like the fact that the cleanup is so far away from the allocation. I would much rather have something like this: def my_fun(): allocate_res1() atreturn.register(cleanup_res1) # do stuff allocate_res2() atreturn.register(cleanup_res2) # do stuff allocate_res3() atreturn.register(cleanup_res3) # do stuff return Has the idea of implementing such "on return" handlers ever come up? Maybe there is some tricky way to do this with function decorators? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

8 18

Re: [Python-ideas] Adding braces to __future__
by Cameron Simpson 09 Dec '11

09 Dec '11

On 10Dec2011 11:01, Steven D'Aprano <steve(a)pearwood.info> wrote: | Guido van Rossum wrote: | >Point of order (repeated), please move this thread to python-ideas. | | Isn't that cruel to the people reading python-ideas? That's what it's _for_! -- Cameron Simpson <cs(a)zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ A monkey has the right to copy what he sees other monkeys doing. Shouldn't humans have equivalent rights? - Alien Being <http://slashdot.org/~Alien%20Being>

1 0

Re: [Python-ideas] [Python-Dev] [PATCH] Adding braces to __future__
by Dirkjan Ochtman 09 Dec '11

09 Dec '11

On Fri, Dec 9, 2011 at 21:26, Cedric Sodhi <manday(a)gmx.net> wrote: > (not proof-read) A good sign that it's too long to inflict upon others... Also, I scrolled and scrolled but failed to see any example of the kind of change you seem to want. Also, if you really really had read all the prior discussions, you'd have known that this post belongs on python-ideas, not python-dev. Redirecting there. Cheers, Dirkjan

2 1

Exploring the 'strview' concept further
by Nick Coghlan 08 Dec '11

08 Dec '11

With encouragement from me (and others) Armin Ronacher recently attempted to articulate his problems in dealing with the migration to Python 3 [1]. They're actually quite similar to the feelings I had during my early attempts at restoring the ability of the URL parsing APIs to deal directly with ASCII-encoded binary data, rather than requiring that the application developer explicitly decode it to text first [2]. Now, I clearly disagree with Armin on at least one point: there already *is* "one true way" to have unified text processing code in Python 3. That way is the way the Python 3.2 urllib.parse module handles it: as soon as it is handed something that isn't a string, it attempts to decode it using a default assumed encoding (specifically 'ascii', at least for now). It keeps track of whether or not the arguments were decoded from bytes and, if they were, encodes the return value on output [3]. If you're pipelining such interfaces, it's obviously more efficiently to just decode once before invoking the pipeline and then (optionally) encoding again at the end (just as is the case in Python 2), but you can still make your APIs largely polymorphic with respect to bytes and text without massive internal code duplication. So, that's always one of my first suggestions to people struggling with Python 3's unicode model: I ask if they have tried putting aside any concerns they may have about possible losses of efficiency, and just tried the decode-on-input-and-return-an-output-coercion-function, coerce-on-output approach. Python used to do this implicitly for you at every string operation (minus the 'coerce on output' part), but now it is asking that you do it manually, and decide for *yourself* on an appropriate encoding, instead of the automatic assumption of ASCII text that is present in Python 2 (we'll leave aside the issue of platform-specific defaults in various contexts - that's a whole different question and one I'm not at all equipped to answer. I don't think I've ever even had to work on a system with any locale other than en_US or en_GB). Often this actually resolves their problem (since they're no longer fighting the new Unicode model, and instead embracing it), and this is why PEP 393 is going to be such a big deal when Python 3.3 is released next year. Protocol developers are *right* to be worried about a four-fold increase in memory usage (and the flow on effects on CPU usage and cache misses) when going from bytes data to the UCS4 internal Unicode format used on most distro-provided Python builds for Linux. With PEP 393's flexible internal representations, the amount of memory used will be as little as possible while still allowing straightforward O(1) lookup of individual code points. However, that urllib.urlparse code also highlights another one of Armin's complaints: like much of the stdlib (and core interpreter!), it doesn't ducktype 'str'. Instead, it demands the real thing and accepts no substitutes (not even collections.UserString). This kind of behaviour is quite endemic - the coupling between the interpreter and the details of the string implementation is, in general, even tighter than that between the interpreter and the dict implementation used for namespaces. With PEP 3118, we introduced the concept of 'memoryview' to make allowance for the fact that it is often useful to look at the same chunk of memory in multiple ways, *without* incurring the costs of making multiple copies. In a discussion back in June [4], I briefly mentioned the idea of a 'strview' type that would extend those concepts to providing a str-like view of a region of memory, *without* necessarily making a copy of the entire thing. DISCLAIMERS: 1. I don't know yet if this is a good idea. It may in fact be a terrible idea. I think it is, at least, an idea worth discussing further. 2. Making this concept work may require actually *classifying* our codecs to some degree (for attributes like 'ASCII-compatible', 'stateless', 'fixed width', etc). That might be tedious, but doesn't seem completely infeasible. 3. There are issues with memoryview itself that should be accounted for if pursuing this idea [5] 4. There is an issue with CPython's operand coercion for sequence concatenation and repetition that may affect attempts to implement this idea, although you should be fine so long as you implement the number methods in addition to the sequence ones (which happens automatically for classes written in Python) [6] So, how might a 'strview' object work? 1. The basic construction would be "strview(object, encoding, errors)". For convenience, actual str objects would just be returned unmodified (alternatively: a factory function could be provided with that behaviour) 2. A 'strview' *wouldn't* try to pass itself off as a real string for all purposes. Instead, it would support a new String ABC (more on that below). 4. The encode() method would work like a string's normal encode() method, decoding the original object to a str, then encoding that to the desired encoding. If the encodings match, then an optimised fast path of simply calling bytes() on the underlying object would be used. 5. If asked to index, slice or iterate over the underlying string, the strview would use the incremental decoder for the relevant codec to build an efficient mapping from code point indices to byte indices and then return real strings (various strategies for doing this have been posted to this list in the past). Alternatively, if codecs were classified to explicitly indicate when they implemented stateless fixed width encodings, then strview could simply be restricted to only working with that subset of possible encodings. The latter strategy might be needed to get around issues with stateful encodings like ShiftJIS and ITA2 - those are hard (impossible?) to index and interpret efficiently without fully decoding them and storing the result. 6. The new type would implement the various binary operators supported by strings, promoting itself to a real string type whenever needed 7. The new type would similarly support the full string API, returning actual string objects rather than any kind of view. What might a String ABC provide? For a very long time, slice indices had to be real integers - we didn't allow other "integer like" types. The reason was that floats implemented __int__, so ducktyping on that method would have allowed binary floating point numbers in functions where we didn't want to permit them. The answer, ultimately, was to introduce __index__ (and, eventually, numbers.Integral) to mark "true" integers, allowing things like NumPy scalars to be used directly as slice indices without inheriting from int. An explicit String ABC, even if not supported for performance critical core functionality like identifiers, would allow the implementation of code like that in urllib.urlparse to be updated to avoid keying behaviour on the concrete builtin str type - instead, it would check against the String ABC, allowing for all the usual explicit type registration goodies that ABCs support (and that make them much better for type checking than concrete types). Just as much of the old UserDict functionality is now available on Mapping and MutableMapping, so much of the existing UserString functionality could be moved to the hypothetical String ABC. Hopefully-the-rambling-isn't-too-incoherent'ly-yours, Nick. [1] http://lucumr.pocoo.org/2011/12/7/thoughts-on-python3/ [2] http://bugs.python.org/issue9873 [3] http://hg.python.org/cpython/file/default/Lib/urllib/parse.py#l74 [4] http://mail.python.org/pipermail/python-ideas/2011-June/010439.html [5] http://bugs.python.org/issue10181 [6] http://bugs.python.org/issue11477 -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

5 8

Different bases format specification
by T.B. 05 Dec '11

05 Dec '11

8 28

Re: [Python-ideas] Portable Continuations in Python
by Anoop Thomas Mathew 03 Dec '11

03 Dec '11

Hi All, I was wondering whether we can have a feature like, portable continuations (as present in Scala 2.8) is possible in Python. It is a feature, by which executions can be stopped in between, and transferred to another computer and resume the process. This is a standard feature in scala 2.8. Something similar is present in Stackless Python, but it doesn't support 'Portable'. I am not sure how can we do that. But, if it is possible, and if some one can throw some light how to achieve that, I am ready to volunteer to dig deeper. Awaiting comments, Anoop Thomas Mathew atm ___ Life is short, Live it hard.

2 1

Python-ideas December 2011