Mailman 3 July 2013 - Python-ideas

Reference variable in assignment: x = foo(?)
by Corey Sarsfield July 12, 2013

July 12, 2013

I've always found +=, -= and the like to be handy, but I had hoped like so many other things in python there would be a generic form of this functionality. x += 5 could be expressed as x = ? + 5 perhaps.

9 12

Allow Enum members to refer to each other during execution of body
by Antony Lee July 11, 2013

July 11, 2013

Currently, during the execution of the body of the Enum declaration, member names are bound to the values, not to the Enum members themselves. For example class StateMachine(Enum): A = {} B = {1: A} # e.g. a transition table StateMachine.B[1] == {}, when one could have expected StateMachine.B[1] == StateMachine.A It seems to me that a behavior where member names are bound to the members instead of being bound to the values is more useful, as one can easily retrieve the values from the members but not the other way round (at least during the execution of class body). Initially, I thought that this could be changed by modifying _EnumDict, so that its __setitem__ method sets the member in the dict, instead of the value, but in fact this doesn't work because while the values are being set in the _EnumDict the class itself doesn't exist yet (and for good reason: the __init__ and __new__ methods may be defined later but there is no way to know that). However, a possible solution could to momentarily create Enum members as instances of some dummy class, and then later, after execution of class body has completed, change the members' class to the actual Enum and initialize them as needed (if an __init__ or a __new__ are actually defined). Well, there are limitations with this approach (e.g. the members are not fully initialized before class body finishes to execute) but this seems better than the current behavior(?) Best, Antony

4 8

unicodedata.itergraphemes (or str.itergraphemes / str.graphemes)
by David Kendal July 10, 2013

July 10, 2013

Hi, Python provides a way to iterate characters of a string by using the string as an iterable. But there's no way to iterate over Unicode graphemes (a cluster of characters consisting of a base character plus a number of combining marks and other modifiers -- or what the human eye would consider to be one "character"). I think this ought to be provided either in the unicodedata library, (unicodedata.itergraphemes(string)) which exposes the character database information needed to make this work, or as a method on the built-in str type. (str.itergraphemes() or str.graphemes()) Below is my own implementation of this as a generator, as an example and for reference. --- import unicodedata def itergraphemes(string): def ismodifier(char): return unicodedata.category(char)[0] == 'M' start = 0 for end, char in enumerate(string): if not ismodifier(char) and not start == end: yield string[start:end] start = end yield string[start:] --- Thanks, dpk

10 21

Why can't we pickle module objects?
by Haoyi Li July 9, 2013

July 9, 2013

>>> import pickle >>> pickle.dumps(pickle.dumps) 'cpickle\ndumps\np0\n.' >>> pickle.dumps(pickle) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Runtimes\Python\lib\pickle.py", line 1374, in dumps Pickler(file, protocol).dump(obj) File "C:\Runtimes\Python\lib\pickle.py", line 224, in dump self.save(obj) File "C:\Runtimes\Python\lib\pickle.py", line 306, in save rv = reduce(self.proto) File "C:\Runtimes\Python\lib\copy_reg.py", line 70, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle module objects I know that you can't, but why not? You can pickle class objects and function objects in modules, and their static path is stored so that when you unpickle it, the correct module is imported and the function is retrieved from that module. It seems odd an inconsistent that you can't pickle the module object itself; can't it just store itself as a name, and have loads() import it and return the resultant module object? This isn't entirely of academic interest; I'm working with some code which is meant to pickle/unpickle arbitrary things, and occasionally it blows up when i accidentally pass in a module. It's always possible to work around the TypeErrors by just changing the stuff I pass in for serializing from the module to the exact function/class i want, but it seems like needless pain. Is there any good reason we don't let people pickle module objects using the same technique that we use to pickle classes and top-level functions? -Haoyi -Haoyi

4 5

exclusively1, common, exclusively2 = set1 - set2, set1 & set2, set2 - set1
by Paddy3118 July 8, 2013

July 8, 2013

I found myself repeating something that I know I have used before, several times: I get two sets of results, may be sets of the passing tests when a design has changed, and I need to work out what has changed so work out 1. What passed first time round 2. What passed both times. 3. What passed only the second time round. I usually use something like the set equations in the title to do this but I recognise that this requires both sets to be traversed at least three times which seems wasteful. I wondered if their was am algorithm to partition the two sets of data into three as above, but cutting down on the number of set traversals? I also wondered that if such an algorithm existed, would it be useful enough to be worth incorporating into the Python library? Maybe defined as: exclusively1, common, exclusively2 = set1.partition(set2)

17 44

Macro-Powered Enums
by Haoyi Li July 6, 2013

July 6, 2013

Hey All, I know this ship has already sailed with PEP 435, but I've been working on a prototype implementation of enums using MacroPy macros<https://github.com/lihaoyi/macropy#enums> . The goal was to have enums whose methods provide most of the nice capabilities of int enums (auto-generated indexes, fast comparison, incrementing, index-arithmetic, find-by-index) and string enums (nice __repr__, find-by-name) but has the ability to smoothly scale to full-fledged objects with methods *and* fields, all this is an extremely concise way. The gimmick here is that it uses macros to provide a concise syntax to allow you to construct each enum instance with whatever constructor parameters you wish, java-enum-style (here<https://github.com/lihaoyi/macropy#complex-enums>). This allows a degree of enums-as-objects which i haven't seen in any other library (most don't allow custom fields). Probably not standard-library worthy, but I thought it was pretty cool. -Haoyi

2 1

format specifier for "not bytes"
by Daniel Holth July 5, 2013

July 5, 2013

While I was implementing JSON-JWS (JSON web signatures), a format which in Python 3 has to go from bytes > unicode > bytes > unicode several times in its construction, I notice I wrote a lot of bugs: "sha256=b'abcdef1234'" When I meant to say: "sha256=abcdef1234" Everything worked perfectly on Python 3 because the verifying code also generated the sha256=b'abcdef1234' as a comparison. I would have never noticed at all unless I had tried to verify the Python 3 output with Python 2. I know I'm a bad person for not having unit tests capable enough to catch this bug, a bug I wrote repeatedly in each layer of the bytes > unicode > bytes > unicode dance, and that there is no excuse for being confused at any time about the type of a variable, but I'm not willing to reform. Instead, I would like a new string formatting operator tentatively called 'notbytes': "sha256=%notbytes" % (b'abcdef1234'). It gives the same error as 'sha256='+b'abc1234' would: TypeError: Can't convert 'bytes' object to str implictly

8 17

Fwd: exclusively1, common, exclusively2 = set1 - set2, set1 & set2, set2 - set1
by Joshua Landau July 4, 2013

July 4, 2013

---------- Forwarded message (apologies, the CC says it all) ---------- From: Joshua Landau <joshua.landau.ws(a)gmail.com> Date: 4 July 2013 00:57 Subject: Re: [Python-ideas] exclusively1, common, exclusively2 = set1 - set2, set1 & set2, set2 - set1 To: Paddy3118 <paddy3118(a)gmail.com> Cc: python-ideas(a)googlegroups.com On 3 July 2013 21:50, Paddy3118 <paddy3118(a)gmail.com> wrote: > I found myself repeating something that I know I have used before, several > times: I get two sets of results, may be sets of the passing tests when a > design has changed, and I need to work out what has changed so work out > > 1. What passed first time round > 2. What passed both times. > 3. What passed only the second time round. > > I usually use something like the set equations in the title to do this but I > recognise that this requires both sets to be traversed at least three times > which seems wasteful. As far as I understand, this requires only 3 traversals in total. > I wondered if their was am algorithm to partition the two sets of data into > three as above, but cutting down on the number of set traversals? You could cut it down to two, AFAIK. This seems like a minor gain. > I also wondered that if such an algorithm existed, would it be useful enough > to be worth incorporating into the Python library? > > Maybe defined as: > > exclusively1, common, exclusively2 = set1.partition(set2) Something more useful, as it's just as good in this case could be: set1.partition(set2) === set1 - set2, set1 & set 2 similarly to how we have divmod.

1 0

Parenthesized Compound With Statement
by Matthew Lefavor July 3, 2013

July 3, 2013

As you all know, Python supports a compound "with" statement to avoid the necessity of nesting these statements. Unfortunately, I find that using this feature often leads to exceeding the 79-character recommendation set forward by PEP 8. # The following is over 79 characters with open("/long/path/to/file1") as file1, open("/long/path/to/file2") as file2: pass This can be avoided by using the line continuation character, like so: with open("/long/path/to/file1") as file1, \ open("/long/path/to/file2") as file2: pass But PEP-8 prefers using implicit continuation with parentheses over line continuation. PEP 328 states that using the line continuation character is "unpalatable", which was the justification for allowing multi-line imports using parentheses: from package.subpackage import (UsefulClass1, UsefulClass2, ModuleVariable1, ModuleVariable2) Is there a reason we cannot do the same thing with compound with statements? Has this been suggested before? If so, why was it rejected? with (open("/long/path/to/file1") as file1, open("/long/path/to/file2") as file2): pass I would be happy to write the PEP for this and get plugged in to the Python development process if this is an idea worth pursuing. ML

8 18

Re: [Python-ideas] Is this PEP-able? for X in ListY while conditionZ:
by Wolfgang Maier July 3, 2013

July 3, 2013

On Jun 27, 2013, at 5:19, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >> If you're willing, I'm actually thinking this may be one of those >> discussions that's worth summarising in a PEP, even if it's just to >> immediately mark it Rejected. Similar to PEP 315 and a few other PEPs, >> it can help to have a document that clearly spells out the objective >> (which I think you summarised nicely as "trying to find a syntactic >> replacement for itertools.takewhile, just as comprehensions replaced >> many uses of map and filter"), even if no acceptable solution has been >> found. > >The ideas are pretty broad-ranging. In particular, #1 got two somewhat >supportive responses that had nothing to do with the comprehension termination >idea. Do side issues like that need to be discussed first/separately before >referencing them in a while clause PEP? > >Also, we seem pretty far from a consensus on what the actual tradeoffs are for >most of the options. For example, is the definition of comps in terms of loops >the core idea behind the abstraction, or a minor triviality that's only useful >in understanding bad code? Are the differences between comps and genexps a bug >or a feature? > >Finally, how does this connect up to the original idea of this thread, which was >to draft a PEP for while clauses in loop statements rather than in >comprehensions? Obviously if that existed, it would change the options for comp >syntax. Do we need to include that idea in the discussion? (I completely forgot >about it while digging up all of the ideas that spun off from it...) I don't think while clauses in loop statements are a big gain for the language. There's break and despite all the programming-style discussions going on in this part of the thread, it has been working well for many years and most people find it intuitive to use. >> That phrasing of the objective also highlights a counter argument I >> hadn't considered before: if we don't consider takewhile a common >> enough use case to make it a builtin, why on *earth* are we even >> discussing the possibility of giving it dedicated syntax? > >Besides early termination for comps, the only good use case I know of is in code >that already makes heavy use of itertools, and making one of the functions a >builtin wouldn't change anything. > >And if early termination for comps is a special case, it doesn't seem too >unreasonable to consider other ways to handle it. > >But you're right that "make takewhile a builtin" should probably be considered >among the alternatives. But a builtin takewhile would still not come with nicer and easier to read syntax! I guess the use cases are not that rare, it's just that right now people switch to explicit loops when they need early termination because it keeps things readable. I'm encountering this situation quite regularly (reading a small header from large files is the most common example, but there are others). Let me suggest one more solution although it requires a new keyword: introduce *breakif* condition and define its translation as if condition: break . You can now write (x for x in iterable breakif x < 0) and I don't see a way how that could possibly be misread by anyone. Also it would translate unambiguously to the explicit: for x in iterable: breakif x<0 # itself translating to if x<0: break yield x It would work with genexps, comprehensions and explicit loops alike (with very little benefit for the later, though maybe it increases readability even there by making it clear from the start of the line what the purpose of the condition test is). Best, Wolfgang

19 40