Mailman 3 May 2011 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley 16 Dec '20

16 Dec '20

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich

10 15

i18n and Python tracebacks
by Andre Roberge 28 Oct '12

28 Oct '12

I think it would be a good idea if Python tracebacks could be translated into languages other than English - and it would set a good example. For example, using French as my default local language, instead of >>> 1/0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: integer division or modulo by zero I might get something like >>> 1/0 Suivi d'erreur (appel le plus récent en dernier) : Fichier "<stdin>", à la ligne 1, dans <module> ZeroDivisionError: division entière ou modulo par zéro André

9 15

Cofunctions PEP - Revision 4
by Greg Ewing 13 Oct '12

13 Oct '12

Here's an updated version of the PEP reflecting my recent suggestions on how to eliminate 'codef'. PEP: XXX Title: Cofunctions Version: $Revision$ Last-Modified: $Date$ Author: Gregory Ewing <greg.ewing(a)canterbury.ac.nz> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Feb-2009 Python-Version: 3.x Post-History: Abstract ======== A syntax is proposed for defining and calling a special type of generator called a 'cofunction'. It is designed to provide a streamlined way of writing generator-based coroutines, and allow the early detection of certain kinds of error that are easily made when writing such code, which otherwise tend to cause hard-to-diagnose symptoms. This proposal builds on the 'yield from' mechanism described in PEP 380, and describes some of the semantics of cofunctions in terms of it. However, it would be possible to define and implement cofunctions independently of PEP 380 if so desired. Specification ============= Cofunction definitions ---------------------- A cofunction is a special kind of generator, distinguished by the presence of the keyword ``cocall`` (defined below) at least once in its body. It may also contain ``yield`` and/or ``yield from`` expressions, which behave as they do in other generators. From the outside, the distinguishing feature of a cofunction is that it cannot be called the same way as an ordinary function. An exception is raised if an ordinary call to a cofunction is attempted. Cocalls ------- Calls from one cofunction to another are made by marking the call with a new keyword ``cocall``. The expression :: cocall f(*args, **kwds) is evaluated by first checking whether the object ``f`` implements a ``__cocall__`` method. If it does, the cocall expression is equivalent to :: yield from f.__cocall__(*args, **kwds) except that the object returned by __cocall__ is expected to be an iterator, so the step of calling iter() on it is skipped. If ``f`` does not have a ``__cocall__`` method, or the ``__cocall__`` method returns ``NotImplemented``, then the cocall expression is treated as an ordinary call, and the ``__call__`` method of ``f`` is invoked. Objects which implement __cocall__ are expected to return an object obeying the iterator protocol. Cofunctions respond to __cocall__ the same way as ordinary generator functions respond to __call__, i.e. by returning a generator-iterator. Certain objects that wrap other callable objects, notably bound methods, will be given __cocall__ implementations that delegate to the underlying object. Grammar ------- The full syntax of a cocall expression is described by the following grammar lines: :: atom: cocall | <existing alternatives for atom> cocall: 'cocall' atom cotrailer* '(' [arglist] ')' cotrailer: '[' subscriptlist ']' | '.' NAME Note that this syntax allows cocalls to methods and elements of sequences or mappings to be expressed naturally. For example, the following are valid: :: y = cocall self.foo(x) y = cocall funcdict[key](x) y = cocall a.b.c[i].d(x) Also note that the final calling parentheses are mandatory, so that for example the following is invalid syntax: :: y = cocall f # INVALID New builtins, attributes and C API functions -------------------------------------------- To facilitate interfacing cofunctions with non-coroutine code, there will be a built-in function ``costart`` whose definition is equivalent to :: def costart(obj, *args, **kwds): try: m = obj.__cocall__ except AttributeError: result = NotImplemented else: result = m(*args, **kwds) if result is NotImplemented: raise TypeError("Object does not support cocall") return result There will also be a corresponding C API function :: PyObject *PyObject_CoCall(PyObject *obj, PyObject *args, PyObject *kwds) It is left unspecified for now whether a cofunction is a distinct type of object or, like a generator function, is simply a specially-marked function instance. If the latter, a read-only boolean attribute ``__iscofunction__`` should be provided to allow testing whether a given function object is a cofunction. Motivation and Rationale ======================== The ``yield from`` syntax is reasonably self-explanatory when used for the purpose of delegating part of the work of a generator to another function. It can also be used to good effect in the implementation of generator-based coroutines, but it reads somewhat awkwardly when used for that purpose, and tends to obscure the true intent of the code. Furthermore, using generators as coroutines is somewhat error-prone. If one forgets to use ``yield from`` when it should have been used, or uses it when it shouldn't have, the symptoms that result can be extremely obscure and confusing. Finally, sometimes there is a need for a function to be a coroutine even though it does not yield anything, and in these cases it is necessary to resort to kludges such as ``if 0: yield`` to force it to be a generator. The ``cocall`` construct address the first issue by making the syntax directly reflect the intent, that is, that the function being called forms part of a coroutine. The second issue is addressed by making it impossible to mix coroutine and non-coroutine code in ways that don't make sense. If the rules are violated, an exception is raised that points out exactly what and where the problem is. Lastly, the need for dummy yields is eliminated by making it possible for a cofunction to call both cofunctions and ordinary functions with the same syntax, so that an ordinary function can be used in place of a cofunction that yields zero times. Record of Discussion ==================== An earlier version of this proposal required a special keyword ``codef`` to be used in place of ``def`` when defining a cofunction, and disallowed calling an ordinary function using ``cocall``. However, it became evident that these features were not necessary, and the ``codef`` keyword was dropped in the interests of minimising the number of new keywords required. The use of a decorator instead of ``codef`` was also suggested, but the current proposal makes this unnecessary as well. It has been questioned whether some combination of decorators and functions could be used instead of a dedicated ``cocall`` syntax. While this might be possible, to achieve equivalent error-detecting power it would be necessary to write cofunction calls as something like :: yield from cocall(f)(args) making them even more verbose and inelegant than an unadorned ``yield from``. It is also not clear whether it is possible to achieve all of the benefits of the cocall syntax using this kind of approach. Prototype Implementation ======================== An implementation of an earlier version of this proposal in the form of patches to Python 3.1.2 can be found here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cofunctions.h… If this version of the proposal is received favourably, the implementation will be updated to match. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

11 29

Adding 'bytes' as alias for 'latin_1' codec.
by INADA Naoki 03 Jun '11

03 Jun '11

Hi, all. There are some situation that I want to use bytes as a string in real world. (I use the 'bstr' for bytes as a string below) Sadly, Python 3's bytes is not bytestring. For example, when I want to make 'cat -n' that is transparent to encoding, Python 3 doesn't permit b'{0:6d}'.format(n) and '{0:6d}'.format(n).encode('ascii') is circuitous way against simple requirements. I think the best way to handle such situation with Python 3 is using 'latin1' codec. For example, encoding transparent 'cat -n' is: import sys fin = open(sys.stdin.fileno(), 'r', encoding='latin1') fout = open(sys.stdout.fileno(), 'w', encoding='latin1') for n, L in enumerate(fin): fout.write('{0:5d}\t{1}'.format(n, L)) If using 'latin1' is Pythonic way to handle encoding transparent string, I think Python should provide another alias like 'bytes'. Any thoughts? -- INADA Naoki <songofacandy(a)gmail.com>

16 57

Minor tweak to PEP 8?
by Mike Meyer 02 Jun '11

02 Jun '11

PEP eight has an interesting omission in the "Code Layout" section. It doesn't say how to indent continuation lines when code is wrapped to comply with the line length limits. It has examples, but no textual guides. Which means you can do a rock-stupid word warp (with no indentation on the continuation lines), point at the resulting mess, and say "See? If we follow this part of the PEP, we get really ugly code!". Mail doing just that is what prompted this suggestion. I therefore propose adding a sentence or two to this section, something along the lines of: The continuation line(s) should be indented to reflect the structure of the statement being continued. This should be at least one space beyond the first open parenthesis that is not closed on the continued line, if present. Nothing hard and fast, just a requirement to use good sense and the minimal indent resulting from doing so. <mike -- Mike Meyer <mwm(a)mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

11 20

Threading hooks and disable gc per thread
by Christian Heimes 27 May '11

27 May '11

Hello, today I've spent several hours debugging a segfault in JCC [1]. JCC is a framework to wrap Java code for Python. It's most prominently used in PyLucene [2]. You can read more about my debugging in [3] With JCC every Python thread must be registered at the JVM through JCC. An unattached thread, that accesses a wrapped Java object, leads to errors and may even cause a segfault. Accessing also includes garbage collection. A code line like a = {} or "a b c".split() can segfault since the allocation of a dict or a bound method runs through _PyObject_GC_New(), which may trigger a cyclic garbage collection run. If the current thread isn't attached to the JVM but triggers a gc.collect() with some Java objects in a cycle, the interpreter crashes. It's quite complicated and hard to "fix" third party tools to attach all threads created in the third party library. The issue could be solved with a simple on_thread_start hook in the threading module. However there is more to it. In order to free memory threads must also be detached from the JVM, when a thread has ended. A second on_thread_stop hook isn't enough since the bound methods may also lead to a gc.collect() run after the thread is detached. I propose three changes to Python in order to fix the issue: on thread start hook -------------------- Similar to the atexit module, third party modules can register a callable with *args and **kwargs. The functions are called inside the newly created thread just before the target is called. The best place for the hook list is threading.Thread._bootstrap_inner() right before the try: self.run() except: block. Exceptions are ignored during the call but reported to the user at the end (same as atexit's atexit_callfunc()) on thread end hook ------------------ Same as on thread start hook but the callables are called inside the dying thread after self.run(). gc.disable_thread(), gc.enable_thread(), gc.isenabled_thread() -------------------------------------------------------------- Right now almost any code can trigger a gc.collect() run non-deterministicly. Some application like JCC want to control if gc.collect() is wanted on a thread level. This could be solved with a new flat in PyThreadState. PyThreadState->gc_enabled is enabled by default. When the flag is false, _PyObject_GC_Malloc() doesn't start a gc.collect() run for that thread. The collection is delayed until another thread or the main thread triggers it. The three functions should also have a C equivalent so C code can prevent gc in a thread. Thoughs? Christian [1] http://lucene.apache.org/pylucene/jcc/index.html [2] http://lucene.apache.org/pylucene/ [3] http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/201105.mbox/br…

5 7

Suggestion: Integrate the script "pindent.py" as standard command for formatting pyhton code
by Serge Hulne 27 May '11

27 May '11

Suggestion: Integrate the script "pindent.py" as standard command for formatting pyhton code Here is the link; http://svn.python.org/projects/python/trunk/Tools/scripts/pindent.py Pindent stands for "Pyton indent": Goal : 1. It provides bloc delimiters (end of blocks) in the for of comments (like "#end if" or "#end for" etc ... ) 2. This allows one to check / restore the indentation of Python code, in cases where> 1. A copy/paste went wrong 2. The indentation of a Python source got corrupted when the script was posted on web page, send via email etc ... 3. Standardise (fix) sources which happily mix whitespaces and tabs 4. Make Python code more readable for developers used to end of blocs delimiters (Ruby, C, C++, C#,Java, etc ...) Basically the idea is the same as the Go language "gofmt" (Go format). Example: #------------------- - Before using pindent: #!/usr/bin env python i = 0 for c in "hello world": if c == 'l': i+=1 print "number of occurrences of `l` :", i #------------------ - After using indent: #!/usr/bin env python i = 0 for c in "hello world": if c == 'l': i+=1 print "number of occurrences of `l` :", i # end if # end for Serge Hulne

12 12

Python 3.x and bytes
by Ethan Furman 24 May '11

24 May '11

As those who have to work with byte strings know, when retrieving a single character from a byte string, what you get back is not a byte string, but an int -- a rather important distinction from unicode strings (str). This has the frustrating side-effect of b'abc'[2] == b'c' being False. It is far too late to change that particular behavior of the byte string (returning int's, that is) -- however, it may not be too late for a non-backwards-incompatible change: have the bytes class' __eq__ method be modified so that it 1) checks to see if the bytes instance is length 1 2) checks to see if a) the other object is an int, and b) 0 <= other_obj < 256 3) if 1 and 2, make the comparison between the int and its single element instead of returning NotImplemented? This makes sense to me -- after all, the bytes class is an array of ints in range(256); it is a special case, but doesn't feel any more special than passing an int into bytes() giving a string of that many null bytes; and it would get rid of the, in my opinion ugly, idiom of some_var[i:i+1] == b'd' It would also not require a new literal syntax. Thoughts? ~Ethan~

13 21

Implementing __dir__ (moving dir implementation to object.__dir__?)
by Michael Foord 24 May '11

24 May '11

Hello all, I'm looking at implementing __dir__ for a class (mock.Mock as it happens) to include some dynamically added attributes, the canonical use case according to the documentation: http://docs.python.org/dev/reference/datamodel.html?highlight=__dir__#objec… What I would like to do is report all the "standard attributes", and then add any dynamically created attributes. So the question is, how do I obtain the "standard list" (the list that dir would normally report in the absence of a custom __dir__ implementation)? There is no object.__dir__ (despite the fact that this is how it is documented...) and obviously calling dir(self) within __dir__ is doomed to failure. The best I have come up with is: def __dir__(self): return dir(type(self)) + list(self.__dict__) + self._get_dynamic_attributes() This works (absent multiple inheritance), but it would be nice to just be able to do: def __dir__(self): standard = super().__dir__() return standard + self._get_dynamic_attributes() Moving the relevant parts of the implementation of dir into object.__dir__ would be one way to solve that. All the best, Michael Foord -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

3 4

__implements__ on arguments to ABCMeta.register
by Eric Snow 24 May '11

24 May '11

ABCMeta.register is great. It adds the cls argument to the _abc_registry of the ABC. However, the class that was passed in does not get touched. If you then want to find out to which classes a class has been registered, you can't find out from that class. Whereas your can find out from an abstract base class which classes have been registered to it. I propose having ABCMeta.register add/update a special method __implements__ to the class that is getting registered. This would not be done to builtin/extension types. It adds the ABC to the __implements__ of the subclass that is getting registered. Something along these lines, right before the final return in the method: if not hasattr(subclass, "__implements__"): try: subclass.__implements__ = {cls} except TypeError: pass else: subclass.__implements__.add(cls) This is a small addition, but I realize it [potentially] adds another special method to classes, so it's not trivial. The use case is that I want to be able to validate that a class implements all of the abstract methods of all the classes to which it has been registered. I don't have a programmatic way of discovering that set without asking every class out there. This is an easy way to accomplish this (for non-extension/non-builtin types). An alternative is to subclass ABCMeta and tack this on, but that only works for my ABCs. Another is to use a class decorator to do this any place I do a register (or even to do the register too), but again, only for the places that I do the registration. Anyway, if it's useful to me then it may be useful to others, so I wanted to put this out there. I expect this has come up before, particularly during discussions about PEP 3119. However, I wasn't able to track down anything specifically about doing this sort of "reverse registration". And, of course, I may be overestimating the value of this functionality. If this does not seem that valuable to anyone else, then no big deal. :) -eric

3 5

CPython optimization: storing reference counters outside of objects
by Amaury Forgeot d'Arc 23 May '11

23 May '11

Hi, 2011/5/23 Sturla Molden <sturla(a)molden.no>: > Instead, we could use multiple heaps: > > Each Python thread could manage it's own heap for malloc and free (cf. > HeapAlloc and HeapFree in Windows). Objects local to one thread only reside > in the locally managed heap. > > When an object becomes shared by seveeral Python threads, it is moved from a > local heap to the global heap of the process. Some objects, such as modules, > would be stored directly onto the global heap. Does this mean that the PyObject* address would change? How would you update all the places that store moved references? -- Amaury Forgeot d'Arc

2 1

Re: [Python-ideas] [Python-Dev] CPython optimization: storing reference counters outside of objects
by Sturla Molden 23 May '11

23 May '11

Den 23.05.2011 06:59, skrev "Martin v. Löwis": > > My expectation is that your approach would likely make the issues > worse in a multi-CPU setting. If you put multiple reference counters > into a contiguous block of memory, unrelated reference counters will > live in the same cache line. Consequentially, changing one reference > counter on one CPU will invalidate the cached reference counters of > that cache line on other CPU, making your problem a) actually worse. In a multi-threaded setting with concurrent thread accessing reference counts, this would certainly worsen the situation. In a single-threaded setting, this will likely be an improvement. CPython, however, has a GIL. Thus there is only one concurrently active thread with access to reference counts. On a thread switch in the interpreter, I think the performance result will depend on the nature of the Python code: If threads share a lot of objects, it could help to reduce the number of dirty cache lines. If threads mainly work on private objects, it would likely have the effect you predict. Which will dominate is hard to tell. Instead, we could use multiple heaps: Each Python thread could manage it's own heap for malloc and free (cf. HeapAlloc and HeapFree in Windows). Objects local to one thread only reside in the locally managed heap. When an object becomes shared by seveeral Python threads, it is moved from a local heap to the global heap of the process. Some objects, such as modules, would be stored directly onto the global heap. This way, objects only used by only one thread would never dirty cache lines used by other threads. This would also be a way to reduce the CPython dependency on the GIL. Only the global heap would need to be protected by the GIL, whereas the local heaps would not need any global synchronization. (I am setting follow-up to the Python Ideas list, it does not belong on Python dev.) Sturla Molden

1 0

function defaults and an empty() builtin
by Jack Diederich 22 May '11

22 May '11

During a code review I got asked a question about a "pythonic" idiom I've been asked about before. The code was like this: def func(optional=None): if optional is None: optional = [] The question was why the optional value wasn't set to an empty list in the first place. The answer is that Really Bad Things can happen if someone actually goes and manipulates that empty list because all future callers will see the modified version. I don't think this defensive programming practice is yet passe - I can think of lots of unit tests that wouldn't trigger bad behavior. You would have to intentionally provoke it by adding some unit tests to be sure. What would make my life a little easier is a builtin container named "empty()" that emulates all builtin containers and raises an exception for any add/subtract manipulations. Something like: class empty(): def _bad_user(self, *args): raise ValueError("empty objects are empty") append = pop = __getitem__ = add = setdeafult = __ior__ = __iand__ = _bad_user def _empty(self): return [] items = keys = values = get = _empty return nothing when asked for something and raise a ValueError when any attempt is made to add/remove items. -Jack

13 34

Filtered "for" loop with list-comprehension-like syntax
by Dan Baker 22 May '11

22 May '11

One common pattern I run across when parsing plain text data files is that I want to skip over blank lines when processing. If I wanted to build a list of all non-blank lines in the file, I could simply do: lines = [line for line in input_file if line.strip()] But as a loop, it almost invariably gets written as: for line in input_file: if not line.strip(): continue # do the real processing It seems odd that "for x in y if z" is allowed in comprehensions but not in a regular for loop. Why not let for x in y if z: do_stuff(x) be a shorthand for for x in y: if not z: continue do_stuff(x) Similarly, I occasionally have multiple sections that need to be handled differently. One way to write this is: for line in input_file: if is_section_delimiter(line): break do_stuff_1(line) for line in input_file: # this picks up where the last one left off if is_section_delimiter(line): break do_stuff_2(line) etc. (This is a little bit of a weird idiom with files since repeated iteration over them remembers where it left off, at least in 2.7.) It would be nice to have this shorthand for it: for line in input_file while not is_section_delimiter(line): do_stuff_1(line) for line in input_file while not is_section_delimiter(line): do_stuff_2(line) etc. This makes it more immediately clear (to me, at least) that it stops at the end of the section. This could also be added to comprehensions; it's somewhat tricky to emulate in comprehensions now. I think the easiest way to do the equivalent of [f(x) for x in y while z] with a comprehension is a = [(f(x) if z else None) for x in y] try: idx = a.index(None) except ValueError: # no None found pass else: # truncate before first None a = a[:idx] but even that fails if None is a potentially valid result of f(x) (or if you forget to use the try/except block and z was always True), and it processes the entire list even though it may throw out a sizable chunk of it immediately after. The only totally safe way I can think of to do it now is by unpacking it into a loop: a = [] for x in y: if not z: break a.append(f(x)) I think adding these would make such idioms a little more readable, but it might not be enough of a gain to justify a syntax addition. Thoughts? Dan Baker

4 4

Type Metadata (and related ideas)
by andrew cooke 19 May '11

19 May '11

Hi, I just finished working on a project that plays around with ABCs and function annotations. The idea was to allow for more delcarative code by adding tools to describe Pytohn data in more detail. While I don't think the result is suitable for adding to Python (it's way too big a change, and it's not yet proven), the process of making something consistent involved working through a lot of ideas about "types in Python" that I recorded at http://www.acooke.org/pytyp.pdf Part of that paper (pages 8 and 9) describes some issues that caused particular problems, including: * The lack of annotations on type generators makes it hard to use annotations as a way of completely describing types. * There seems to be a missing ABC for __getitem__ (which unites lists, dicts and tuples). * As ever, mutability is complicated :o) If we had copy on write lists (which already exist), perhaps we could hash instances efficiently? (OK, this may be already discussed, but I had to mention it) * Given duck typing, shouldn't AttributeError be a TypeError (or vice versa?) * For this particular use-case, an __instancehook__ (which would work much like __subclasshook__) for ABCMeta would have been useful (as described in the paper, polymorphism in Python occurs at the instance level, so asking about the types of instances makes a surprising amount of sense, if done right). Anyway, apologies if some or all of this is old news or inapprorpiate. I just thought people here might find it interesting (you can do things like type check functions and use dynamic dispatch by type - all in a fairly pythonic way... (imho)) Cheers, Andrew PS The project home and more docs are at http://www.acooke.org/pytyp/ ; the code is at http://code.google.com/p/pytyp/ ; pypi page is http://pypi.python.org/pypi/pytyp

1 0

PEP-3151 pattern-matching
by Devin Jeanpierre 19 May '11

19 May '11

Hello, PEP-3151 < http://www.python.org/dev/peps/pep-3151/ > mentions a really weird syntax for pattern-matching. I was hoping I could suggest an alternative that's both more concise, and possible to implement without doing something drastic like changing existing syntax or semantics. The PEP offers the pattern-matching syntax: > except IOError as e if e.errno == errno.ENOENT: ... I'd instead suggest something along the lines of > except io_error(errno.ENOENT): ... Implementing this is fairly straightforward, I've included it in the bottom of this email. Raising an exception becomes `raise io_error(errno.ENOENT)(msg)`. Some notational sugar can be implemented (for example, `raise io_error(errno.ENOENT, msg)` could also be made to work). I'm not fussed about the details. I personally prefer keeping the errnos as a big part of handling exceptions, they're common across OSes and involve less research / memorization for those that are already aware of errno. I guess what I'd like to see is the same exception hierarchy proposed by the PEP, but with the addition of allowing errnos to be specified by pattern-matching, so that errors not covered by the hierarchy, or more specific than the hierarchy, can be concisely caught. However, I'm not really well-versed in the pros and cons for all of this. Above all, I'd like for the pattern matching alternative to be a bit more reasonable. It doesn't have to be verbose and it doesn't have to involve new syntax. Apologies for any mistakes in the code, they are my own. Here's the code: # evil global state or something error_classes = {} def io_error(errno_, msg=None): # or something, you get the idea try: cls = error_classes[errno_] except LookupError: class IOErrorWithErrno(IOError): errno = errno_ cls = error_classes[errno_] = IOErrorWithErrno return error_classes[errno_] # example of usage import errno try: raise io_error(errno.ENOENT)("<Error message>") except io_error(errno.ENOENT): print("success") Thanks for your time! Devin Jeanpierre

15 44

Re: [Python-ideas] [Python-Dev] Python 3.x and bytes
by Ethan Furman 18 May '11

18 May '11

Martin v. Löwis wrote [from python-dev]: > Immutable objects that compare equal should hash equal; > so we would also have to change the hashing of byte strings. Not sure > whether that, in turn, has undesirable consequences. I thought it was the other-way-round -- if they hash equal, they should compare equal? Or is this just for immutables? > In addition, equality should be transitive, so b'A' == 65.0. I'm not sure what you're getting at... we could certainly have step 2 check for a number instead of an int, and then step 3 could extract the one element, giving an int, and then let that int compare itself with the other number, whether it be int, float, fraction, what-have-you. ~Ethan~

2 1

Rename python.exe to python3.exe on Windows
by Devin Jeanpierre 18 May '11

18 May '11

Hello, On most *nix systems, Python 3.x is available as the python3 executable, and Python 2.x as the 'python' executable. This lets both exist side-by-side and be usable from the command-line. The alternative (used by Arch), is to name Python 2.x 'python2', and 3.x 'python'. The Windows distribution of Python does neither, it names them both 'python.exe', meaning that you can't install and use both at once. Moreover, if you install Python 2.7 and then Python 3.2, the default handler for .py files is set to Python 3.2, and changing it to 2.7 is difficult because of a quirk in Eexplorer that forces you to choose between two non-distinguishable "python.exe"s. This is made much more difficult if in fact you installed five or so different Python versions. Also any automated tests using something like Cram that use python3 will not work, and any batch scripts that use python.exe will work differently depending on the host system. (It wouldn't be awful to get python-X.Y.exe executables, either). The downside of this is that any code that tries to use C:\Python3Y\python.exe breaks. Such code is probably broken anyway, there are multiple Ys around, and Python can be installed in My Documents or wherever. PEP 397 should relieve the issues with opening .py files, making some of this unnecessary with that change, as well. I'm guessing that it would also be appropriate to rename pythonw.exe to python3w.exe. I doubt that particular change matters at all, it's solely to do with opening .pyw files, and that should be handled by PEP 397. I'd appreciate any thoughts or comments you might have. Thanks for your time, Devin Jeanpierre

6 6

Suggestion for Style Guide for Python Code PEP 8
by Rahul Amaram 15 May '11

15 May '11

Hi, I was wondering if the following programming recommendation would be added to the Style Guide for Python Code (PEP 8) page. The preferred way for checking if a key (k) exists in a dictionary (d) is "if k in d". This is faster than "if k in d.keys()" and this has superseded "d.has_key(k)" Cheers, Rahul.

3 4

a few decorator recipes
by Mathias Panzenböck 14 May '11

14 May '11

A few decorator recipes that might be worthwhile to add to functools: A way to assign annotations to functions/classes: def annotations(**annots): def deco(obj): if hasattr(obj,'__annotations__'): obj.__annotations__.update(annots) else: obj.__annotations__ = annots return obj return deco _NONE = object() def getannot(obj, key, default=_NONE): if hasattr(obj, '__annotations__'): if default is _NONE: return obj.__annotations__[key] else: return obj.__annotations__.get(key, default) elif default is _NONE: raise KeyError(key) else: return default def setannot(obj, key, value): if hasattr(obj, '__annotations__'): obj.__annotations__[key] = value else: obj.__annotations__ = {key: value} Usage: >>> @annotations(foo='bar',egg='spam') >>> def foo(): >>> pass >>> >>> getannot(foo, 'egg') 'spam' A way to assign values to classes/functions (not of much use for classes, of course): def assign(**values): def deco(obj): for key in values: setattr(obj, key, values[key]) return obj return deco Usage: >>> @assign(bla='bleh',x=12) >>> def foo(): >>> pass >>> >>> foo.x 12

2 2

triple-quoted strings and indendation
by Matthias Lehmann 14 May '11

14 May '11

Hi all, two times in one day I read about the problems of triple-quoted strings and indendation (one time on stackoverflow, one time one this list). Python is well known for its readability and its use of idendation to this end. But with triple-quoted strings, nice indendation is not possible without the need to post-process the resulting string. Problem ======= Most often, the desired result of >>> some_string = """Hello ... World.""" is simply Hello World instead of Hello World. Idea ===== What about the idea, to use a string-flag to indicate, that the triple-quoted string is to be trimmed. Like: >>> some_string = t"""Hello ... World.""" This would blend in with the 'u' and 'r' flags that already exist. The triple-quoted string is trimmed to remove all whitespace up to the column where the first line of the string started OR all common whitespace of the subsequent lines, if the subsequent lines start on a column before the first line. The second rule makes it possible to also write: >>> some_string = t"""Hello ... World.""" Pros ===== The advantages above textwrap.dedent are: 1) textwrap.dedent only removes whitespace common to ALL lines, so to achieve the desired result, one has to add an additional newline >>> some_string = """ ... Hello ... World.""" >>> result = textwrap.dedent(some_string)[1:] 2) Also, it does not work, if one actually does want some common whitespace before all lines: >>> some_string = """ ... Hello ... Wold.""" >>> result = textwrap.dedent(some_string)[1:] gives again Hello World which is not, what I wanted. But >>> some_string = t""" Hello ... World.""" would give Hello World. 3) And finally to quote a post from earlier today "I know about textwrap.dedent, but having to use a Python function call to code a literal has always made me uncomfortable." Problems ========= Common indendation style for triple-quoted string (as far as I know) is >>> foo = """blubber ... bla""" (align to first quote-char) but with this auto-trimming, it would look better to use >>> foo = t"""blubber ... bar""" (align to first char after triple quotes) The other stlye would still work, though - as long as one does not want to preserve leading whitespace. Maybe the t flag could also cause a leading and trailing newline to be removed, so that >>> foo = t""" ... Hallo ... World. ... """ would also result in Hello World. Maybe something like this has been proposed before - please be kind, if it is an old hat. Mat <javascript:void(0);>

8 19

Re: [Python-ideas] Suggestion for Style Guide for Python Code PEP 8
by Weeble 13 May '11

13 May '11

On 12.05.2011 15:44, Rahul Amaram wrote: > The preferred way for checking if a key (k) exists in a dictionary (d) > is "if k in d". This is faster than "if k in d.keys()" and this has > superseded "d.has_key(k)" While 'k in d' is the right way to do it, I feel the claim it's faster that 'k in d.keys()' is somewhat weak. While this is technically true, it's a constant overhead, not some cost linear in the size of the collection - at least in Python 3 - because .keys() returns a view. >>> timeit("'1234567' in d", "d=dict((str(x),x) for x in range(5000000))") 0.09317641210044993 >>> timeit("'1234567' in d.keys()", "d=dict((str(x),x) for x in range(5000000))") 0.1938305479460105 >>> timeit("'1234567' in dkeys", "dkeys=dict((str(x),x) for x in range(5000000)).keys()") 0.0903750153983367 So "x in d.keys()" is slower than "x in d", but only by the cost of a method lookup. I don't see any reason ever to recommend using "x in d.keys()", but I think it's misleading to say that this is because of performance reasons, assuming that we are talking about Python 3. (I also completely agree with everything Georg said, FWIW.)

1 0

Allow 'import star' with namespaces
by Brendan Moloney 11 May '11

11 May '11

Hello, I posted this on python-dev, but was told that this is the more appropriate list. Currently if I do: $ import pkg Then all of the public subpackages/submodules are not automatically pulled into the 'pkg' namespace. I can do: $ from pkg import * To get all of the public subpackages/submodules, but that dumps them all into the current namespace. Why not allow: $ import pkg.* This would allow easier interactive use (by eliminating the need to import individual subpackages/submodules) while keeping the 'pkg' namespace around. Thanks, Brendan Moloney

10 20

EuroPython: Early Bird will end in 2 days!
by Palla 11 May '11

11 May '11

Hi all, If you plan to attend, you could save quite a bit on registration fees! Buy your ticket now! http://ep2011.europython.eu/registration/ The end of Early bird is on May 12th, Friday, 23:59:59 CEST. We'd like to ask to you to forward this post to anyone that you feel may be interested. We have an amazing lineup of tutorials, events and talks. We have some excellent keynote speakers and a very complete partner program... but early bird registration ends in 2 days! Right now, you still get discounts on talks and tutorials so if you plan to attend Register Now: http://ep2011.europython.eu/registration/ While you are booking, remember to have a look at the partner program and our offer for a prepaid, data+voice+tethering SIM. All the best, -- ->PALLA

1 0

1_000_000
by Matt Chaput 08 May '11

08 May '11

Not sure if this has been proposed before: A syntax change to allow underscores as thousands separators in literal numbers to improve readability, e.g.: for i in range(1, 1_000_000): pass I believe D allows this and while it's a small thing it really is much more readable. Worth a PEP? Thanks, Matt

20 41