Mailman 3 April 2011 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

i18n and Python tracebacks
by Andre Roberge Oct. 28, 2012

Oct. 28, 2012

I think it would be a good idea if Python tracebacks could be translated into languages other than English - and it would set a good example. For example, using French as my default local language, instead of >>> 1/0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: integer division or modulo by zero I might get something like >>> 1/0 Suivi d'erreur (appel le plus récent en dernier) : Fichier "<stdin>", à la … [View More]

9 15

Cofunctions PEP - Revision 4
by Greg Ewing Oct. 13, 2012

Oct. 13, 2012

Here's an updated version of the PEP reflecting my recent suggestions on how to eliminate 'codef'. PEP: XXX Title: Cofunctions Version: $Revision$ Last-Modified: $Date$ Author: Gregory Ewing <greg.ewing(a)canterbury.ac.nz> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Feb-2009 Python-Version: 3.x Post-History: Abstract ======== A syntax is proposed for defining and calling a special type of generator called a 'cofunction'. It is designed to provide a … [View More]streamlined way of writing generator-based coroutines, and allow the early detection of certain kinds of error that are easily made when writing such code, which otherwise tend to cause hard-to-diagnose symptoms. This proposal builds on the 'yield from' mechanism described in PEP 380, and describes some of the semantics of cofunctions in terms of it. However, it would be possible to define and implement cofunctions independently of PEP 380 if so desired. Specification ============= Cofunction definitions ---------------------- A cofunction is a special kind of generator, distinguished by the presence of the keyword ``cocall`` (defined below) at least once in its body. It may also contain ``yield`` and/or ``yield from`` expressions, which behave as they do in other generators. From the outside, the distinguishing feature of a cofunction is that it cannot be called the same way as an ordinary function. An exception is raised if an ordinary call to a cofunction is attempted. Cocalls ------- Calls from one cofunction to another are made by marking the call with a new keyword ``cocall``. The expression :: cocall f(*args, **kwds) is evaluated by first checking whether the object ``f`` implements a ``__cocall__`` method. If it does, the cocall expression is equivalent to :: yield from f.__cocall__(*args, **kwds) except that the object returned by __cocall__ is expected to be an iterator, so the step of calling iter() on it is skipped. If ``f`` does not have a ``__cocall__`` method, or the ``__cocall__`` method returns ``NotImplemented``, then the cocall expression is treated as an ordinary call, and the ``__call__`` method of ``f`` is invoked. Objects which implement __cocall__ are expected to return an object obeying the iterator protocol. Cofunctions respond to __cocall__ the same way as ordinary generator functions respond to __call__, i.e. by returning a generator-iterator. Certain objects that wrap other callable objects, notably bound methods, will be given __cocall__ implementations that delegate to the underlying object. Grammar ------- The full syntax of a cocall expression is described by the following grammar lines: :: atom: cocall | <existing alternatives for atom> cocall: 'cocall' atom cotrailer* '(' [arglist] ')' cotrailer: '[' subscriptlist ']' | '.' NAME Note that this syntax allows cocalls to methods and elements of sequences or mappings to be expressed naturally. For example, the following are valid: :: y = cocall self.foo(x) y = cocall funcdict[key](x) y = cocall a.b.c[i].d(x) Also note that the final calling parentheses are mandatory, so that for example the following is invalid syntax: :: y = cocall f # INVALID New builtins, attributes and C API functions -------------------------------------------- To facilitate interfacing cofunctions with non-coroutine code, there will be a built-in function ``costart`` whose definition is equivalent to :: def costart(obj, *args, **kwds): try: m = obj.__cocall__ except AttributeError: result = NotImplemented else: result = m(*args, **kwds) if result is NotImplemented: raise TypeError("Object does not support cocall") return result There will also be a corresponding C API function :: PyObject *PyObject_CoCall(PyObject *obj, PyObject *args, PyObject *kwds) It is left unspecified for now whether a cofunction is a distinct type of object or, like a generator function, is simply a specially-marked function instance. If the latter, a read-only boolean attribute ``__iscofunction__`` should be provided to allow testing whether a given function object is a cofunction. Motivation and Rationale ======================== The ``yield from`` syntax is reasonably self-explanatory when used for the purpose of delegating part of the work of a generator to another function. It can also be used to good effect in the implementation of generator-based coroutines, but it reads somewhat awkwardly when used for that purpose, and tends to obscure the true intent of the code. Furthermore, using generators as coroutines is somewhat error-prone. If one forgets to use ``yield from`` when it should have been used, or uses it when it shouldn't have, the symptoms that result can be extremely obscure and confusing. Finally, sometimes there is a need for a function to be a coroutine even though it does not yield anything, and in these cases it is necessary to resort to kludges such as ``if 0: yield`` to force it to be a generator. The ``cocall`` construct address the first issue by making the syntax directly reflect the intent, that is, that the function being called forms part of a coroutine. The second issue is addressed by making it impossible to mix coroutine and non-coroutine code in ways that don't make sense. If the rules are violated, an exception is raised that points out exactly what and where the problem is. Lastly, the need for dummy yields is eliminated by making it possible for a cofunction to call both cofunctions and ordinary functions with the same syntax, so that an ordinary function can be used in place of a cofunction that yields zero times. Record of Discussion ==================== An earlier version of this proposal required a special keyword ``codef`` to be used in place of ``def`` when defining a cofunction, and disallowed calling an ordinary function using ``cocall``. However, it became evident that these features were not necessary, and the ``codef`` keyword was dropped in the interests of minimising the number of new keywords required. The use of a decorator instead of ``codef`` was also suggested, but the current proposal makes this unnecessary as well. It has been questioned whether some combination of decorators and functions could be used instead of a dedicated ``cocall`` syntax. While this might be possible, to achieve equivalent error-detecting power it would be necessary to write cofunction calls as something like :: yield from cocall(f)(args) making them even more verbose and inelegant than an unadorned ``yield from``. It is also not clear whether it is possible to achieve all of the benefits of the cocall syntax using this kind of approach. Prototype Implementation ======================== An implementation of an earlier version of this proposal in the form of patches to Python 3.1.2 can be found here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cofunctions.h… If this version of the proposal is received favourably, the implementation will be updated to match. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

11 29

PEP-3151 pattern-matching
by Devin Jeanpierre May 19, 2011

May 19, 2011

Hello, PEP-3151 < http://www.python.org/dev/peps/pep-3151/ > mentions a really weird syntax for pattern-matching. I was hoping I could suggest an alternative that's both more concise, and possible to implement without doing something drastic like changing existing syntax or semantics. The PEP offers the pattern-matching syntax: > except IOError as e if e.errno == errno.ENOENT: ... I'd instead suggest something along the lines of > except io_error(errno.ENOENT): ... … [View More]

15 44

a few decorator recipes
by Mathias Panzenböck May 14, 2011

May 14, 2011

A few decorator recipes that might be worthwhile to add to functools: A way to assign annotations to functions/classes: def annotations(**annots): def deco(obj): if hasattr(obj,'__annotations__'): obj.__annotations__.update(annots) else: obj.__annotations__ = annots return obj return deco _NONE = object() def getannot(obj, key, default=_NONE): if hasattr(obj, '__annotations__'): if default is _NONE: return obj.__annotations__[key] else: return obj.__annotations__.get(… [View More]

2 2

Disallow orderring comparison to NaN
by Alexander Belopolsky April 30, 2011

April 30, 2011

Another spin-off from the "[Python-Dev] PyObject_RichCompareBool identity shortcut" thread: > I would like to discuss another peculiarity of NaNs: > >>>> float('nan') < 0 > False >>>> float('nan') > 0 > False > > This property in my experience causes much more trouble than nan == > nan being false. The problem is that common sorting or binary search > algorithms may degenerate into infinite loops in the presence of nans. > … [View More]

12 36

Proposal for new-style decorators
by Christophe Schlick April 29, 2011

April 29, 2011

Hello everybody, My name is Christophe Schlick, from the University of Bordeaux, France. I've been using Python for years, but it happens that this is my first post on a Python-related mailing list. For the first time, I feel that I may have some topic interesting enough for the Python community, so I would be very happy to get any kind of feeback (positive or negative) on it. Thanks in advance... The goal of this post is to propose a new syntax for defining decorators in Python. I would like … [View More]to fully explain the rationale and the benefits of this proposal, as I guess that the discussion might be more fruitful if all details are written down once for all. However this has led me to a very long post (a kind of informal PEP) that was blocked by the moderators. So, following their recommendation, I've divided the text in three successive parts: (1) the rationale, (2) the proposal, (3) the proposed implementation and additional questions/remarks. For better clarity of the description, I will call the proposed syntax as "new-style decorators" (NSD, for short) and rename the classic syntax as "old-style decorators" (OSD), following the terms used some years ago with the (re)definition of classes. By the way, the introducing process for NSD shares many aspects with the process used for introducing new-style classes, including the following features: * No existing syntax is broken: the only thing required to create a new-style decorator is to decorate itself by a newly-introduced decorator called... "decorator" (well, this sentence is less recursive than it might appear at the first reading). * Every thing that can be done with OSD is possible with NSD, but NSD offer additional more user-friendly features. * NSD can peacefully live together with OSD in the same code. An NSD may even decorate an OSD (and vice-versa), however some properties of the NSD are lost with such a combination. -------------------------------------------------- 1 - Why bother with a new syntax? To explain what I don't like with the current syntax of decorators, let me take the example of a basic decorator (called 'old_style_repeat_fix') that simply repeats 3 times its undecorated function, and adds some tracing to the standard output. Here is the code: #--- def old_style_repeat_fix(func): """docstring for decorating function""" # @wraps(func) def dummy_func_name_never_used(*args, **keys): """docstring for decorated function""" print "apply 'old_style_repeat_fix' on %r" % func.__name__ for loop in range(3): func(*args, **keys) return dummy_func_name_never_used #--- Even if such code snippets have become quite usual since the introduction of decorators in Python 2.2, many people have argued (and I am obviously one of them) that the decorator syntax is a bit cumbersome. First, it imposes the use of nested functions, which often reduces readability by moving the function signature and docstring too far from the corresponding code. Second, as anonymous lambdas expressions can usually not be employed for decorating functions, the programmer has no other choice than to create a dummy function name (only used for one single 'return' statement), which is never a good coding principle, whatever the programming language. Once you have tried to teach decorators to a bunch of students, you really understand how much this syntax leverages the difficulty to grab the idea. The situation is even worse when the decorator needs some arguments: let's create an extended decorator (called 'old_style_repeat_var) that includes an integer 'n' to control the number of iterations, and a boolean 'trace' to control the tracing behavior. Here is the code: #--- def old_style_repeat_var(n=3, trace=True): """docstring for decorating function""" def dummy_deco_name_never_used(func): """docstring never used""" # @wraps(func) def dummy_func_name_never_used(*args, **keys): """docstring for decorated function""" if trace: print "apply 'old_style_repeat_var' on %r" % func.__name__ for loop in range(n): func(*args, **keys) return dummy_func_name_never_used return dummy_deco_name_never_used #--- This time a two-level function nesting is required and the code needs two dummy names for these two nested functions. Note that the docstring of the middle nested function is even totally invisible for introspection tools. So whether you like nested functions or not, there is some evidence here that the current syntax is somehow suboptimal. Another drawback of OSD is that they do not gently collaborate with introspection and documentation tools. For instance, let's apply our decorator on a silly 'test' function: #--- @old_style_repeat_var(n=5) # 'trace' keeps its default value def test(first=0, last=0): """docstring for undecorated function""" print "test: first=%s last=%s" % (first, last) #--- Now, if we try 'help' on it, we get the following answer: #--- >>> help(test) dummy_func_name_never_used(*args, **keys) docstring for decorated function #--- which means that neither the name, nor the docstring, nor the signature of the 'test' function are correct. Things are a little better when using the 'wraps' function from the standard 'functools' module (simply uncomment the line '@wraps(func)' in the code of 'old_style_repeat_var'): #--- >>> help(test) test(*args, **keys) """docstring for undecorated function""" #--- '@wraps(func)' copies the name and the docstring from the undecorated function to the decorated one, in order to get some useful piece of information when using 'help'. However, the signature of the function still comes from the decorated function, not the genuine one. The reason is that signature copying is not an easy process. The only solution is to inspect the undecorated function and then use 'exec' to generate a wrapper with a correct signature. This is basically what is done in the 'decorator' package (available at PyPI) written by Michele Simionato. There has been a lengthy discussion in python-dev (in 2009 I guess, but I can't find the archive right now) whether to include or not this package in the standard library. As far as I know, there is currently no clear consensus whether this is a good idea or not, because there has always been a mixed feeling from the community about transparent copy from the undecorated to the decorated function (even about the 'wraps' function): on one hand, transparent copy is cool for immediate help, for automatic documentation and for introspection tools, but on the other hand, it totally hides the decorating process which is not always what is wanted... or needed. The syntax for NSD presented in this proposal tries to improve this situation by offering two desirable features, according to the Zen of Python: * "flat is better than nested": no nested functions with dummy names are required, even when parameters are passed to the decorator; only one single decorating function has to be written by the programmer, whatever the kind of decorator. * "explicit is better than implicit": introspection of a decorated function explicitely reveals the decoration process, and allows one to get the name/signature/docstring not only for the corresponding undecorated function, but also for any number of chained decorators that have been applied on it. ------ to be continued in Part 2... CS [View Less]

7 15

[[Python-Dev] PyObject_RichCompareBool identity shortcut] About specializing comparing objects in collections
by ZS April 29, 2011

April 29, 2011

Returning to the original topic post on the comparison of objects in containers ... Could be asked to enter a special method __equals__ of objects for comparison in containers (by default, if it is not defined - use usual method for compatibility), just as is done in C# (it's special object's method Equals is used to compare items in the collection). ----- Zaur

2 1

[[Python-Dev] PyObject_RichCompareBool identity shortcut] About specializing comparing objects in collections
by ZS April 29, 2011

April 29, 2011

It was necessary to clarify the reason for the possible introduction of a special method __equals__. The fact that the objects in python are often interpreted as values. Method __eq__, IMHO is used more for comparing objects as values. Introduction of method __equals__ can explicitly indicate the method of comparison of the objects as objects and __eq__ objects as values. If method __equals__ is not defined then __eq__ will be used. -- Zaur

1 0

Make all keywords legal as an identifier
by haael April 29, 2011

April 29, 2011

Hello, guys. I did post this idea a few months ago. Now the revised version. Goal: Let _all_ alphanumeric keywords be legal as names for variables, functions and classes, even the ones that are reserved words now. Rationale: 1. Python took most good English words as reserved tokens. Situation goes worse from version to version. I often have hard time searching for acceptable synonyms. 2. Because of that, old Python programs cease to work, even if they do not use any abandoned features. … [View More]

20 32

Re: [Python-ideas] [Python-Dev] the role of assert in the standard library ?
by Tarek Ziadé April 28, 2011

April 28, 2011

This is a thread from python-dev I am moving to python-ideas, because I want to debate the "assert" keyword :) On Thu, Apr 28, 2011 at 12:27 PM, Michael Foord <fuzzyman(a)voidspace.org.uk> wrote: > On 28/04/2011 09:34, Terry Reedy wrote: >> >> On 4/28/2011 3:54 AM, Tarek Ziadé wrote: >>> >>> Hello >>> >>> I removed some assert calls in distutils some time ago because the >>> package was not behaving correctly when people were … [View More]using Python with >>> the --optimize flag. In other words, assert became a full part of the >>> code logic and removing them via -O was changing the behavior. >>> >>> In my opinion assert should be avoided completely anywhere else than >>> in the tests. If this is a wrong statement, please let me know why :) >> >> My understanding is that assert can be used in production code but only to >> catch logic errors by testing supposed invariants or postconditions. It >> should not be used to test usage errors, including preconditions. In other >> words, assert presence or absence should not affect behavior unless the code >> has a bug. But it does affect the behaviour at the end: when the code has a bug, then the way the code works is affected and differs depending if -O is used. Let me take an example: bucket = [] def add(stuff): bucket.append(stuff) def purge_and_do_something(): bucket.clear() assert len(bucket) == 0 ... do something by being sure the bucket is empty ... here, we could say assert is legitimate, as it just checks a post-condition. But this code is not thread-safe and if it's run via several threads, some code could add something in the bucket while purge() check for the assertion. So, if I run this code using -O it will not behave the same: it will seem to work. I am not arguing against the fact that this code should be changed and set up a lock. My point is that I do not understand why there are assert calls to check post-conditions. Those seem to be relics from the developer that marked something she needed to take care of in her code, but did not yet. e.g. like a TODO or a XXX or a NotImplementedError Moreover, why -O and -OO are removing assertions in that case ? even if the assert is "supposed not to happen" it's present in the code and can happen (or well, remove it). So "optimize" potentially changes the behaviour of the code. >From what I understood so far, a line with an assert should be considered as a dead code if we want the code to behave always the same way. I remove dead code in my code... Cheers Tarek -- Tarek Ziadé | http://ziade.org [View Less]

4 3

Equality of same NaN instances?
by ZS April 28, 2011

April 28, 2011

Last debate on python-dev about behavior NaNs in containers raised another important question: Should the instance of NaN be equal to itself? P.S. Of course I suppose that different instances of NaN do not have to be equal. --- Zaur

4 3

Re: [Python-ideas] Proposal for new-style decorators
by Christophe Schlick April 27, 2011

April 27, 2011

On Tue, Apr 26, 2011 at 11:20 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > > There is no special syntax for defining decorators -- just normal nested > function or class definition syntax. To put it another way, Python does not > have decorator objects. A decorator is simply a callable (function, class, > or class instance with __call__ method) applied to a object I totally agree with that. When defining a decorating function, you don't have any syntactic element that … [View More]could explain the reader of your code that this function is actually a decorator. It is only when applied on a function with the @-syntax that the mechanism becomes visible (but this is not always done in the same file). This can be considered as a strength (any function with a correct input/output is likely to later become a decorator, even if the original author did not thought about it). However, according to my own little experience, more than 9 times out of 10, you perfectly now when writting such a function that it is actually a decorator. Using the proposed @decorator idiom (hey see, I haven't written "syntax" ;-) has at least the advantage to be explicit when you want to be explicit (besides the other features it provides), > What you are actually proposing is a meta-decorator (class) whose instances > can be used as decorators because the class has a __call__ instance method. > This sort of thing is a known alternative to the nested function pattern. Yes, I know that using callable class can be an alternative to the nested function pattern. In the pattern you talk about, the __init__ method gets the decorator arguments, the __call__ method serves as a decorator making function, and a third method is used to generate the actual decorating function. As a result, the boilerplate is approximatively the same as with the nested functions idiom. But this is not what is done here, because the end-user only writes a single function, not a whole class. In my proposal, the two nested functions are avoided by the fact that (1) the decorator attributes are automatically injected as meta-attributes (this is the role of the middle nested function in the standard idiom), and (2) the decorating function is in charge to pass the whole set of arguments to the undecorated function (this is the role of the inner nested function in the standard idiom). As far as I know, I haven't seen the combination of these two elements before. > Stick with the two real problems. > > 1. The double or triple nested function pattern has a lot of boilerplate and > can be difficult to learn. Hiding boilerplate in a class makes the use > pattern simpler and easier to learn. This is a real benefit. > 2. Introspection (more comments below), which your class also addresses. OK. I'll drop the arguments on nested function, and simply focus on boilerplate and introspection. That makes sense. >> def old_style_repeat_var(n=3, trace=True): >> """docstring for decorating function""" > > Actually, this is the docstring for the decorator making function. Also agree. I've written "decorating function" by symmetry with the new idiom, but I knew that I would get some remark here ;-) >> '@wraps(func)' copies the name and the docstring from the undecorated >> function to the decorated one, in order to get some useful piece of >> information when using 'help'. However, the signature of the function >> still comes from the decorated function, not the genuine one. > > I am not sure what you mean. If the two signatures are different, then one > must use the signature of the wrapper when calling it, not the signature of > the wrappee, which is perhaps what you mean by 'the genuine > one'. The problem of generic wrappers having generic signatures (*args, > **kwds) is endemic to using generic wrappers instead of special case > wrappers. What I wanted to say is that wraps only does half of the job: it correctly copies the name and the docstring, but the signature presented by the help function is still test(*args, **keys), while it should actually be test(first=0, last=0) according to the undecorated function. The alternative proposed by the new idiom is to copy nothing at all: it simply says "OK, 'test' is a decorated function. If you want to know more look at 'test.func' to get the info about the undecorated one, and at 'test.deco' to see what the decorator has done". I prefer such a raw-but-explicit approach rather than an automatic half-baked, half-bogus one. Moreover, it is not easy with the 'functools' module to provide introspection of the decorating function, once you've got the decorated one. >> The only >> solution is to inspect the undecorated function and then use 'exec' to >> generate a wrapper with a correct signature. This is basically what is >> done in the 'decorator' package (available at PyPI) written by Michele >> Simionato. There has been a lengthy discussion in python-dev (in 2009 >> I guess, but I can't find the archive right now) whether to include or >> not this package in the standard library. > > The other solution is to not use a generic wrappers with generic signatures > but to write specific wrappers with the actual signature, which people did, > for instance, before functools and partial() were added to Python. Right again, but this overweights the boilerplate even more compared to the '@wraps' decorator, no? > I second the other recommendations to make your proposal > available on the cookbook site, etc. That sounds good... Thanks a lot, Terry! [View Less]

3 6

Proposal for new-style decorators
by Christophe Schlick April 26, 2011

April 26, 2011

Hi Jim, The new version. Now cut-and-paste code should work... ------ Hello everybody, My name is Christophe Schlick, from the University of Bordeaux, France. I've been using Python for years, but it happens that this is my first post on a Python-related mailing list. For the first time, I feel that I may have some topic interesting enough for the Python community, so I would be very happy to get any kind of feeback (positive or negative) on it. Thanks in advance... The goal of this post is … [View More]to propose a new syntax for defining decorators in Python. I would like to fully explain the rationale and the benefits of this proposal, as I guess that the discussion might be more fruitful if all details are written down once for all. However this has led me to a very long post (a kind of informal PEP) that was blocked by the moderators. So, following their recommendation, I've divided the text in three successive parts: (1) the rationale, (2) the proposal, (3) the proposed implementation and additional questions/remarks. For better clarity of the description, I will call the proposed syntax as "new-style decorators" (NSD, for short) and rename the classic syntax as "old-style decorators" (OSD), following the terms used some years ago with the (re)definition of classes. By the way, the introducing process for NSD shares many aspects with the process used for introducing new-style classes, including the following features: * No existing syntax is broken: the only thing required to create a new-style decorator is to decorate itself by a newly-introduced decorator called... "decorator" (well, this sentence is less recursive than it might appear at the first reading). * Every thing that can be done with OSD is possible with NSD, but NSD offer additional more user-friendly features. * NSD can peacefully live together with OSD in the same code. An NSD may even decorate an OSD (and vice-versa), however some properties of the NSD are lost with such a combination. -------------------------------------------------- 1 - Why bother with a new syntax? To explain what I don't like with the current syntax of decorators, let me take the example of a basic decorator (called 'old_style_repeat_fix') that simply repeats 3 times its undecorated function, and adds some tracing to the standard output. Here is the code: #--- def old_style_repeat_fix(func): """docstring for decorating function""" # @wraps(func) def dummy_func_name_never_used(*args, **keys): """docstring for decorated function""" print "apply 'old_style_repeat_fix' on %r" % func.__name__ for loop in range(3): func(*args, **keys) return dummy_func_name_never_used #--- Even if such code snippets have become quite usual since the introduction of decorators in Python 2.2, many people have argued (and I am obviously one of them) that the decorator syntax is a bit cumbersome. First, it imposes the use of nested functions, which often reduces readability by moving the function signature and docstring too far from the corresponding code. Second, as anonymous lambdas expressions can usually not be employed for decorating functions, the programmer has no other choice than to create a dummy function name (only used for one single 'return' statement), which is never a good coding principle, whatever the programming language. Once you have tried to teach decorators to a bunch of students, you really understand how much this syntax leverages the difficulty to grab the idea. The situation is even worse when the decorator needs some arguments: let's create an extended decorator (called 'old_style_repeat_var) that includes an integer 'n' to control the number of iterations, and a boolean 'trace' to control the tracing behavior. Here is the code: #--- def old_style_repeat_var(n=3, trace=True): """docstring for decorating function""" def dummy_deco_name_never_used(func): """docstring never used""" # @wraps(func) def dummy_func_name_never_used(*args, **keys): """docstring for decorated function""" if trace: print "apply 'old_style_repeat_var' on %r" % func.__name__ for loop in range(n): func(*args, **keys) return dummy_func_name_never_used return dummy_deco_name_never_used #--- This time a two-level function nesting is required and the code needs two dummy names for these two nested functions. Note that the docstring of the middle nested function is even totally invisible for introspection tools. So whether you like nested functions or not, there is some evidence here that the current syntax is somehow suboptimal. Another drawback of OSD is that they do not gently collaborate with introspection and documentation tools. For instance, let's apply our decorator on a silly 'test' function: #--- @old_style_repeat_var(n=5) # 'trace' keeps its default value def test(first=0, last=0): """docstring for undecorated function""" print "test: first=%s last=%s" % (first, last) #--- Now, if we try 'help' on it, we get the following answer: #--- >>> help(test) dummy_func_name_never_used(*args, **keys) docstring for decorated function #--- which means that neither the name, nor the docstring, nor the signature of the 'test' function are correct. Things are a little better when using the 'wraps' function from the standard 'functools' module (simply uncomment the line '@wraps(func)' in the code of 'old_style_repeat_var'): #--- >>> help(test) test(*args, **keys) """docstring for undecorated function""" #--- '@wraps(func)' copies the name and the docstring from the undecorated function to the decorated one, in order to get some useful piece of information when using 'help'. However, the signature of the function still comes from the decorated function, not the genuine one. The reason is that signature copying is not an easy process. The only solution is to inspect the undecorated function and then use 'exec' to generate a wrapper with a correct signature. This is basically what is done in the 'decorator' package (available at PyPI) written by Michele Simionato. There has been a lengthy discussion in python-dev (in 2009 I guess, but I can't find the archive right now) whether to include or not this package in the standard library. As far as I know, there is currently no clear consensus whether this is a good idea or not, because there has always been a mixed feeling from the community about transparent copy from the undecorated to the decorated function (even about the 'wraps' function): on one hand, transparent copy is cool for immediate help, for automatic documentation and for introspection tools, but on the other hand, it totally hides the decorating process which is not always what is wanted... or needed. The syntax for NSD presented in this proposal tries to improve this situation by offering two desirable features, according to the Zen of Python: * "flat is better than nested": no nested functions with dummy names are required, even when parameters are passed to the decorator; only one single decorating function has to be written by the programmer, whatever the kind of decorator. * "explicit is better than implicit": introspection of a decorated function explicitely reveals the decoration process, and allows one to get the name/signature/docstring not only for the corresponding undecorated function, but also for any number of chained decorators that have been applied on it. ------ to be continued in Part 2... CS [View Less]

1 0

('blue', 'red', 'orange' if something, 'green')
by cool-RR April 25, 2011

April 25, 2011

Here's an idea that would have helped me today while coding. Allow something like this: ('blue', 'red', 'orange' if some_condition, 'green') So 'orange' is included in the tuple only if `some_condition` evaluates to `True`. This could be applied to list literals, tuple literals, set literals, and possibly dict literals, though the latter might be too clunky. I expect this to be rejected, but I really couldn't think of an elegant way to achieve the same thing with existing syntax. Ram.

14 21

random.boolean or bernoulli
by Erich Blume April 22, 2011

April 22, 2011

I am yet a novice python user, and not a strong statistician to boot, but I had an idea about how to enhance the 'random' module with one or two new functions for generating boolean values instead of floating-point values. This idea has a lot of flexibility in how it might be implemented, but I propose two new functions for the random module that might be implemented in python as follows def boolean(): return choice((True,False)) def bernoulli(p): return random() <= p It's true … [View More]

10 14

documentation: mention the version in which a syntax construct was added
by Mathias Panzenböck April 19, 2011

April 19, 2011

I wonder in which version the "except Foo as e" syntax was added (as opposed to "except Foo, e"). The documentations does not state this but I think it wasn't there in Python 2.4. I propose that such things are always mentioned in the documentation and formated so that you can easily find the version numbers. See how Mozilla marks such things (scroll down): https://developer.mozilla.org/en/DOM/range -panzi

5 5

Re: [Python-ideas] [Python-Dev] python and super
by Michael Foord April 18, 2011

April 18, 2011

On 16/04/2011 00:38, Greg Ewing wrote: > Michael Foord wrote: > >> consider the "recently" introduced problem caused by object.__init__ > > not taking arguments. This makes it impossible to use super correctly > > in various circumstances. > > > > ... > > >> It is impossible to inherit from both C and A and have all parent >> __init__ methods called correctly. Changing the semantics of super as >> described would fix this problem. > … [View More]

3 5

Simple class initialization
by Adam Matan April 18, 2011

April 18, 2011

0. Abstract =========== A class initialization often begins with a long list of explicit variable declaration statements at the __init__() method. This repetitively copies arguments into local data attributes. This article suggests some semi-automatic techniques to shorten and clarify this code section. Comments and responses are highly appreciated. 1. Credit ========= The idea emerged from my question at stackoverflow.com. I would like to thank all those who answered and commented on that … [View More]thread. http://stackoverflow.com/questions/1389180 2. The problem ============== Consider the following class: class Process: def __init__(self, pid, ppid, cmd, fd, reachable, user): If the instance needs to hold these arguments internally, local data attributes are declared and the value of the argument is being copied, using two mainstream notations: a. self.pid=pid self.ppid=ppid self.cmd=cmd self._fd=fd self.reachable=reachable self.user=user b. self.pid, self.ppid, self.cmd, self._fd, self.reachable, self.user = pid, ppid, cmd, fd, reachable, user a. takes an unreasonable amount of lines and has a repetative form. b. is long and prone to errors, especially when the argument list changes during development. 3. Solution outline =================== 1. Generally comply with the Zen of Python. 1. Explicit. The instance should not store any value unless told. 2. Short. 3. Backward compatible. Current Class syntax should work with the new solution. 4. Readable and intuitive. 5. Flexible. There should be a way to store any given subset of the arguments. 6. Allow storage of "private" variables by adding a single or double underscore before the variable name. 4. Solutions ============ 4.1 Decorator ------------- Nadia Alramli suggested this at the aforementiond thread at stackoverflow: from functools import wraps import inspect def initializer(fun): names, varargs, keywords, defaults = inspect.getargspec(fun) @wraps(fun) def wrapper(self, *args): for name, arg in zip(names[1:], args): setattr(self, name, arg) fun(self, *args) return wrapper class Process: @initializer def __init__(self, pid, ppid, cmd, fd, reachable, user) Pros: Simple, short, explicit and intuitive. Easy to add to the standard library, fully backward-compatible. Cons: Stores all arguments. Does not support private data attributes notation (underscore prefix). See http://stackoverflow.com/questions/1389180/python-automatically-initialize-… 4.2. Argument tagging --------------------- Arguments that needed to be stored within the instance could be marked with a special character, e.g. '~'. The character would be placed after the argument name for private variables: class Process: def __init__(self, ~pid, ~ppid, ~cmd, fd~, ~reachable, ~user) Pros: Simple, short and explicit. Can store any subset of the arguments. Supports private variable notation. Cons: Not intuitive. Changes the method signature and might be confusing. 4.3 Standard function --------------------- A function will be called to store the argument as data attributes. class Process: def __init__(self, pid, ppid, cmd, fd, reachable, user) acquire(pid, ppid, cmd, reachable, user) acquire(fd, prefix='_') Possible keywords can ba acquire, store, absorp. Pros: Explicit, clear and intuitive. Cons: Long - especially if more than a single prefix is used. 4.4 Initialization list ----------------------- The argument list would include the name of the local data attribute, a separator, and the argument name. class Process: def __init__(self, pid:pid, ppid:ppid, cmd:cmd, _fd:fd, reachable:reachable, user:user) """ pid, ppid, cmd, reachable and user are stored as data properties with the same name. fd is stored as _fd.""" Or: class Process: def __init__(self, :pid, :ppid, :cmd, _fd:fd, :reachable, :user) """Same, but local data attributes with the same name as arguments would be stored without stating their name twice.""" This is a developed argument tagging (4.2). Pros: See 4.2 Cons: Alters the method signature Not intuitive. Looking forward for comments, Adam matan [View Less]

11 14

parameter annotation semantics
by Bruce Leban April 17, 2011

April 17, 2011

Subject was: Re: [Python-ideas] Simple class initialization On Sat, Apr 16, 2011 at 6:27 PM, Steven D'Aprano <steve(a)pearwood.info>wrote: > Bruce Leban wrote: > > One >> of the strange aspects (to me) of parameter annotations is that they have >> no >> semantics which opens them up to multiple conflicting uses. >> > > That's not a bug, that's a feature. > > It's been stated many times by Guido that it's far too early to standardize > … [View More]on a single meaning for annotations. (We may *never* standardize on a single > meaning.) Instead, it is up to the library or decorator to impose whatever > meaning makes sense for that particular library or decorator. > I understand that. I can still think it a bit strange, can't I? To be more specific, you hit the crux of the problem with the statement "it is up to * THE* library or decorator to impose whatever meaning makes sense" [emphasis added] is that it assumes the singular. If I want to use two decorators which impose different meanings, I'm stuck. Imagine I have two decorators like this: @memoize # memoizes return values; if memo_value annotation is used, memoizes using # the value returned by applying the function def a(foo:memo_value(int), bar:memo_value(tuple)): pass @log_me # logs function calls and return values excluding parameters annotated no_log def b(foo, bar:no_log): pass Can I use both @memoize and @logging? Not if they're defined as they are above. I can if the decorators expect a list of annotations instead of a solo annotation: @memoize def a(foo:[memo_value(int)], bar:[memo_value(tuple)]): pass @log_me def b(foo, bar:[no_log]): pass @memoize @log_me def c(foo:[memo_value(int)], bar:[no_log, memo_value(tuple)]): pass Note that I am NOT proposing a change to the language (none is necessary), just suggesting that this would be good advice to offer authors. PEP 8 offers no advice on annotation syntax and PEP 3107 explicitly does not offer any advice saying "Despite yet more discussion, it was decided not to standardize a mechanism for annotation interoperability." What I would propose is something more like this: There is no standard for annotation interoperability. Annotations that wish to support interoperability should consider treating the annotation as an enumerable of values and checking value in annotation (or something similar) instead of value == annotation. Of course an obvious alternative is @memoize @log_me def c(foo:{memoize:memo_value(int)}, bar:{log_me:no_log, memoize:memo_value(tuple)}): pass but I think that's too verbose and therefore less likely to be adopted. --- Bruce *New! *Puzzazz newsletter: http://j.mp/puzzazz-news-2011-04 including April Fools! *New!** *Blog post: http://www.vroospeak.com Ironically, a glaring Google grammatical error [View Less]

6 5

Re: [Python-ideas] [Python-Dev] python and super
by Carl Meyer April 16, 2011

April 16, 2011

(Moving this thread to python-ideas, as it seems better suited for here). On 04/15/2011 11:02 AM, Michael Foord wrote: > Well yes, but it is also a bug in the copy of unittest2 embedded in > django - so whilst it can be fixed in unittest2 (simply deleting the > setUp and tearDown methods which do nothing but override > unittest.TestCase.setUp and tearDown) it *also* needs to be fixed in > django. Yup. > This particular issue does illustrate the problem well though - the … [View More]

3 3

Copy-on-write when forking a python process
by jac April 16, 2011

April 16, 2011

Hi all, Sorry for cross posting, but I think that this group may actually be more appropriate for this discussion. Previous thread is at: http://groups.google.com/group/comp.lang.python/browse_thread/thread/1df510… I am wondering if anything can be done about the COW (copy-on-write) problem when forking a python process. I have found several discussions of this problem, but I have seen no proposed solutions or workarounds. My understanding of the problem is that an object's reference count … [View More]

9 19

PEP-3150
by Carl M. Johnson April 15, 2011

April 15, 2011

Can we start a new thread just for discussion of PEP-3150? I personally don't like any of the proposals to add assignment to list comprehensions/generator expressions and I think that no matter how it's spelt the idea is a misfeature since there's no need to cram more info into a one-liner list comp/gen ex, but I do find the given-clause very interesting and potentially useful. In terms of positives for the feature, assignment in a list comp/gen ex has the potential to save a few characters and … [View More]

3 2

Re: [Python-ideas] Assignments in list/generator expressions
by Laura Creighton April 15, 2011

April 15, 2011

In a message of Sun, 10 Apr 2011 20:18:35 EDT, Eugene Toder writes: <snip> >This particular feature is not the top on my wish list either. I just >don't see a good explanation for why it's missing. It's a lesser-known >feature in Haskell (no mention in Wikipedia! :-), but it is handy at >times. As mentioned above, people suggest it for Python from time to >time. <snip> The problem, as I see it is that you see it as ``missing''. You have some internal model of … [View More]

15 43

Re: [Python-ideas] PEP-3150
by Carl M. Johnson April 15, 2011

April 15, 2011

On Wed, Apr 13, 2011 at 3:48 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Thu, Apr 14, 2011 at 8:33 AM, Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: >> assigned to the __doc__ attribute of whatever object the >> RHS expression returns. > > Ah, true, that would make a lot of sense. It also generalises nicely > to other cases like "return" and "yield" as well. I'll see if I can > thrash out something sensible along those lines. So, in the case … [View More]

5 9