I think it would be a good idea if Python tracebacks could be translated
into languages other than English - and it would set a good example.
For example, using French as my default local language, instead of
>>> 1/0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero
I might get something like
>>> 1/0
Suivi d'erreur (appel le plus récent en dernier) :
Fichier "<stdin>", à la ligne 1, dans <module>
ZeroDivisionError: division entière ou modulo par zéro
André
Greg Ewing wrote:
> Mark Shannon wrote:
>
>> Why not have proper co-routines, instead of hacked-up generators?
>
> What do you mean by a "proper coroutine"?
>
A parallel, non-concurrent, thread of execution.
It should be able to transfer control from arbitrary places in
execution, not within generators.
Stackless provides coroutines. Greenlets are also coroutines (I think).
Lua has them, and is implemented in ANSI C, so it can be done portably.
See: http://www.jucs.org/jucs_10_7/coroutines_in_lua/de_moura_a_l.pdf
(One of the examples in the paper uses coroutines to implement
generators, which is obviously not required in Python :) )
Cheers,
Mark.
Here's an updated version of the PEP reflecting my
recent suggestions on how to eliminate 'codef'.
PEP: XXX
Title: Cofunctions
Version: $Revision$
Last-Modified: $Date$
Author: Gregory Ewing <greg.ewing(a)canterbury.ac.nz>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 13-Feb-2009
Python-Version: 3.x
Post-History:
Abstract
========
A syntax is proposed for defining and calling a special type of generator
called a 'cofunction'. It is designed to provide a streamlined way of
writing generator-based coroutines, and allow the early detection of
certain kinds of error that are easily made when writing such code, which
otherwise tend to cause hard-to-diagnose symptoms.
This proposal builds on the 'yield from' mechanism described in PEP 380,
and describes some of the semantics of cofunctions in terms of it. However,
it would be possible to define and implement cofunctions independently of
PEP 380 if so desired.
Specification
=============
Cofunction definitions
----------------------
A cofunction is a special kind of generator, distinguished by the presence
of the keyword ``cocall`` (defined below) at least once in its body. It may
also contain ``yield`` and/or ``yield from`` expressions, which behave as
they do in other generators.
From the outside, the distinguishing feature of a cofunction is that it cannot
be called the same way as an ordinary function. An exception is raised if an
ordinary call to a cofunction is attempted.
Cocalls
-------
Calls from one cofunction to another are made by marking the call with
a new keyword ``cocall``. The expression
::
cocall f(*args, **kwds)
is evaluated by first checking whether the object ``f`` implements
a ``__cocall__`` method. If it does, the cocall expression is
equivalent to
::
yield from f.__cocall__(*args, **kwds)
except that the object returned by __cocall__ is expected to be an
iterator, so the step of calling iter() on it is skipped.
If ``f`` does not have a ``__cocall__`` method, or the ``__cocall__``
method returns ``NotImplemented``, then the cocall expression is
treated as an ordinary call, and the ``__call__`` method of ``f``
is invoked.
Objects which implement __cocall__ are expected to return an object
obeying the iterator protocol. Cofunctions respond to __cocall__ the
same way as ordinary generator functions respond to __call__, i.e. by
returning a generator-iterator.
Certain objects that wrap other callable objects, notably bound methods,
will be given __cocall__ implementations that delegate to the underlying
object.
Grammar
-------
The full syntax of a cocall expression is described by the following
grammar lines:
::
atom: cocall | <existing alternatives for atom>
cocall: 'cocall' atom cotrailer* '(' [arglist] ')'
cotrailer: '[' subscriptlist ']' | '.' NAME
Note that this syntax allows cocalls to methods and elements of sequences
or mappings to be expressed naturally. For example, the following are valid:
::
y = cocall self.foo(x)
y = cocall funcdict[key](x)
y = cocall a.b.c[i].d(x)
Also note that the final calling parentheses are mandatory, so that for example
the following is invalid syntax:
::
y = cocall f # INVALID
New builtins, attributes and C API functions
--------------------------------------------
To facilitate interfacing cofunctions with non-coroutine code, there will
be a built-in function ``costart`` whose definition is equivalent to
::
def costart(obj, *args, **kwds):
try:
m = obj.__cocall__
except AttributeError:
result = NotImplemented
else:
result = m(*args, **kwds)
if result is NotImplemented:
raise TypeError("Object does not support cocall")
return result
There will also be a corresponding C API function
::
PyObject *PyObject_CoCall(PyObject *obj, PyObject *args, PyObject *kwds)
It is left unspecified for now whether a cofunction is a distinct type
of object or, like a generator function, is simply a specially-marked
function instance. If the latter, a read-only boolean attribute
``__iscofunction__`` should be provided to allow testing whether a given
function object is a cofunction.
Motivation and Rationale
========================
The ``yield from`` syntax is reasonably self-explanatory when used for the
purpose of delegating part of the work of a generator to another function. It
can also be used to good effect in the implementation of generator-based
coroutines, but it reads somewhat awkwardly when used for that purpose, and
tends to obscure the true intent of the code.
Furthermore, using generators as coroutines is somewhat error-prone. If one
forgets to use ``yield from`` when it should have been used, or uses it when it
shouldn't have, the symptoms that result can be extremely obscure and confusing.
Finally, sometimes there is a need for a function to be a coroutine even though
it does not yield anything, and in these cases it is necessary to resort to
kludges such as ``if 0: yield`` to force it to be a generator.
The ``cocall`` construct address the first issue by making the syntax directly
reflect the intent, that is, that the function being called forms part of a
coroutine.
The second issue is addressed by making it impossible to mix coroutine and
non-coroutine code in ways that don't make sense. If the rules are violated, an
exception is raised that points out exactly what and where the problem is.
Lastly, the need for dummy yields is eliminated by making it possible for a
cofunction to call both cofunctions and ordinary functions with the same syntax,
so that an ordinary function can be used in place of a cofunction that yields
zero times.
Record of Discussion
====================
An earlier version of this proposal required a special keyword ``codef`` to be
used in place of ``def`` when defining a cofunction, and disallowed calling an
ordinary function using ``cocall``. However, it became evident that these
features were not necessary, and the ``codef`` keyword was dropped in the
interests of minimising the number of new keywords required.
The use of a decorator instead of ``codef`` was also suggested, but the current
proposal makes this unnecessary as well.
It has been questioned whether some combination of decorators and functions
could be used instead of a dedicated ``cocall`` syntax. While this might be
possible, to achieve equivalent error-detecting power it would be necessary
to write cofunction calls as something like
::
yield from cocall(f)(args)
making them even more verbose and inelegant than an unadorned ``yield from``.
It is also not clear whether it is possible to achieve all of the benefits of
the cocall syntax using this kind of approach.
Prototype Implementation
========================
An implementation of an earlier version of this proposal in the form of patches
to Python 3.1.2 can be found here:
http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cofunctions.h…
If this version of the proposal is received favourably, the implementation will
be updated to match.
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
Hello,
I often have code of the form:
def my_fun():
allocate_res1()
try:
# do stuff
allocate_res2()
try:
# do stuff
allocate_res3()
try:
# do stuff
finally:
cleanup_res3()
finally:
cleanup_res2()
finally:
cleanup_res1()
return
With increasing number of managed resources, the indentation becomes
really annoying, there is lots of line noise, and I don't like the fact
that the cleanup is so far away from the allocation.
I would much rather have something like this:
def my_fun():
allocate_res1()
atreturn.register(cleanup_res1)
# do stuff
allocate_res2()
atreturn.register(cleanup_res2)
# do stuff
allocate_res3()
atreturn.register(cleanup_res3)
# do stuff
return
Has the idea of implementing such "on return" handlers ever come up?
Maybe there is some tricky way to do this with function decorators?
Best,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
I just had something pointed out to me in #python that min and max accept
*args, I know for a fact that any and all only use a single iterable
password.
Shouldn't we allow either one of the two ideas?
Either max/min take only iterable arguments.
OR
Allow any/all to use *args
In the second case this becomes legal
any(1 in a, 2 in b, 3 in c)
In the first case this becomes illegal
max(1,2,3,4,5)
Hello,
I would like to propose the following PEP for discussion and, if
possible, acceptance. I think the proposal shouldn't be too
controversial (I find it quite simple and straightforward myself :-)).
I also have a draft implementation that's quite simple
(http://hg.python.org/features/pep-3155).
Regards
Antoine.
PEP: 3155
Title: Qualified name for classes and functions
Version: $Revision$
Last-Modified: $Date$
Author: Antoine Pitrou <solipsis(a)pitrou.net>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2011-10-29
Python-Version: 3.3
Post-History:
Resolution: TBD
Rationale
=========
Python's introspection facilities have long had poor support for nested
classes. Given a class object, it is impossible to know whether it was
defined inside another class or at module top-level; and, if the former,
it is also impossible to know in which class it was defined. While
use of nested classes is often considered poor style, the only reason
for them to have second class introspection support is a lousy pun.
Python 3 adds insult to injury by dropping what was formerly known as
unbound methods. In Python 2, given the following definition::
class C:
def f():
pass
you can then walk up from the ``C.f`` object to its defining class::
>>> C.f.im_class
<class '__main__.C'>
This possibility is gone in Python 3::
>>> C.f.im_class
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 'im_class'
>>> dir(C.f)
['__annotations__', '__call__', '__class__', '__closure__', '__code__',
'__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__',
'__eq__', '__format__', '__ge__', '__get__', '__getattribute__',
'__globals__', '__gt__', '__hash__', '__init__', '__kwdefaults__',
'__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__']
This limits again the introspection capabilities available to the user.
It can produce actual issues when porting software to Python 3, for example
Twisted Core where the issue of introspecting method objects came up
several times. It also limits pickling support [1]_.
Proposal
========
This PEP proposes the addition of a ``__qname__`` attribute to functions
and classes. For top-level functions and classes, the ``__qname__``
attribute is equal to the ``__name__`` attribute. For nested classed,
methods, and nested functions, the ``__qname__`` attribute contains a
dotted path leading to the object from the module top-level.
The repr() and str() of functions and classes is modified to use ``__qname__``
rather than ``__name__``.
Example with nested classes
---------------------------
>>> class C:
... def f(): pass
... class D:
... def g(): pass
...
>>> C.__qname__
'C'
>>> C.f.__qname__
'C.f'
>>> C.D.__qname__
'C.D'
>>> C.D.g.__qname__
'C.D.g'
Example with nested functions
-----------------------------
>>> def f():
... def g(): pass
... return g
...
>>> f.__qname__
'f'
>>> f().__qname__
'f.g'
Limitations
===========
With nested functions (and classes defined inside functions), the dotted
path will not be walkable programmatically as a function's namespace is not
available from the outside. It will still be more helpful to the human
reader than the bare ``__name__``.
As the ``__name__`` attribute, the ``__qname__`` attribute is computed
statically and it will not automatically follow rebinding.
References
==========
.. [1] "pickle should support methods":
http://bugs.python.org/issue9276
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
I tried to capture all the ideas generated during the concurrency
discussion early this month in the form of an informational PEP. I
believe that information is worth preserving where others can find it,
and a PEP seems like a logical place. Since it doesn't make a specific
proposal, it's informational PEP. Possibly this is an abuse of the PEP
mechanism, in which case I'll find another forum for it.
The point this time is not to debate the ideas proposed, but to check
for errors and omissions. If there's an alternative I missed, let me
know. If there's a problem with some alternative I missed, let me
know. If I think it's already in the document, I'll probably point it
out and ask for suggestions on how to make it more obvious.
Thanks,
<mike
I was looking through the "What's New for Python 3.3" file and I saw this example code:
try:
with open("document.txt") as f:
content = f.read()
except FileNotFoundError:
print("document.txt file is missing")
except PermissionError:
print("You are not allowed to read document.txt")
and I thought it would be more elegant if you could drop the try.
with open("document.txt") as f:
content = f.read()
except FileNotFoundError:
print("document.txt file is missing")
except PermissionError:
print("You are not allowed to read document.txt")
I assume that this has already been proposed and rejected. Does anyone have a good explanation of why? ISTM that the with-statement is mostly just a way of abstracting out try-except blocks, so it might be nice to have a way of handling errors that pop up during the initialization phase instead of just the code-block phase.
I guess one obvious counter argument is that naive programmers might think that the "except" applies to things in the block instead of just things in the initializer. But naive programmers think lots of wrong things. ;-)
I was able to create a patch for testing this idea. The hard part was
in getting to know cpython well enough to do it. :-)
To get it to work, I made the following change in ceval.c so that the
main loop will accept a pointer rather than an int for the throwflag.
That allows an opcode to set it before it yields an exception. ie... a
reverse throw.
PyObject *
PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) {
return PyEval_EvalFrame_Ex(f, &throwflag);
}
PyObject *
PyEval_EvalFrame_Ex(PyFrameObject *f, int *throwflag)
{
...
TARGET(YIELD_EXCEPT)
*throwflag = 1;
retval = POP();
f->f_stacktop = stack_pointer;
why = WHY_YIELD;
goto fast_yield;
...
The genobject gen_send_ex() function checks the throwflag value after a
send to see if it got a thrown out exception back. (Rather than one
yielded out.)
A new keyword 'throws' was needed to go with the YIELD_EXCEPT opcode. I
didn't see anyway to do it with a function or method. It should be
possible to set the exception in the ceval loop rather than yielding it
out, but I think that would be more complex.
Because the exception isn't raised inside the generator code object, but
is yielded out first, the generator can be continued as if it was a
regular yielded value. No magic required, and no fiddling with the
exception stack was needed. It doesn't effect unexpected exceptions, or
exceptions raised with raise. Those will terminate a generator as
always.
Python 3.3.0a0 (qbase qtip tip yield_except:92ac2848438f+, Nov 18 2011,
21:59:57)
[GCC 4.6.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def G():
... yield 1
... throws ValueError
... yield 2
...
>>> g = G()
>>> next(g)
1
>>> next(g)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError
>>> next(g)
2
>>> next(g)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
The three main benefits of being able to do this are...
* To use switch like exception structures for flow control in schedulers
and coroutines.
* For consumer type coroutines to be able reject and re-request data in
a nice way without terminating. ie.. the reverse of throwing in an
exception in, in order to change what a generator does.
* It creates alternative channels for data input and output by being
able to both throw exceptions in and out of generators. Those signals
can carry objects in and out and not burden the fast yield data path
with testing for special wrapper objects.
Here's an example of it being used in a simple scheduler.
-----------
class Suspend(Exception): pass
def Person(name, count, mode):
n = 0
while n < count:
if mode == 0:
# The normal data path.
yield name, count
else:
# Use an exception as an alternative data path.
throws Suspend(name, count)
n += 1
# return
raise StopIteration(name, n)
def main(data, mode):
stack = [Person(*(args + (mode,))) for args in data]
results = []
while stack:
done = []
for ct in stack:
try:
print('yield', *next(ct)) # from yield
except Suspend as exc:
print('throws', *exc.args) # from throws
except StopIteration as exc:
results.append(exc.args)
continue
done.append(ct)
stack = done
print(results)
return results
if __name__ == "__main__":
data = [("John", 2), ("Micheal", 3), ("Terry", 4)]
results1 = main(data, 0)
results2 = main(data, 1)
assert(results1 == results2 == data)
-------------
The output looks like...
yield John 2
yield Micheal 3
yield Terry 4
yield John 2
yield Micheal 3
yield Terry 4
yield Micheal 3
yield Terry 4
yield Terry 4
[('John', 2), ('Micheal', 3), ('Terry', 4)]
throws John 2
throws Micheal 3
throws Terry 4
throws John 2
throws Micheal 3
throws Terry 4
throws Micheal 3
throws Terry 4
throws Terry 4
[('John', 2), ('Micheal', 3), ('Terry', 4)]
This shows that 'throws' works a lot like 'yield'.
Open issues:
* A better name than 'throws' might be good.
* Should it get the object sent in.
<object> = throws <exception>
Or should it be ...
throws <exception>
* What would be the best argument form.. Should it take the same
arguments as raise or just a single expression.
Python's test suite passes as this doesn't change anything that already
works.
I haven't tested it with the yield-from patch yet, but I think if it can
throw out exceptions in the same way yield-from yields out, that it will
make some things easier and nicer to do.
If anyone is interested, I can create a tracker item and put the patch
there where it can be improved further.
Cheers,
Ron
