Please see http://www.python.org/dev/peps/pep-0434/ for the complete
PEP. The idea for the PEP was generated because a right click menu
was added to IDLE in Python 2.7 this started a debate about bug fix VS
enhancement see the PEP references for more information. IDLE has
many patches that already exist but are difficult to get committed.
This PEP is designed to make it easier to bring IDLE up to speed with
modern GUI standards. The PEP is not perfect some issues that need to
be solved:
-How do we test IDLE patches to make sure they are high quality?
Apparently the build bots don't test user interface issues. My
philosophy is keep it simple so I was thinking that a simple procedure
for testing on each of the major platforms should be performed before
a commit. Creating a build bot that tests user interfaces seems
difficult to me and could put IDLE improvements even more behind.
-Does it make sense to separate IDLE from the stdlib for Python 3.4?
I understand the batteries included argument, but if IDLE development
is going to continue to proceed at a more accelerated pace than
Python, it might make sense to separate it out into its own project or
a sub-project.
Please provide comments or other concerns. Thanks.
With the removal of unbound methods in Python 3, introspecting the
class on which a method was defined is not so simple. Python 3.3
gives us __qualname__ for classes and functions (see PEP 3155), which
helps somewhat by giving us a string composed at compile-time.
However, I was considering something a little more powerful.
I propose that what would have formerly been an unbound method get a
new attribute, __origin__. It will be bound to class where the method
is defined.
Motivation
==========
One motivator for this proposal is http://bugs.python.org/issue15582,
allowing inspect.getdoc() to "inherit" docstrings. Without a concrete
connection between a method and the "origin" class, dynamically
determining the docstring is basically a non-starter. The __origin__
attribute would help.
Extended Idea
=============
The name, __origin__, is a bit generic to accommodate the possible
extension of the proposal to other objects:
* f.__origin__ (the module or function (nested) where the function
was defined)
* cls.__origin__ (the module, class (nested), or function where the
class was defined)
* code.__origin__ (the function for which the code object was
originally created)
* module.__origin__ (a.k.a. module.__file__)
For functions and classes defined in a function, __origin__ would be
bound to the function rather than the locals or the code object. For
modules, __origin__ is more accurate in the cases that the module was
generated from something other than a file. It would still be a
string, though. Also, we currently have a convention of setting
__module__ to the name of the module rather than the module itself.
Whether to break with that convention for __origin__ is an open
question which relates to how __origin__ would be used.
Conceivably, each use case for __origin__ could be covered by a name
more specific to the object, e.g. module.__file__. However, having
the one name would help make the purpose clear and consistent.
Objections
==========
The downside of binding the objects to __origin__ is in memory usage
and, particularly, in ref-counts/reference-cycles. I expect that this
is where the biggest objections will lie. However, my understanding
is that this is less of an issue now than it used to be. If need be,
weakref proxies could address some of the concerns.
The status quo has a gap in the case of "unbound" methods and of code
objects, though __qualname__ does offer an improvement for methods.
The actual use case I have relates strictly to those methods. Having
something like __origin__ for those methods, at least, would be
helpful.
-eric
An exchange with Antoine in one of the enum threads sparked a thought.
A recurring suggestion for collections.namedtuple is that it would be
nice to be able to define them like this (as it not only avoids having
to repeat the class name, but also allows them to play nicely with
pickle and other name-based reference mechanisms):
class MyTuple(collections.NamedTuple):
__fields__ = "a b c d e".split()
However, one of Raymond's long standing objections to such a design
for namedtuple is the ugliness of people having to remember to include
the right __slots__ definition to ensure it doesn't add any storage
overhead above and beyond that for the underlying tuple.
For the intended use case as a replacement for short tuples, an unused
dict per instance is a *big* wasted overhead, so that concern can't be
dismissed as premature optimisation:
>>> import sys
>>> class Slots(tuple): __slots__ = ()
...
>>> class InstanceDict(tuple): pass
...
>>> sys.getsizeof(tuple([1, 2, 3]))
72
>>> x = Slots([1, 2, 3])
>>> sys.getsizeof(x)
72
>>> y = InstanceDict([1, 2, 3])
>>> sys.getsizeof(y) # All good, right?
72
>>> sys.getsizeof(y.__dict__) # Yeah, not so much...
96
However, the thought that occurred to me is that the right metaclass
definition allows the default behaviour of __slots__ to be flipped, so
that you get "__slots__ = ()" defined in your class namespace
automatically, and you have to write "del __slots__" to get normal
class behaviour back:
>>> class SlotsMeta(type):
... def __prepare__(cls, *args, **kwds):
... return dict(__slots__=())
...
>>> class SlotsByDefault(metaclass = SlotsMeta): pass
...
>>> class Slots(tuple, SlotsByDefault): pass
...
>>> class InstanceDict(tuple, SlotsByDefault): del __slots__
...
>>> sys.getsizeof(Slots([1, 2, 3]))
72
>>> Slots().__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Slots' object has no attribute '__dict__'
>>> sys.getsizeof(InstanceDict([1, 2, 3]))
72
>>> sys.getsizeof(InstanceDict([1, 2, 3]).__dict__)
96
So, what do people think? Too much magic? Or just the right amount to
allow a cleaner syntax for named tuple definitions, without
inadvertently encouraging people to do bad things to their memory
usage? (Note: for backwards compatibility reasons, we couldn't use a
custom metaclass for the classes returned by the existing collections
namedtuple API. However, we could certainly provide a distinct
collections.NamedTuple type which used a custom metaclass to behave
this way).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
After all the defenses I still don't like Tim's proposed syntax. Color me Barry.
On Tue, Feb 12, 2013 at 11:54 AM, Tim Delaney <tim.delaney(a)aptare.com> wrote:
> On 13 February 2013 06:13, Guido van Rossum <guido(a)python.org> wrote:
>>
>> So with Tim's implementation, what happens here:
>>
>> class Color(Enum):
>> RED, GREEN, BLUE
>> if sys.platform == 'win32':
>> MAGENTA
>>
>> and you happen to have no "import sys" in your module? The sys lookup
>> will succeed, create a new enum, and then you will get an error
>> something like this:
>>
>> AttributeError: 'Enum' object has no attribute 'platform'
>
>
> Well, that particular case would work (you'd get a NameError: sys) due to
> having done an attribute lookup on sys, but the following:
>
> class Color(Enum):
> RED, GREEN, BLUE
> if platfor == 'win32':
> MAGENTA
>
> would create an enum value 'platfor'.
>
> Personally, I don't think it's possible to have an enum with a simple
> interface and powerful semantics with just python code without it being
> fragile. I think I've got it fairly close, but there is horrible magic in
> there (multiple kinds) and there are definitely still edge cases. Any
> complete enum implementation is going to need some special handling by the
> parser I think.
>
> I'm actually thinking that to simplify things, I need a sentinel object to
> mark the end of the enum list (which allows other names after it). But that
> still wouldn't handle the case above (the if statement).
>
> BTW, for anyone who hasn't seen the magic code (largely uncommented, no
> explanations yet of how it's doing it - I probably won't get to that until
> the weekend) it's here: https://bitbucket.org/magao/enum
>
> Tim Delaney
--
--Guido van Rossum (python.org/~guido)
I realize this isn't going very far, but I would still appreciate
feedback. :)
The code is here:
https://bitbucket.org/stoneleaf/enum
It is based on ideas from Michael Foord, Antoine Pitrou, and
(eventually) Ted Delaney.
Enum only accepts upper-case names as enum candidates; all enum values
must be the same: either 'sequence' (0, 1, 2, etc) or 'flag' (1, 2, 4,
etc.) or 'unique' (north, south, east, west, etc.); and extending via
subclassing is possible.
Sample code:
class Color(Enum):
type = 'sequence'
BLACK
RED
GREEN = 4
BLUE
print Color
# Color(BLACK:0, RED:1, GREEN:4, BLUE:5)
class MoreColor(Color):
MAGENTA
YELLOW
CYAN
print MoreColor
# MoreColor(BLACK:0, RED:1, GREEN:4, BLUE:5, MAGENTA:6, YELLOW:7,
# CYAN:8)
class DbfType(Enum):
type = 'unique'
CLP
DB3
VFP
--> print(enum.DbfType)
# DbfType(DB3:'db3', CLP:'clp', VFP:'vfp')
class SomeFlags(Enum):
type = 'flag'
ON_DISK
HAS_MEMO
LARGE_CHAR
UNICODE
--> print(enum.SomeFlags)
# SomeFlags(ON_DISK:1, HAS_MEMO:2, LARGE_CHAR:4, UNICODE:8)
class Error(Enum):
type = 'sequence'
THIS
THAT
The_Other # raises NameError
THOSE = 1 # raises InvalidEnum
-->enum.Color.RED
Color('RED')
-->enum.Color.RED == 1
True
-->int(enum.Color.RED)
1
-->enum.SomeFlags.ON_DISK
SomeFlags('ON_DISK')
-->enum.SomeFlags.ON_DISK == 1
True
-->int(enum.SomeFlags.ON_DISK)
1
-->enum.SomeFlags.ON_DISK == enum.Color.RED
False
-->enum.MoreColor.RED == 1
True
-->enum.MoreColor.RED == enum.Color.RED
True
-->enum.Color(1)
Color('RED')
-->enum.Color('RED')
Color('RED')
--> for color in enum.Color:
... print(color)
...
BLACK
RED
GREEN
BLUE
--> for color in enum.Color:
... print(repr(color))
...
Color('BLACK')
Color('RED')
Color('GREEN')
Color('BLUE')
I would appreciate any comments on both the API, and the code behind it.
Thanks.
--
~Ethan~
This idea is not new - but it is stalled -
Last I remember it came around in Python-devel in 2010, in this thread:
http://mail.python.org/pipermail/python-dev/2010-November/thread.html#105967
There is an even older PEP (PEP 354) that was rejected just for not
being enough interest at the time -
And it was not dismissed at all - to the contrary the last e-mail in the thread
is a message from the BDLF for it to **be** ! The discussion happened in a bad
moment as Python was mostly freature froozen for 3.2 - and it did not
show up again for Python 3.3;
The reasoning for wanting enums/ constants has been debated already -
but one of the main reasons that emerge from that thread are the ability to have
named constants (just like we have "True" and "False".
why do I think this is needed in the stdlib, and having itin a 3rd
party module is not enough? because they are an interesting thing to have,
not only on the stdlib, but on several widely used Python projects that
don't have other dependencies.
Having a feature like this into the stdlib allow these projects to
make use of it, without needing other dependencies, and moreover,
users which will
benefit the most out of such constants will have a wll known
"constant" type which
won't come as a surprise in each package he is using interactively or debugging.
Most of the discussion on the 2010 thread was summed up in a message by
Michael Foord in this link
http://mail.python.org/pipermail/python-dev/2010-November/106063.html
with some follow up here:
http://mail.python.org/pipermail/python-dev/2010-November/106065.html
js
-><-
----------
--
Hi,
While analyzing some code I came across the following situation:
- User calls run_until_complete for some Task
- The task itself calls loop.stop() at some point
- run_until_complete raises TimeoutError
Here is a very simple example: https://gist.github.com/saghul/4754117
Something seems a bit off here. While one could argue that stopping the
loop while run_until_complete is ongoing, getting a TimeoutError doesn't
feel right. I think we should detect that the loop was stopped but the
future is not done and raise something like NotCompleted or
FutureNotCompleted.
Thoughts?
--
Saúl Ibarra Corretgé
http://saghul.net/blog | http://about.me/saghul
I think Tulip should have synchronization primitives by default.
Here is my motivation:
1. it is more convenient to use locking primitive with existing semantics rather than tulip magic.
crawl.py example could use semaphore instead of tasks.wait with timeout 0.
2. while it seems easy to implement semaphore with tulip, it still requires deep understanding of
tulips control flow. i had 2 not very obvious bugs in my semaphore implementation:
a) release() could be called from different co-routines during same scheduling step.
b) bug with acquire and release during same scheduling step.
example:
1. task tries to acquire locked semaphore
2. other task releases semaphore
in result - semaphore gets acquired 2 times at the end of scheduling
step, because actual semaphore acquisition happens in "call_soon" and at this stage
release() call is already released semaphore but first waiter does not acquire it yet.
Thoughts?
my implementation: https://codereview.appspot.com/download/issue7230045_15001.diff
Hi all!
I haven't been able to keep up with all the tulip development on the
mailing list (hopefully I will now!) so please excuse me if something I
mention has already been discussed.
For those who may not know it, libuv is the platform layer library for
nodejs, which implements a uniform interface on top of epoll, kqueue,
event ports and iocp. I wrote Python bindings [1] for it a while ago,
and I was very excited to see Tulip, so I thought I'd give this a try.
Here [2] is the source code, along with some notes I took during the
implementation.
I know that the idea is not to re-implement the PEP itself but for
people to create different EventLoop implementations. On rose I bundled
tulip just to make a single package I could play with easily, once tulip
makes it to the stdlib only the EventLop will remain.
Here are some thoughts (in no particular order):
- add_connector / remove_connector seem to be related to Windows, but
being exposed like that feels a bit like leaking an implementation
detail. I guess there was no way around it.
- libuv implements a type of handle (Poll) which provides
level-triggered file descriptor polling which also works on Windows,
while being highly performant. It uses something called AFD Polling
apparently, which is only available on Windows >= Vista, and a select
thread on XP. I'm no Windows expert, but thanks to this the API is
consistent across all platforms, which is nice. mAybe it's worth
investigating? [3]
- The transport abstraction seems quite tight to socket objects. pyuv
provides a TCP and UDP handles, which provide a completion-style API and
use a better approach than Poll handles. They should give better
performance since EINTR in handled internally and there are less
roundtrips between Python-land and C-land. Was it ever considered to
provide some sort of abstraction so that transports can be used on top
of something other than regular sockets? For example I see no way to get
the remote party from the transport, without checking the underlying socket.
Thanks for reading this far and keep up the good work.
Regards,
[1]: https://github.com/saghul/pyuv
[2]: https://github.com/saghul/rose
[3]: https://github.com/joyent/libuv/blob/master/src/win/poll.c
--
Saúl Ibarra Corretgé
http://saghul.net/blog | http://about.me/saghul
The event loop interface in PEP 3156 has an extensibility problem. It
seems likely that it will have a method like listen_udp() by the time it's
done, but suppose that doesn't make it into the first official release.
Third-party event loop implementations may want to provide UDP support as
an extension, but the most consistent way to provide that extension is by
adding new methods on the event loop object, where various extensions risk
conflicting with each other or with new methods that become standardized
later.
The PEP specifies the add_reader family of methods in part so that core
protocol implementations can be shared across all EventLoop implementations
that support these methods. However, if those transports are implemented
in a common base class (like Twisted's PosixReactorBase), there is no way
for third-party transports to take advantage of a similar structure. I'd
like to make it possible for transports to be developed independent of a
particular EventLoop implementation in a way that is consistent with the
way the core transports work. (This is a bigger concern for tulip than it
is for twisted because twisted can update PosixReactorBase more frequently
than the stdlib can change)
I propose turning the interface around so that transport creation uses
static functions that take the event loop as an argument, with a
double-dispatch mechanism to allow the event loop to provide the actual
implementation when it can:
def create_connection(protocol_factory, host, port, event_loop=None):
if event_loop is None:
event_loop = get_event_loop()
# Note the use of a fully-qualified name in the registry
impl = event_loop.get_implementation('tulip.create_connection')
return impl(protocol_factory, host, port)
New third-party transports could provide fallbacks for event loops that
don't have their own implementations:
if impl is None:
# These supports_*() functions are placeholders for a to-be-determined
introspection interface.
if supports_fd_interface(event_loop):
return posix_udp_implementation(*args)
elif supports_iocp_interface(event_loop):
return iocp_udp_implementation(*args)
else:
raise Exception("This transport is not supported on this event loop")
Or they could plug into the event loop's implementation registry:
LibUVEventLoop.register_implementation('mymodule.listen_udp',
libuv_udp_implementation)
This does introduce a little magic (is there any precedent for this kind of
multiple-dispatch in the standard library?), but I like the way it keeps
the event loop interface from getting too big and monolithic. Third-party
transports can avoid naming conflicts without looking fundamentally
different from standard ones, and there's a clean path from doing something
that's platform-specific (e.g. with add_reader and friends) to supporting
multiple event loops to full standardization.
-Ben
