I'm going to try and snip as much as I can to get the the heart of this...
Thinking about what you could mean by "more esoteric protocols",
there's really not much at the level of TCP and UDP that comes to
mind. UNIX-domain sockets, and perhaps the (root-only) protocol for
sniffing packets (raw sockets?).
A new feature just landed in Tulip (I still have to update PEP 3156)
where you can pass a pre-constructed socket object to
create_connection() and start_serving(), which will make it a little
easier to support esoteric ways of setting up the socket; however,
create_connection() is still limited to sockets that implement a byte
stream, because of the way the transport/protocol API works.
Glad that is out of the way. But I'm still skeptical -- first, as I
> Right. Third-party extensions to the event loop interface are inherently
> problematic, so we'll have to provide them in some other way. I'm proposing
> a pattern for that "some other way" and then realizing that I like it even
> for first-party interfaces.
explained before, I am actually in favor of using different styles for
1st and 3rd party interfaces, so the status of the interface used is
obvious to the reader (and the coder, in case they are copy-pasting
recipes :-); second, I don't expect there will be too many
opportunities to put the pattern at work.
> TheAha! This is the executive summary of your proposal, or at least your
> idea is that someone can propose a transport interface in a third-party
> module (mymodule.listen_udp in this example), implement it themselves for
> some event loop implementations, and other event loops can declare
> themselves compatible with it.
goal for it.
This is hypothesizing rather a lot of goodwill and coordination
between different 3rd party developers.
And the registry offered by
the event loop comes down to not much more than a dictionary with keys
that follow a certain convention (e.g. fully-qualified package+module
name plus some identifier for the feature) and nothing can be said
about what the items stored in the registry are (since a packet
transport and a stream transport are not interchangeable, and even two
stream transports may not be).
Given that for each 3rd party transport the details of how to
implement a compatible version of it will vary hugely, both depending
on what the transport is trying to do and how the event loop works, I
expect that the market for this registry will be rather small. And
when a particular 3rd party transport wants to enable other 3rd party
events to support them, they can implement their own registry, which
the other 3rd party could then plug into. (But see below.)
I do see one argument in favor of having a standard registry on the
> (And in an admittedly far-fetched scenario, if there were two third-party
> UDP interfaces and LibUVEventLoop implemented one of them, yet another party
> could build a bridge between the two, and then they'd plug it in with
> register_implementation)
event loop, even if it's just a dict with register/lookup APIs and a
naming convention, and no semantics assigned to the items registered.
That argument is to make 3rd party transport implementers aware of the
possibility that some other 3rd party might want to offer a compatible
implementation aimed at an event loop that's not supported natively by
the (former) 3rd party transport. And I could even be convinced that
the standard protocols should use this registry so that the source
code serves as an example of best practices.
Still, it's a pretty weak argument IMO -- I don't expect there to be a
significant cottage industry cranking out 3rd party protocol
implementations, assuming we add UDP to PEP 3156, which is my plan.
And I don't think that create_connection() should be used to create
UDP connections -- the signature of the protocol factory passed in
would be quite different, for starters, and the set of options needed
to configure the transport is also different (assuming we want to
support both connected and connection-less UDP).
Yeah, so that's the rub: I'm not so keen on adding extra machinery to
> I don't think the status quo prevents the development of third-party
> transports, but it does implicitly encourage two bad habits: A) adding
> methods to the event loop, inviting name collisions, or B) just building on
> add_reader and friends without thinking about non-posix platforms. Of
> course, no one expects a groundswell of third-party development at the event
> loop and transport level, so this could just be so much overengineering, but
> I like it from a stylistic perspective even without the third-party
> benefits.
the PEP that I don't expect to be used much. I have more important
fish to fry (such as adding UDP :-). And adding a registry one whole
Python release cycle later (e.g. in 3.5, assuming PEP 3156 is
standardized and included in 3.4) doesn't strike me as such a bad
thing -- I don't think we're painting ourselves into much of a corner
by not having a registry right from the start.