[Python-Dev] PEP 3156 - Asynchronous IO Support Rebooted

Guido van Rossum guido at python.org
Fri Dec 21 22:10:45 CET 2012


--Guido van Rossum (sent from Android phone)

On Dec 21, 2012 11:47 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
> Hello,
> > To get the current event loop, use get_event_loop(). This returns an
> > instance of the EventLoop class defined below or an equivalent
> > object. It is possible that get_event_loop() returns a different
> > object depending on the current thread, or depending on some other
> > notion of context.
> >
> > To set the current event loop, use set_event_loop(event_loop), where
> > event_loop is an instance of the EventLoop class or equivalent. This
> > uses the same notion of context as get_event_loop().
> So can we instantiate an EventLoop directly and then call
> set_event_loop() with it? Or is the use case different?

That's an abstract class, but if you know a concert implementation you can
use this. E.g. latest Tulip unit tests.

> > - ``create_transport(protocol_factory, host, port, **kwargs)``.
> >   Creates a transport and a protocol and ties them together.  Returns
> >   a Future whose result on success is a (transport, protocol) pair.
> >   Note that when the Future completes, the protocol's
> >   ``connection_made()`` method has not yet been called; that will
> >   happen when the connection handshake is complete.  When it is
> >   impossible to connect to the given host and port, the Future will
> >   raise an exception instead.
> >
> >   Optional keyword arguments:
> >
> >   - ``family``, ``type``, ``proto``, ``flags``: Address familty,
> >     socket type, protcol, and miscellaneous flags to be passed through
> >     to ``getaddrinfo()``.  These all default to ``0`` except ``type``
> >     which defaults to ``socket.SOCK_STREAM``.
> >
> >   - ``ssl``: Pass ``True`` to create an SSL transport (by default a
> >     plain TCP is created).  Or pass an ``ssl.SSLContext`` object to
> >     override the default SSL context object to be used.
> >
> >   TBD: Should this be called create_connection()?
> Either create_connection() or create_client(). create_transport() is
> wrong, since server transports wouldn't use that function.
> I would favour create_client() if this function is also meant to
> support UDP (I know you haven't thought about UDP yet, but it is an
> important and common use case).


> I have another question about that API: if I want to cancel the
> connection attempt after a given delay, how do I do that? If I call
> cancel() on the future, does it cancel the connect() call?

It does in Tulip, because it's really a task. Maybe this should be in the

> As for SSL, there are security issues with having a "default SSL
> context" (notably, any decent client use of SSL *must* check the server
> certificate against an appropriate set of CAs). It's much better to
> force users to pass a context explicitly. Choosing default settings
> should only be for higher-level APIs like urllib.request.

Hm. That makes simple tests harder. But I understand the concern.

> (btw, don't you mean that family defaults to AF_INET?)
> > If executor is None, a default ThreadPoolExecutor with 5 threads is used
> Is it because Twisted's thread pool has minThreads=5? :)

Yes, and to encourage the use of set_default_executor() ... :-)

> > The transport is free to buffer the bytes, but it must eventually
> > cause the bytes to be transferred to the entity at the other end, and
> > it must maintain stream behavior. That is, t.write(b'abc');
> > t.write(b'def') is equivalent to t.write(b'abcdef')
> I think this is a bad idea. The kernel's network stack should do the
> buffering (and choose appropriate algorithms for that), not the
> user-level framework. The transport should write the bytes as soon as
> the fd is ready for writing, and it should write the same chunks as
> given by the user, not a concatenation of them.

I asked Glyph about this. It depends on the OS... Mac syscalls are so slow
that it is better to join in user space. This should really be up to the
transport, although for stream transports the given equivalency should
definitely hold.

> Besides, it would be better if transports weren't automatically
> *streaming* transports. There are connected datagram protocols, such as
> named pipes under Windows (multiprocessing already uses non-blocking
> Windows named pipes).

I think we need to support datagrams, but the default ought to be stream.

> > Proposal: let the transport call protocol.pause() and
> > protocol.resume() if they exist; if they don't exist, the protocol
> > doesn't support flow control.
> +1. The Protocol base class can provide default no-op implementations.


> > TBD: Discuss whether user code needs to do anything to make sure that
> > protocol and transport aren't garbage-collected prematurely.
> The transport should be tied to the event loop as long as the
> connection holds, and the protocol will hold to the transport.


> > TBD: Need an interface to wait for the first of a collection of Futures.
> Have you looked at Twisted's DeferredList?

No, I am trying to stay away from them.

> I think par() could take a keyword-only argument to specify you want
> the callback to be triggered on the first result (and perhaps being
> able to choose between "the first success result" and "the first
> success or error result").

Good idea. This is unexplored.

> > A trick used by Richard Oudkerk in the tulip project's proactor
> > branch makes calls like recv() either return a regular result or
> > raise a Future. The caller (likely a transport) must then write code
> > like this:
> Isn't it a case of premature optimization?

Yeah, we should not do this.

> If we want to keep this, there should be a nicer API, perhaps like
> Twisted's maybeDeferred:


> > We might also introduce explicit locks (though these will be a bit of
> > a pain to use, as we can't use the with lock: block syntax).
> I don't understand why you couldn't use "with lock" in a coroutine. Am
> I misunderstanding something?

If another task has the lock we must yield. But 'with' can't do that.

> >  Is it reasonable to map write(), writelines(), data_received() to
> > single datagrams?
> Well, at least that's how Twisted does it (not sure about writelines()).


> Regards
> Antoine.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121221/e73a195b/attachment-0001.html>

More information about the Python-Dev mailing list