[Python-Dev] PEP 3156 - Asynchronous IO Support Rebooted

[Antoine Pitrou]

Hello,

> To get the current event loop, use get_event_loop(). This returns an
> instance of the EventLoop class defined below or an equivalent
> object. It is possible that get_event_loop() returns a different
> object depending on the current thread, or depending on some other
> notion of context.
> 
> To set the current event loop, use set_event_loop(event_loop), where
> event_loop is an instance of the EventLoop class or equivalent. This
> uses the same notion of context as get_event_loop().

So can we instantiate an EventLoop directly and then call
set_event_loop() with it? Or is the use case different?

> - ``create_transport(protocol_factory, host, port, **kwargs)``.
>   Creates a transport and a protocol and ties them together.  Returns
>   a Future whose result on success is a (transport, protocol) pair.
>   Note that when the Future completes, the protocol's
>   ``connection_made()`` method has not yet been called; that will
>   happen when the connection handshake is complete.  When it is
>   impossible to connect to the given host and port, the Future will
>   raise an exception instead.
> 
>   Optional keyword arguments:
> 
>   - ``family``, ``type``, ``proto``, ``flags``: Address familty,
>     socket type, protcol, and miscellaneous flags to be passed through
>     to ``getaddrinfo()``.  These all default to ``0`` except ``type``
>     which defaults to ``socket.SOCK_STREAM``.
> 
>   - ``ssl``: Pass ``True`` to create an SSL transport (by default a
>     plain TCP is created).  Or pass an ``ssl.SSLContext`` object to
>     override the default SSL context object to be used.
> 
>   TBD: Should this be called create_connection()?

Either create_connection() or create_client(). create_transport() is
wrong, since server transports wouldn't use that function.

I would favour create_client() if this function is also meant to
support UDP (I know you haven't thought about UDP yet, but it is an
important and common use case).

I have another question about that API: if I want to cancel the
connection attempt after a given delay, how do I do that? If I call
cancel() on the future, does it cancel the connect() call?

As for SSL, there are security issues with having a "default SSL
context" (notably, any decent client use of SSL *must* check the server
certificate against an appropriate set of CAs). It's much better to
force users to pass a context explicitly. Choosing default settings
should only be for higher-level APIs like urllib.request.

(btw, don't you mean that family defaults to AF_INET?)

> If executor is None, a default ThreadPoolExecutor with 5 threads is used

Is it because Twisted's thread pool has minThreads=5? :)

> The transport is free to buffer the bytes, but it must eventually
> cause the bytes to be transferred to the entity at the other end, and
> it must maintain stream behavior. That is, t.write(b'abc');
> t.write(b'def') is equivalent to t.write(b'abcdef')

I think this is a bad idea. The kernel's network stack should do the
buffering (and choose appropriate algorithms for that), not the
user-level framework. The transport should write the bytes as soon as
the fd is ready for writing, and it should write the same chunks as
given by the user, not a concatenation of them.

Besides, it would be better if transports weren't automatically
*streaming* transports. There are connected datagram protocols, such as
named pipes under Windows (multiprocessing already uses non-blocking
Windows named pipes).

> Proposal: let the transport call protocol.pause() and
> protocol.resume() if they exist; if they don't exist, the protocol
> doesn't support flow control. 

+1. The Protocol base class can provide default no-op implementations.

> TBD: Discuss whether user code needs to do anything to make sure that
> protocol and transport aren't garbage-collected prematurely.

The transport should be tied to the event loop as long as the
connection holds, and the protocol will hold to the transport.

> TBD: Need an interface to wait for the first of a collection of Futures.

Have you looked at Twisted's DeferredList?
http://twistedmatrix.com/documents/12.1.0/api/twisted.internet.defer.DeferredList.html

I think par() could take a keyword-only argument to specify you want
the callback to be triggered on the first result (and perhaps being
able to choose between "the first success result" and "the first
success or error result").

> A trick used by Richard Oudkerk in the tulip project's proactor
> branch makes calls like recv() either return a regular result or
> raise a Future. The caller (likely a transport) must then write code
> like this:

Isn't it a case of premature optimization?
If we want to keep this, there should be a nicer API, perhaps like
Twisted's maybeDeferred:
http://twistedmatrix.com/documents/current/api/twisted.internet.defer.html#maybeDeferred

> We might also introduce explicit locks (though these will be a bit of
> a pain to use, as we can't use the with lock: block syntax). 

I don't understand why you couldn't use "with lock" in a coroutine. Am
I misunderstanding something?

>  Is it reasonable to map write(), writelines(), data_received() to
> single datagrams?

Well, at least that's how Twisted does it (not sure about writelines()).

Regards

Antoine.