[Python-Dev] News from asyncio

[Victor Stinner]

Hi,

I'm working for eNovance on the asyncio module, the goal is to use it
in the huge OpenStack project (2.5 millions line of code) which
currently uses eventlet. I'm trying to fix remaining issues in the
asyncio module before Python 3.4 final.

The asyncio project is very active but discussions are splitted
between its own dedicated mailing list (python-tulip Google group),
Tulip bug tracker and Python bug tracker. Please join Tulip mailing
list if you are interested to contribute.

   http://code.google.com/p/tulip/

I would like to share with you the status of the module. Many bugs
have been fixed recently. I suppose that new bugs are found because
new developers started to play with asyncio since Python 3.4 beta 1.

asyncio issues fixed in Python 3.4 beta 3, in a random order:

- I wrote most of the asyncio documentation, please help me to improve
it! I tried to add many short examples, each time explaining one
feature or concept (ex: callbacks, signals, futures, etc.):

   http://docs.python.org/dev/library/asyncio.html

- Characters devices (TTY/PTY) are now supported, useful to control
real terminal (not pipes) for subprocesses. On Mac OS X older than
Maverick (10.9), the SelectSelector should be used instead of
KqueueSelector (kqueue didnd't support character devices)

- Tulip #111: StreamReader.readexactly() now raises an
IncompleteReadError if the
end of stream is reached before we received enough bytes, instead of returning
less bytes than requested.

- Python #20311: asyncio had a performance issue related to the
resolution of selectors and clocks. For example,
selectors.EpollSelector has a resolution of 1 millisecond (10^-3),
whereas asyncio uses arbitrary timestamps. The issue was fixed by
adding a resolution attribute to selectors and a private granularity
attribute to asyncio.BaseEventLoop, and use the granularity in asyncio
event loop to round times.

- New Task.current_task() class method

- Guido wrote a web crawler, see examples/crawl.py in Tulip

- More symbols are exported in the main asyncio module (ex: Queue,
iscouroutine(), etc.)

- Charles-François improved the signal handlers: SA_RESTART flag is
now set to limit EINTR errors in syscalls

- Some optimizations (ex: don't call logger.log() when it's not needed)

- Many bugfixes

- (sorry if I forgot other changes, see also Tulip history and history
of the asyncio module in Python)


I also would like to change asyncio to support a "stream-like" API for
subprocesses, see Tulip issue #115 (and Python issue #20400):

   http://code.google.com/p/tulip/issues/detail?id=115


I ported ayncio on Python 2.6 and 2.7, because today OpenStack only
uses these Python versions. I created a new project called "Trollius"
(instead of "Tulip") because the syntax is a little bit different.
"yield from" becomes "yield", and "return x" becomes "raise
Return(x)":

   https://bitbucket.org/enovance/trollius
   https://pypi.python.org/pypi/trollius


If you are interested by the OpenStack part, see my blueprint
(something similar to PEPs but for smaller changes) for Oslo Messaing:

   https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio

There is an ongoing effort to port OpenStack to Python 3, eNovance is
also working on the portage:

   https://wiki.openstack.org/wiki/Python3

Victor