[Python-Dev] generic async io (was: microthreading vs. async io)

[Nick Maclaren]

dustin at v.igoro.us wrote:
>
> Knowing the history of something like this is very helpful, but I'm not
> sure what you mean by this first paragraph.  I think I'm most unclear
> about the meaning of "The 'threading' approach to asynchronous I/O"?
> Its opposite ("separating asynchronous I/O from threading") doesn't
> illuminate it much more.  Could you elaborate?

I'll try.  Sorry about being unclear - it is one of my failings.
Here is an example draft of some interfaces:

Threading
---------

An I/O operation passes a buffer, length, file and action and receives a
token back.  This token can be queried for completion, waited on and so
on, and is cancelled by waiting on it and getting a status back.  I.e.
it is a thread-like object.  This is the POSIX-style operation, and is
what I say cannot be made to work effectively.

Streaming
---------

An I/O operation either writes some data to a stream or reads some data
from it; such actions are sequenced within a thread, but not between
threads (even if the threads coordinate their I/O).  Data written goes
into limbo until it is read, and there is no way for a reader to find
the block boundaries it was written with or whether data HAS been
written.  A non-blocking read merely tests if data are ready for
reading, which is not the same.

There are no positioning operations, and only open, close and POSSIBLY a
heavyweight synchronise or rewind (both equivalent to close+open) force
written data to be transferred.  Think of Fortran sequential I/O without
BACKSPACE or C I/O without ungetc/ungetchar/fseek/fsetpos.

Transactions
------------

An I/O operation either writes some data to a file or reads some data
from it.  There is no synchronisation of any form until a commit.  If
two transfers between a pair of commits overlap (including file length
changes), the behaviour is undefined.  All I/O includes its own
positioning, and no positioning is relative.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679