[Python-Dev] Draft PEP to make file objects support non-blockingmode.

Donovan Baarda abo at minkirri.apana.org.au
Mon Mar 21 11:06:51 CET 2005


From: "Greg Ward" <gward at python.net>
> On 18 March 2005, Donovan Baarda said:
> > Currently the built in file type does not support non-blocking mode very
> > well.  Setting a file into non-blocking mode and reading or writing to
> > can only be done reliably by operating on the file.fileno() file
> > This requires using the fnctl and os module file descriptor manipulation
> > methods.
> Is having to use fcntl and os really so awful?  At least it requires
> the programmer to prove he knows what he's doing putting this file
> into non-blocking mode, and that he really wants to do it.  ;-)

It's not that bad I guess... but then I'm proposing a very minor change to
fix it.

The bit that annoys me is popen2() and select() give this false sense of
"File Object compatability", when in reality you can't use them reliably
with file objects.

It is also kind of disturbing that file.read() actually does work in
non-blocking mode, but file.write() doesn't. The source for file.read()
shows a fair bit of effort towards making it work for non-blocking mode...
why not do the same for file.write()?

> > Details
> > =======
> >
> > The documentation of file.read() warns; "Also note that when in
> > mode, less data than what was requested may be returned, even if no size
> > parameter was given".  An empty string is returned to indicate an EOF
> > condition.  It is possible that file.read() in non-blocking mode will
> > produce any data before EOF is reached.  Currently there is no
> > way to identify the difference between reaching EOF and an empty
> > non-blocking read.
> >
> > The documented behaviour of file.write() in non-blocking mode is
> > When writing to a file in non-blocking mode, it is possible that not all
> > the data gets written.  Currently there is no documented way of handling
> > indicating a partial write.
> That's more interesting and a better motivation for this PEP.

The other solution to this of course is to simply say "file.read() and
file.write() don't work in non-blocking mode", but that would be a step
backwards for the current file.read().

> > file.read([size]) Changes
> > --------------------------
> >
> > The read method's current behaviour needs to be documented, so its
> > behaviour can be used to differentiate between an empty non-blocking
> > and EOF.  This means recording that IOError(EAGAIN) is raised for an
> > non-blocking read.
> >
> >
> > file.write(str) Changes
> > --------------------
> >
> > The write method needs to have a useful behaviour for partial
> > writes defined, implemented, and documented.  This includes returning
> > many bytes of "str" are successfully written, and raising
> > for an unsuccessful write (one that failed to write anything).
> Proposing semantic changes to file.read() and write() is bound to
> raise hackles.  One idea for soothing such objections: only make these
> changes active when setblocking(False) is in effect.  I.e., a
> setblocking(True) file (the default, right?) behaves as you described
> above, warts and all.  (So old code that uses fcntl() continues to
> "work" as before.)  But files that have had setblocking(False) called
> could gain these new semantics that you propose.

There is nothing in this proposal that would break or change the behaviour
of any existing code, unless it was relying on file.write() returning None.
or checking that file objects don't have a "setblocking" method.

Note that the change for file.read() is simply to document the current
behaviour... not to actually change it.

The change for file.write() is a little more dramatic, but I really can't
imagine anyone relying on file.write() returning None. A compromise would be
to have file.write() return None in blocking mode, and a count in
non-blocking mode... but I still can't believe people will rely on it
returning None :-) It would be more useful to always return a count, so that
methods using them could handle both modes easily.

Note that I did consider some more dramatic changes that would have made
them even easier to use. Things like raising an exception for EOF instead of
EAGAIN would actually make a lot of things easier to code... but it would be
too big a change.

Donovan Baarda                http://minkirri.apana.org.au/~abo/

More information about the Python-Dev mailing list