non blocking read()
steve at holdenweb.com
Thu Dec 2 00:14:00 CET 2004
Donn Cave wrote:
> In article <mailman.6995.1101939055.5135.python-list at python.org>,
> Gustavo CÃ³rdova Avila <gustavo.cordova at q-voz.com> wrote:
>>David Bolen wrote:
>>>Jp Calderone <exarkun at divmod.com> writes:
>>>> def nonBlockingReadAll(fileObj):
>>>> bytes = 
>>>> while True:
>>>> b = fileObj.read(1024)
>>>> if len(b) < 1024:
>>>> return ''.join(bytes)
>>>Wouldn't this still block if the input just happened to end at a
>>>multiple of the read size (1024)?
>>No, it'll read up to 1024 bytes or as much as it can, and
>>then return an apropriatly sized string.
> Depends. I don't believe the original post mentioned
> that the file is a pipe, socket or similar, but it's
It did actually specifically mention files.
> kind of implied by the use of select() also mentioned.
> It's also kind of implied by use of the term "block" -
> disk files don't block.
Erm, you mean they operate at infinite speed? Wrong. There's a definite
risk that a process will block when reading from disk, and the reason
that (Unix) files can be the object of select() calls is to allow users
to perform non-blocking reads and continue to process while waiting for
disk data to arrive.
I agree the blocking period is *more predictable* than for network data,
but "disk file access won't cause a process to block" is a misleading
and incorrect, though frequent, assumption.
I agree that it's difficult to actually write asynchronous disk I/O in
such a way as to improve performance, but it can be done. Except on
platforms where select() can't be applied to disk files, of course. In
practice most applications that need the scalability will use threading
or multiple processes.
> If we are indeed talking about a pipe or something that
> really can block, and you call fileobject.read(1024),
> it will block until it gets 1024 bytes.
> Donn Cave, donn at u.washington.edu
Disk access can block. When access is made to a file object in
non-blocking mode it will return whatever data there are immediately
available in the buffers, up to the number of bytes requested. If the
buffers are currently empty the process will generate an error (which in
Python becomes an exception).
Holden Web LLC +1 800 494 3119
More information about the Python-list