Negative block sizes with file-like objects

[Peter Otten]

Steven D'Aprano wrote:

> I have a proxy class that wraps an arbitrary file-like object fp and
> reads blocks of data from it. Is it safe to assume that fp.read(-1) will
> read until EOF? I know that's true for file.read() and StringIO.read(),
> but is it a reasonable assumption to make for arbitrary file-like objects?
> 
> To put it in more concrete terms, I have a class like this:
> 
> class C(object):
>     # Much simplified version.
>     def __init__(self, fp):
>         self.fp = fp
>     def read(self, size=-1):
>         return self.fp.read(size)
> 
> 
> Should I re-write the read() method like this?
> 
>     def read(self, size=-1):
>         if size < 0:
>             return self.fp.read()
>         else:
>             return self.fp.read(size)

Grepping through the python source shows that both -1 and None are used as
the default size.

# python2.5.2
>>> import tarfile
>>> open("sample", "w").write("contents-of-sample")
>>> t = tarfile.open("sample.tar", "w")
>>> t.add("sample")
>>> t.close()
>>> t = tarfile.open("sample.tar")
>>> t.extractfile("sample").read()
'contents-of-sample'
>>> t.extractfile("sample").read(-1)[:50]
'contents-of-sample\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

So you may fare better with the rewrite. But even that may fail:

http://docs.python.org/lib/bltin-file-objects.html#l2h-295

"""
read(  [size])
[...] If the size argument is negative or omitted, read all data until EOF
is reached. [...]
Also note that when in non-blocking mode, less data than what was requested
may be returned, even if no size parameter was given. 
"""

Peter