[Python-3000] Draft PEP for New IO system

[Daniel Stutzbach]

On 3/1/07, Adam Olsen <rhamph at gmail.com> wrote:
> Why do non-blocking operations need to use the same methods when
> they're clearly not the same semantics?  Although long,
> .nonblockflush() would be explicit and allow .flush() to still block.

.nonblockflush() would be fine with me, but I don't think .flush()
should block on a non-blocking object.  To accomplish that, it would
either have to be smart enough to switch the object into blocking
mode, or internally use select().

How about .flush() write as much as it can, and throw an exception if
not all of the bytes can be written to the device?  (again, this would
only come up when a user has set the underlying file descriptor to
non-blocking mode)

> I'm especially wary of infinite buffers.  They allow a malicious peer
> to consume all your memory, DoSing the process or even the whole box
> if Linux's OOM killer doesn't kick in fast enough.

For a write-buffer, you start eating up memory only if an application
is buffer-ignorant and tries to dump a massive amount of data to the
socket all at once.  A naive HTTP server implementation might do this
by calling something like s.write(open(filename)).  This isn't a DoS
by a peer though, it's a local implementation problem.

For a read-buffer, you start eating up all of memory only if you call
.read() with no arguments and the peer dumps a few gig on you.  If you
call read(n) to get as much data as you need, then the buffer class
will only grab reasonably sized chunks from the network.  Network
applications don't normally call .read() without arguments, since they
need to communicate both ways.  If the object is an ordinary file,
then DoS isn't so much of an issue and reading the whole files seems
very reasonable.

I suppose for an Text object wrapped around a socket, .readline()
could be dangerous if a malicious peer sends a few gig all on one
line.  That's a problem for the encoding layer to sort out, not the
buffering layer though.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC