Buffered streams design + raw io gotchas
Hello, As I continue experimenting with advanced streams, I'm currently beginning an important modification of io's Buffered and Text streams (removal of locks, adding of methods...), to fit the optimization process of the whole library. However, I'm now wondering what the idea is behind the 3 main buffer classes : Bufferedwriter, Bufferedreader and Bufferedrandom. The i/o PEP claimed that the two first ones were for sequential streams only, and the latter for all kinds of seekable streams; but as it is implemented, actually the 3 classes can be returned by open() for seekable files. Am I missing some use case in which this distinction would be useful (for optimizations ?) ? Else, I guess I should just create a RSBufferedStream class which handles all kinds of situations, raising InsupportedOperation exceptions whenever needed.... after all, text streams act that way (there is no TextWriter or TextReader stream), and they seem fine. Also, io.open() might return a raw file stream when we set buffering=0. The problem is that raw file streams are NOT like buffered streams with a buffer limit of zero : raw streams might fail writing/reading all the data asked, without raising errors. I agree this case should be rare, but it might be a gotcha for people wanting direct control of the stream (eg. for locking purpose), but no silently incomplete read/write operation. Shouldn't we rather return a "write through" buffered stream in this case "buffering=0", to cleanly handle partial read/write ops ? regards, Pascal PS : if you have 3 minutes, I'd be very interested by your opinion on the "advanced modes" draft below. Does it seem intuitive to you ? In particular, shouldn't the "+" and "-" flags have the opposite meaning ? http://bytebucket.org/pchambon/python-rock-solid-tools/wiki/rsopen.html
IIRC here is the use case for buffered reader/writer vs. random: a disk file opened for reading and writing uses a random access buffer; but a TCP stream stream, while both writable and readable, should use separate read and write buffers. The reader and writer don't have to worry about reversing the I/O direction. But maybe I'm missing something about your question? --Guido On Thu, Feb 18, 2010 at 1:59 PM, Pascal Chambon <chambon.pascal@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
Not really, BufferedRandom is only suitable when the file is open for reading *and* writing. The 'rb' and 'wb' modes should return BufferedReader and BufferedWriter, respectively. On Sat, Feb 20, 2010 at 6:20 AM, Pascal Chambon <chambon.pascal@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
IIRC here is the use case for buffered reader/writer vs. random: a disk file opened for reading and writing uses a random access buffer; but a TCP stream stream, while both writable and readable, should use separate read and write buffers. The reader and writer don't have to worry about reversing the I/O direction. But maybe I'm missing something about your question? --Guido On Thu, Feb 18, 2010 at 1:59 PM, Pascal Chambon <chambon.pascal@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
Not really, BufferedRandom is only suitable when the file is open for reading *and* writing. The 'rb' and 'wb' modes should return BufferedReader and BufferedWriter, respectively. On Sat, Feb 20, 2010 at 6:20 AM, Pascal Chambon <chambon.pascal@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
participants (2)
-
Guido van Rossum -
Pascal Chambon