On 4 March 2013 22:59, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le Mon, 4 Mar 2013 19:44:27 +1300, Robert Collins <robertc@robertcollins.net> a écrit :
Yes exactly. A little more context on how I came to ask the question. I wanted to accumulate all input on an arbitrary stream within 5ms, without blocking for longer. Using raw IO + select, its possible to loop, reading one byte at a time. The io module doesn't have an API (that I could find) for putting an existing stream into non-blocking mode, so reading a larger amount and taking what is returned isn't viable.
What do you mean exactly by that?
Just what I said. I'll happily try to rephrase. What bit was unclear?
However, without raw I/O, select() will timeout because it consults the underlying file descriptor bypassing the buffer. So - the only reason to want raw I/O is to be able to use select reliably.
That's a pretty good reason actually. Raw I/O is exactly for those cases. Non-blocking buffered I/O is a hard conceptual problem: http://bugs.python.org/issue13322
Sure, it can get tricky to reason about. But - the whole point of libraries like io is to encapsulate common solutions to tricky things - so that we don't have a hundred incompatible not-quite-the-same layers sitting on top. Right now select + BufferedIOReader is plain buggy regardless of non-blocking or not... ; I'd like to fix that - for instance, if select consulted the buffer somehow and returned immediately if the buffer had data, that would be an improvement (as select doesn't say *how much* data can be read).
An alternative would be being able to drain the buffer with no underlying I/O calls at all, then use select + read1, then rinse and repeat.
Have you tried peek()?
Per http://docs.python.org/3.2/library/io.html#io.BufferedReader.peek peek may cause I/O. Only one, but still you cannot control it. -Rob -- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Cloud Services