[Python-ideas] Atomic file.get(offset, length)

Guido van Rossum guido at python.org
Mon Jul 23 02:16:57 CEST 2012


On Sun, Jul 22, 2012 at 4:47 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 22 Jul 2012 15:36:44 -0700
> Guido van Rossum <guido at python.org> wrote:
>> On Sun, Jul 22, 2012 at 2:25 PM, Victor Stinner
>> <victor.stinner at gmail.com> wrote:
>> >> "man pread" on OS/X suggests it exists there too
>> >
>> > "man pread" or "import os; help(os.pread" ;-) pread() and pwrite()
>> > have been added to Python 3.3.
>>
>> Awesome. :-) But does the io module offer an API that uses it? It's
>> kind of awkward to have to call os.pread() with stream.fileno() as an
>> argument.
>
> It doesn't. I guess we could add an "offset" keyword-only argument to
> read() and write(), but then we need to provide a Windows
> implementation as well (it seems using overlapped I/O with
> ReadFile() / WriteFile() could make it possible). Also, I'm not sure it
> makes sense for buffered I/O, or only unbuffered.

Given that the use case is to avoid race conditions when more than one
thread is doing random-access reads on the same open file, I think it
makes some sense to implement it for both buffered and unbuffered
streams -- and even for text streams, since those support seek() as
well, so the race condition exists for those too.

But note that the pread() man page (at least the one I checked :-)
specifies that pread() doesn't affect the file pointer. So I suppose
it should also not affect the buffer. That may make it hard to
implement it for text streams (which IIRC rely quite heavy on
buffering for their implementation), but it should be easy for
buffered streams: it should just be passed on to the underlying
unbuffered stream.

(For those jumping in the middle of the thread: I know it's past the
feature freeze, so these considerations are for 3.4. Also, os.pread()
is in 3.3.)

-- 
--Guido van Rossum (python.org/~guido)



More information about the Python-ideas mailing list