[Python-ideas] Atomic file.get(offset, length)

[Guido van Rossum]

On Mon, Jul 23, 2012 at 2:52 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 22 Jul 2012 17:16:57 -0700
> Guido van Rossum <guido at python.org> wrote:
>
>> On Sun, Jul 22, 2012 at 4:47 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > On Sun, 22 Jul 2012 15:36:44 -0700
>> > Guido van Rossum <guido at python.org> wrote:
>> >> On Sun, Jul 22, 2012 at 2:25 PM, Victor Stinner
>> >> <victor.stinner at gmail.com> wrote:
>> >> >> "man pread" on OS/X suggests it exists there too
>> >> >
>> >> > "man pread" or "import os; help(os.pread" ;-) pread() and pwrite()
>> >> > have been added to Python 3.3.
>> >>
>> >> Awesome. :-) But does the io module offer an API that uses it? It's
>> >> kind of awkward to have to call os.pread() with stream.fileno() as an
>> >> argument.
>> >
>> > It doesn't. I guess we could add an "offset" keyword-only argument to
>> > read() and write(), but then we need to provide a Windows
>> > implementation as well (it seems using overlapped I/O with
>> > ReadFile() / WriteFile() could make it possible). Also, I'm not sure it
>> > makes sense for buffered I/O, or only unbuffered.
>>
>> Given that the use case is to avoid race conditions when more than one
>> thread is doing random-access reads on the same open file, I think it
>> makes some sense to implement it for both buffered and unbuffered
>> streams -- and even for text streams, since those support seek() as
>> well, so the race condition exists for those too.
>>
>> But note that the pread() man page (at least the one I checked :-)
>> specifies that pread() doesn't affect the file pointer. So I suppose
>> it should also not affect the buffer. That may make it hard to
>> implement it for text streams (which IIRC rely quite heavy on
>> buffering for their implementation), but it should be easy for
>> buffered streams: it should just be passed on to the underlying
>> unbuffered stream.
>
> Indeed, it should not affect the buffer. That's why I'm questioning the
> addition of this feature to buffered streams (whose whole point is their
> implicit buffer management). Also, there are implementation subtleties
> when e.g. reading from an area which overlaps the current buffer :-)
>
> As you pointed out, I think a reasonable solution to the race condition
> problem is to use several file descriptors. It may not work so well if
> you also write to the file, though.

If you write and read a file from multiple threads you're crazy.

-- 
--Guido van Rossum (python.org/~guido)