[Python-Dev] Thread-safe file objects, the return

[Guido van Rossum]

On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>  Guido van Rossum <guido <at> python.org> writes:
>  > Your solution (a counter) seems fine except I think perhaps the
>  > close() call should not raise IOError -- instead, it should set a flag
>  > so that the thread that makes the counter go to zero can close the
>  > thread (after all the file got closed while it was being used).
>
>  I agree with Gregory: we should be explicit about what happens. I wonder
>  what we would gain from that approach - apart from encouraging dangerous
>  coding practices :)
>  It also depends how far we want to go: I am merely proposing to fix the
>  crashes, do we want to provide a "smarter" close() variation that does what
>  you suggest for people that want (or need) to take the risk?

I also agree with Gregory now -- at least on the issue of fclose().

I think that for other (non-closing) operations we should be fine,
given the Posix requirement that streams have an internal lock. While
multiple threads reading from a file sounds insane, multiple files
writing to a file is pretty common (think of stdout or stderr) and
should be supported.

>  > There are of course other concurrency issues besides close -- what if
>  > two threads both try to do I/O on the file? What will the C stdio
>  > library do in that case? Are stdio files thread-safe at the C level?
>
>  According to the glibc documentation, at
>  http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html :
>
>  « The POSIX standard requires that by default the stream operations are
>  atomic. I.e., issuing two stream operations for the same stream in two
>  threads at the same time will cause the operations to be executed as if
>  they were issued sequentially. The buffer operations performed while
>  reading or writing are protected from other uses of the same stream. To do
>  this each stream has an internal lock object which has to be (implicitly)
>  acquired before any work can be done. »
>
>  So according to POSIX rules it should be perfectly safe.

Cool.

>  In any case, someone would have to try my patch under Windows and OS X and
>  see if test_file.py passes without crashing.

I know Windows has internal locks on stdio. I trust that OSX, being a
BSD descendant, is posix compliant. So I'm not worried about these.

+1 on your patch, assuming some other developer reviews it and submits it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)