Guido van Rossum writes:

Your solution (a counter) seems fine except I think perhaps the close() call should not raise IOError -- instead, it should set a flag so that the thread that makes the counter go to zero can close the thread (after all the file got closed while it was being used).

I agree with Gregory: we should be explicit about what happens. I wonder what we would gain from that approach - apart from encouraging dangerous coding practices :) It also depends how far we want to go: I am merely proposing to fix the crashes, do we want to provide a "smarter" close() variation that does what you suggest for people that want (or need) to take the risk?

There are of course other concurrency issues besides close -- what if two threads both try to do I/O on the file? What will the C stdio library do in that case? Are stdio files thread-safe at the C level?

According to the glibc documentation, at http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html : « The POSIX standard requires that by default the stream operations are atomic. I.e., issuing two stream operations for the same stream in two threads at the same time will cause the operations to be executed as if they were issued sequentially. The buffer operations performed while reading or writing are protected from other uses of the same stream. To do this each stream has an internal lock object which has to be (implicitly) acquired before any work can be done. » So according to POSIX rules it should be perfectly safe. In any case, someone would have to try my patch under Windows and OS X and see if test_file.py passes without crashing. Regards Antoine.

I've reviewed the patch on http://bugs.python.org/issue815646 and have uploaded my modified version (mostly test improvements and some formatting to keep C code under 80 columns with proper 8 space tabs). I would have committed it already but I have a sneaking suspicion that its unit test will barf on windows since it could depend on some posix-like file system semantics. Could someone with a windows build environment please test it as asked in the issue and report back in the tracker? thanks! -gps On Wed, Apr 2, 2008 at 11:47 AM, Guido van Rossum wrote:

On Wed, Apr 2, 2008 at 3:17 AM, Antoine Pitrou wrote:

...

Guido van Rossum writes:

...

Your solution (a counter) seems fine except I think perhaps the close() call should not raise IOError -- instead, it should set a flag so that the thread that makes the counter go to zero can close the thread (after all the file got closed while it was being used).

I agree with Gregory: we should be explicit about what happens. I wonder what we would gain from that approach - apart from encouraging dangerous coding practices :) It also depends how far we want to go: I am merely proposing to fix the crashes, do we want to provide a "smarter" close() variation that does what you suggest for people that want (or need) to take the risk?

I also agree with Gregory now -- at least on the issue of fclose().

I think that for other (non-closing) operations we should be fine, given the Posix requirement that streams have an internal lock. While multiple threads reading from a file sounds insane, multiple files writing to a file is pretty common (think of stdout or stderr) and should be supported.

...

...

There are of course other concurrency issues besides close -- what if two threads both try to do I/O on the file? What will the C stdio library do in that case? Are stdio files thread-safe at the C level?

According to the glibc documentation, at

http://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.html:

...

« The POSIX standard requires that by default the stream operations are atomic. I.e., issuing two stream operations for the same stream in two threads at the same time will cause the operations to be executed as if they were issued sequentially. The buffer operations performed while reading or writing are protected from other uses of the same stream. To

do

...

this each stream has an internal lock object which has to be (implicitly) acquired before any work can be done. »

So according to POSIX rules it should be perfectly safe.

Cool.

...

In any case, someone would have to try my patch under Windows and OS X and see if test_file.py passes without crashing.

I know Windows has internal locks on stdio. I trust that OSX, being a BSD descendant, is posix compliant. So I'm not worried about these.

+1 on your patch, assuming some other developer reviews it and submits it.

-- --Guido van Rossum (home page: http://www.python.org/~guido/http://www.python.org/%7Eguido/ ) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org