On Fri, Jul 27, 2018 at 6:41 AM, Giampaolo Rodola' <g.rodola@gmail.com> wrote:
Being TESTFN a global name it appears not suited for parallel testing.
It was designed for parallel testing though: # Disambiguate TESTFN for parallel testing, while letting it remain a valid # module name. TESTFN = "{}_{}_tmp".format(TESTFN, os.getpid()) https://github.com/python/cpython/blob/aee632dfbb0abbc0d2bcc988c43a736afd568...
Windows makes this more noticeable than POSIX as it's more strict, but accessing the same file from 2 unit tests is technically a problem regardless of the platform. I don't think that means we should ditch TESTFN in favor of tempfile.mktemp() though. Instead the file cleanup functions (support.unlink() and support.rmtree()) may be more clever and (important) they should always be used in setUp / tearDown. For instance, the traceback you pasted refers to a test class which doesn't do this.
The "test_file" test method referenced in the traceback calls os.remove(TESTFN) in finally blocks preceding its calls to open(TESTFN, "wb"), and inspecting the method shows that it must have been able to open TESTFN earlier in the method (the same test method uses TESTFN multiple times): https://github.com/python/cpython/blob/aee632dfbb0abbc0d2bcc988c43a736afd568... So I think one should investigate what can be causing the error / how it can be happening. TESTFN uses the pid of the process, so it doesn't seem like another test case could be interfering and opening the same TESTFN while the "test_file" test method is running. On Stack Overflow, there are some comments suggesting that in some cases os.remove() doesn't always complete right away (e.g. because of anti-malware software briefly holding a second reference). --Chris
In psutil I've had occasional Windows failures like this for years then I got tired of it and came up with this: https://github.com/giampaolo/psutil/blob/1b09b5fff78f705dfb42458726ff9789c26... ...which basically aggressively retries os.unlink or shutil.rmtree for 1 sec in case of (any) error, and I haven't had this problem since then.
I suppose test.support's unlink() and rmtree() can do something similar, maybe just by using a better exception handling, and all unit tests should use them in setUp / tearDown. I think this will diminish the occasional failures on Windows, although not completely.
-- Giampaolo - http://grodola.blogspot.com _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.co...