On Sat, Jul 28, 2018 at 11:20 AM Tim Golden <mail@timgolden.me.uk> wrote:
Although things have moved on since that discussion and test.support.unlink has grown some extra legs, all it's done really is to push the bump along the carpet for a bit. I've got a newly-installed Win10 machine with the typical MS Antivirus & TortoiseGitCache vying for locks with every other process. I've just done a full test run:
python -mtest -j0 -v > test.log
I, for one, would like to see that log. The issues you are have are fairly unique. Just check out the buildbot status page. I know that some of the workers are somewhat limited, but my worker (https://buildbot.python.org/all/#/workers/12) is running on dedicated hardware. Before https://bugs.python.org/issue15496 was applied, the errors you describe were indeed happening, but no longer.
Here's the thing. The TESTFN approach creates a directory per process test_python_<pid> and some variant of @test_<pid>_tmp inside that directory. But the same filename in the same directory is used for every test in that process. Now, leaving aside the particular mechanism by which Windows processes might be holding locks which prevent removal or re-addition, that already seems like an odd choice.
Well, since every test (well, test file) is run in its own process, a directory per process doesn't seem that odd. You seem to be focusing on 2 tests in particular, both of which have not been converted to use test.support.unlink for removing. The "trick" with test.support.unlink now is that it waits until the file has truly removed from the directory before continuing. Using this, in turn, would mean that following create *should* succeed without error. If test.support.unlink returns without warning, the removed file is really gone.
I think that was my starting point: rather than develop increasingly involved and still somewhat brittle mechanisms, why not do what you'd naturally do with a new test and use tempfile? I was expecting someone to come forward to highlight some particular reason why the TESTFN approach is superior, but apart from a reference to the possibly cost of creating a new temporary file per test, no-one really has.
*PLEASE*, don't use tempfile to create files/directories in tests. It is unfriendly to (Windows) buildbots. The current approach of directory-per-process ensures no test turds are left behind, whereas the tempfile solution slowly fills up my buildbot. Windows doesn't natively clean out the temp directory. Additionally, I'm working on a solution to allow the test machinery to use a given directory for test outputs to allow for a RAM-backed filesystem for temporary files. With this, I've had the full test suite (with -uall including bigmem) take ~5min. Having some tests use the Windows default test directory would break this. It is simply not feasible to have the entire Windows TEMP in RAM, but just Python's test suite usage is small enough (<6Gb). Another point, some tests require that the temporary filename resides on the same drive as the source directory (discovered in developing the above). This means that, Windows development would be restricted to the same drive as the user directory, if the per-directory approach isn't used. -- Jeremy Kloth