[Python-Dev] Tests failing on Windows with TESTFN

[Chris Jerdonek]

On Mon, Jul 30, 2018 at 8:46 AM, Tim Golden <mail at timgolden.me.uk> wrote:
> On 30/07/2018 16:41, Nick Coghlan wrote:
>>
>> On 29 July 2018 at 03:20, Tim Golden <mail at timgolden.me.uk> wrote:
>>>
>>> I think that was my starting point: rather than develop increasingly
>>> involved and still somewhat brittle mechanisms, why not do what you'd
>>> naturally do with a new test and use tempfile? I was expecting someone to
>>> come forward to highlight some particular reason why the TESTFN approach
>>> is
>>> superior, but apart from a reference to the possibly cost of creating a
>>> new
>>> temporary file per test, no-one really has.
>>
>>
>> For higher level modules, "just use tempfile to create a new temporary
>> directory, then unlink it at the end" is typically going to be a good
>> answer (modulo the current cleanup issues that Jeremy is reporting,
>> but ideally those will be fixed rather than avoided, either by
>> improving the way the module is being used, or fixing any underlying
>> defects).

If there's a desire to use tempfile, another option is to have
tempfile create the temp files inside the temporary directory the test
harness creates specifically for testing -- using the "dir" argument
to many of tempfile's functions.

Here is where the process-specific temp directory is created for
testing (inside test.libregrtest.main):
https://github.com/python/cpython/blob/9045199c5aaeac9b52537581be127d999b5944ee/Lib/test/libregrtest/main.py#L511

This would also facilitate the clean-up of any leftover temp files.

Again, I think it would be best to use any tempfile functions behind
one or more test-support functions, so the choice of location, etc.
can be changed centrally without needing to modify code everywhere.

--Chris

>>
>> For lower level modules though, adding a test suite dependency on
>> tempfile introduces a non-trivial chance of adding an operational
>> dependency from a module's test suite back to the module itself. When
>> that happens, module regressions may show up as secondary failures in
>> tempfile that then need to be debugged, rather than as specific unit
>> test failures that point you towards exactly what you broke.
>>
>> Cheers,
>> Nick.
>>
>
> Thanks Nick; I hadn't thought about the possible interdependency issue.
>
> I think for the moment my approach will be to switch to support.unlink
> wherever possible to start with. Before introducing other (eg tempfile)
> changes, this should at least narrow the issues down. I've made a start on
> that (before inadvertently blowing away all the changes since my
> hours-previous commit!)
>
> If further changes are necessary then I'll probably look case-by-case to see
> whether a tempfile or some other solution would help.
>
> That said, that's potentially quite a lot of change -- at least in terms of
> files changed if not strictly of functionality. So I'm thinking of
> trickle-feeding the changes through as people will understandably baulk at a
> patchbomb (PR-bomb?) hitting the codebase all at once.
>
> TJG
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com