[Python-Dev] Tests failing on Windows with TESTFN

[eryk sun]

On Sun, Jul 29, 2018 at 9:13 AM, Tim Golden <mail at timgolden.me.uk> wrote:
>
> For an example:
>
> http://tjg.org.uk/test.log
>
> Thinkpad T420, 4Gb, i5, SSD
>
> Recently rebuilt and reinstalled: Win10, VS2017, TortoiseGit, standard
> Windows Antimalware, usual developer tools. That particular run was done
> with the laptop unattended (ie nothing else going on at the front end).
> But the problem is certainly not specific to this laptop.

On my last run I had one test directory that wasn't removed properly,
but nothing like the flood of EACCES and ERROR_ACCES_DENIED errors you
have in that log. Then again, I had Defender disabled by policy. I'll
enable it and add exceptions for my source and build directories, and
see how it goes.

It would be nice if OSError instances always captured the last Windows
error and NT status values when instantiated. We have no guarantees
that these values are valid, but in many contexts they are. In the
case of a test log, it would certainly help to clarify errors without
having to individually investigate each one.

For example, trying to open a directory as a file is a common error,
but all Python tells us on Windows is that it failed with EACCES. In
this case the last Windows error is ERROR_ACCESS_DENIED, which doesn't
help, but the last NT status code is STATUS_FILE_IS_A_DIRECTORY
(0xc00000ba).
Here's a file opener that adds last_winerror and last_ntstatus values.

    import os
    ntdll = ctypes.WinDLL('ntdll')
    kernel32 = ctypes.WinDLL('kernel32')

    def nt_opener(path, flags):
        try:
            return os.open(path, flags)
        except OSError as e:
            last_ntstatus = ntdll.RtlGetLastNtStatus()
            last_winerror = kernel32.GetLastError()
            e.last_ntstatus = last_ntstatus & 2**32 - 1
            e.last_winerror = (last_winerror if e.winerror is None
                else e.winerror)
            if e.errno is not None or e.winerror is not None:
                # hack the last error/status into the error message
                e.strerror = '[Last NtStatus {:#08x}] {}'.format(
                    e.last_ntstatus, e.strerror or '')
                if e.winerror is None:
                    e.strerror = '[Last WinError {}] {}'.format(
                        e.last_winerror, e.strerror or '')
            raise e from None

Opening a directory as a file:

    >>> open('C:/Windows', opener=nt_opener)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 17, in nt_opener
      File "<stdin>", line 3, in nt_opener
    PermissionError: [Errno 13] [Last WinError 5] [Last NtStatus 0xc00000ba]
    Permission denied: 'C:/Windows'