[New-bugs-announce] [issue35804] v3.6.8 _ctypes win32 compiled with pgo crash

[Anselm Kruis]

New submission from Anselm Kruis <a.kruis at science-computing.de>:

During the QA for Stackless 3.6.8 I observed a crash in _ctypes compiled for win32 with PGO, that also exists with plain C-Python v3.6.8. I didn't check other versions yet.

OS: Win7 (64bit)
Compiler: Visual Studio 2017 professional 15.9.5

How to reproduce

1. Checkout v3.6.8

I'm using the git-bash
$ git status
HEAD detached at v3.6.8
nothing to commit, working tree clean

2. Clean the sandbox and compile. It is sufficient to use a short pgo-job, but the default pgo-job works a well (but takes much more time).

$ cd PCbuild/
$ rm -rf obj win32 amd64
$ PYTHON=/e/Pythons/3.6.4-64/python.exe cmd //c build.bat --pgo-job "-m test --pgo test_ctypes" 2>&1 | tee build.log

The file "build.log" is attached. Nothing conspicuous in it.

3. Run the test case ctypes.test.test_win32.WindowsTestCase.test_callconv_1
Sometimes the test passes, sometimes it fails with a Segmentation Fault.

$ win32/python.exe -X faulthandler -m ctypes.test.test_win32 WindowsTestCase.test_callconv_1
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

$ win32/python.exe -X faulthandler -m ctypes.test.test_win32 WindowsTestCase.test_callconv_1
Windows fatal exception: access violation

Current thread 0x00001574 (most recent call first):
  File "C:\build\python36\lib\unittest\case.py", line 178 in handle
  File "C:\build\python36\lib\unittest\case.py", line 733 in assertRaises
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 20 in test_callconv_1
  File "C:\build\python36\lib\unittest\case.py", line 605 in run
  File "C:\build\python36\lib\unittest\case.py", line 653 in __call__
  File "C:\build\python36\lib\unittest\suite.py", line 122 in run
  File "C:\build\python36\lib\unittest\suite.py", line 84 in __call__
  File "C:\build\python36\lib\unittest\suite.py", line 122 in run
  File "C:\build\python36\lib\unittest\suite.py", line 84 in __call__
  File "C:\build\python36\lib\unittest\runner.py", line 176 in run
  File "C:\build\python36\lib\unittest\main.py", line 256 in runTests
  File "C:\build\python36\lib\unittest\main.py", line 95 in __init__
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 165 in <module>
  File "C:\build\python36\lib\runpy.py", line 85 in _run_code
  File "C:\build\python36\lib\runpy.py", line 193 in _run_module_as_main
Segmentation fault


4. I observed another variant of the crash, if I run all tests in test_ctypes

$ cmd //c rt.bat -q -v test_ctypes 2>&1 | tee test_ctypes.log

The file "test_ctypes.log" is attached. Relevant content:
test_callconv_1 (ctypes.test.test_win32.WindowsTestCase) ... XXX lineno: 124, opcode: 0
ERROR
test_callconv_2 (ctypes.test.test_win32.WindowsTestCase) ... XXX lineno: 124, opcode: 0
ERROR
test_variant_bool (ctypes.test.test_wintypes.WinTypesTest) ... test test_ctypes failed
ok

======================================================================
ERROR: test_callconv_1 (ctypes.test.test_win32.WindowsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 27, in test_callconv_1
    self.assertRaises(ValueError, IsWindow, 0, 0, 0)
  File "C:\build\python36\lib\unittest\case.py", line 733, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "C:\build\python36\lib\unittest\case.py", line 157, in handle
    if not _is_subtype(self.expected, self._base_type):
  File "C:\build\python36\lib\unittest\case.py", line 124, in _is_subtype
    if isinstance(expected, tuple):
SystemError: unknown opcode

======================================================================
ERROR: test_callconv_2 (ctypes.test.test_win32.WindowsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\build\python36\lib\ctypes\test\test_win32.py", line 36, in test_callconv_2
    self.assertRaises(ValueError, IsWindow, None)
  File "C:\build\python36\lib\unittest\case.py", line 733, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "C:\build\python36\lib\unittest\case.py", line 157, in handle
    if not _is_subtype(self.expected, self._base_type):
  File "C:\build\python36\lib\unittest\case.py", line 124, in _is_subtype
    if isinstance(expected, tuple):
SystemError: unknown opcode

----------------------------------------------------------------------


I had a quick look at  _call_function_pointer() in Modules/_ctypes/callproc.c, but I didn't see anything obvious. A very speculative first guess is the calling convention of ffi_call() or a related function written in (inline) assembly.

Work around: compile _ctypes for win32 without PGO.

----------
components: ctypes
files: build.log
messages: 334202
nosy: anselm.kruis
priority: normal
severity: normal
status: open
title: v3.6.8 _ctypes win32 compiled with pgo crash
type: crash
versions: Python 3.6
Added file: https://bugs.python.org/file48071/build.log

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35804>
_______________________________________