Need help with test_ctypes failing on Windows (test_load_dll_with_flags)
I have a large PR (https://github.com/python/cpython/pull/18239, for PEP 585) that's failing in the Azures pipeline on Win32 and Win64 only. My trusty assistant who has a Windows laptop couldn't reproduce the failure. Can I buy a hint from someone? Steve? The relevant failure output is: ====================================================================== ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll', winmode=0)] ---------------------------------------------------------------------- Traceback (most recent call last): File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass subprocess.check_output( File "d:\a\1\s\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "d:\a\1\s\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit status 1. ---------------------------------------------------------------------- -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
(Trusty assistant reporting in) I should also note that further up the output there is a FileNotFoundError for sqlite3.dll, perhaps it isn't built or is in the wrong place? Ethan On Mon, Apr 6, 2020, 7:19 PM Guido van Rossum <guido@python.org> wrote:
I have a large PR (https://github.com/python/cpython/pull/18239, for PEP 585) that's failing in the Azures pipeline on Win32 and Win64 only. My trusty assistant who has a Windows laptop couldn't reproduce the failure. Can I buy a hint from someone? Steve?
The relevant failure output is:
====================================================================== ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll', winmode=0)] ---------------------------------------------------------------------- Traceback (most recent call last): File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass subprocess.check_output( File "d:\a\1\s\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "d:\a\1\s\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit status 1.
----------------------------------------------------------------------
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY... Code of Conduct: http://python.org/psf/codeofconduct/
Hm, looking over the diff from a different perspective I think I have changed a few `__class_getitem__` methods that I shouldn't have. I'll see if that fixes it later tonight. On Mon, Apr 6, 2020 at 7:23 PM Ethan Smith <ethan@ethanhs.me> wrote:
(Trusty assistant reporting in) I should also note that further up the output there is a FileNotFoundError for sqlite3.dll, perhaps it isn't built or is in the wrong place?
Ethan
On Mon, Apr 6, 2020, 7:19 PM Guido van Rossum <guido@python.org> wrote:
I have a large PR (https://github.com/python/cpython/pull/18239, for PEP 585) that's failing in the Azures pipeline on Win32 and Win64 only. My trusty assistant who has a Windows laptop couldn't reproduce the failure. Can I buy a hint from someone? Steve?
The relevant failure output is:
====================================================================== ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll', winmode=0)] ---------------------------------------------------------------------- Traceback (most recent call last): File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass subprocess.check_output( File "d:\a\1\s\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "d:\a\1\s\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit status 1.
----------------------------------------------------------------------
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
Looking over the commit history for the PR ( https://github.com/python/cpython/pull/18239/commits), it looks like that specific Azure Pipelines failure did not start occurring until upstream/master was merged into the PR branch ( https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea7706054...). Therefore, I suspect that the failure is very likely unrelated to the PR; instead either an intermittent failure that was merged into master recently or a possible issue on Azure's end. For now, I'd suggest closing and re-opening the PR again tomorrow to see if the failure still occurs. Note: I'm also seeing the same exact failure occur in the following separate CPython PRs that were opened recently: https://github.com/python/cpython/pull/19403 https://github.com/python/cpython/pull/19402 https://github.com/python/cpython/pull/19399 Seeing as it was also occurring in entirely unrelated PRs, it seems to be unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this failure, so I'll open a new one for it. On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum <guido@python.org> wrote:
I have a large PR (https://github.com/python/cpython/pull/18239, for PEP 585) that's failing in the Azures pipeline on Win32 and Win64 only. My trusty assistant who has a Windows laptop couldn't reproduce the failure. Can I buy a hint from someone? Steve?
The relevant failure output is:
====================================================================== ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll', winmode=0)] ---------------------------------------------------------------------- Traceback (most recent call last): File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass subprocess.check_output( File "d:\a\1\s\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "d:\a\1\s\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit status 1.
----------------------------------------------------------------------
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY... Code of Conduct: http://python.org/psf/codeofconduct/
FWIW, this test is meant to verify that the old, unsafe DLL load logic still works. I suspect what has happened here is that a new VM image has been rolled out and another app has installed an incompatible _sqlite3.dll on PATH (most likely another copy of Python :) ), thereby proving why the old logic is unsafe. We've disabled the test for now, so if you merge and resubmit it should be fine. Now we just have to decide whether to disable this part of the test forever, or try and manipulate the test environment enough to make it pass (which I suspect is just setting PATH back to a sane value). Sorry for the inconvenience! Cheers, Steve On 07Apr2020 0420, Kyle Stanley wrote:
Looking over the commit history for the PR (https://github.com/python/cpython/pull/18239/commits), it looks like that specific Azure Pipelines failure did not start occurring until upstream/master was merged into the PR branch (https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea7706054...). Therefore, I suspect that the failure is very likely unrelated to the PR; instead either an intermittent failure that was merged into master recently or a possible issue on Azure's end. For now, I'd suggest closing and re-opening the PR again tomorrow to see if the failure still occurs.
Note: I'm also seeing the same exact failure occur in the following separate CPython PRs that were opened recently:
https://github.com/python/cpython/pull/19403 https://github.com/python/cpython/pull/19402 https://github.com/python/cpython/pull/19399
Seeing as it was also occurring in entirely unrelated PRs, it seems to be unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this failure, so I'll open a new one for it.
On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum <guido@python.org <mailto:guido@python.org>> wrote:
I have a large PR (https://github.com/python/cpython/pull/18239, for PEP 585) that's failing in the Azures pipeline on Win32 and Win64 only. My trusty assistant who has a Windows laptop couldn't reproduce the failure. Can I buy a hint from someone? Steve?
The relevant failure output is:
====================================================================== ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll', winmode=0)] ---------------------------------------------------------------------- Traceback (most recent call last): File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass subprocess.check_output( File "d:\a\1\s\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "d:\a\1\s\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit status 1.
----------------------------------------------------------------------
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
Thanks Steve for the solution and Kyle for confirming it wasn't me! After the latest merge from master my PR's tests all pass, so I'm happy. On Tue, Apr 7, 2020 at 3:33 AM Steve Dower <steve.dower@python.org> wrote:
FWIW, this test is meant to verify that the old, unsafe DLL load logic still works.
I suspect what has happened here is that a new VM image has been rolled out and another app has installed an incompatible _sqlite3.dll on PATH (most likely another copy of Python :) ), thereby proving why the old logic is unsafe.
We've disabled the test for now, so if you merge and resubmit it should be fine.
Now we just have to decide whether to disable this part of the test forever, or try and manipulate the test environment enough to make it pass (which I suspect is just setting PATH back to a sane value).
Sorry for the inconvenience!
Cheers, Steve
On 07Apr2020 0420, Kyle Stanley wrote:
Looking over the commit history for the PR (https://github.com/python/cpython/pull/18239/commits), it looks like that specific Azure Pipelines failure did not start occurring until upstream/master was merged into the PR branch ( https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea7706054...).
Therefore, I suspect that the failure is very likely unrelated to the PR; instead either an intermittent failure that was merged into master recently or a possible issue on Azure's end. For now, I'd suggest closing and re-opening the PR again tomorrow to see if the failure still occurs.
Note: I'm also seeing the same exact failure occur in the following separate CPython PRs that were opened recently:
https://github.com/python/cpython/pull/19403 https://github.com/python/cpython/pull/19402 https://github.com/python/cpython/pull/19399
Seeing as it was also occurring in entirely unrelated PRs, it seems to be unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this failure, so I'll open a new one for it.
On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum <guido@python.org <mailto:guido@python.org>> wrote:
I have a large PR (https://github.com/python/cpython/pull/18239, for PEP 585) that's failing in the Azures pipeline on Win32 and Win64 only. My trusty assistant who has a Windows laptop couldn't reproduce the failure. Can I buy a hint from someone? Steve?
The relevant failure output is:
======================================================================
ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest) [WinDLL('_sqlite3.dll', winmode=0)]
----------------------------------------------------------------------
Traceback (most recent call last): File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass subprocess.check_output( File "d:\a\1\s\lib\subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "d:\a\1\s\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *; import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit status 1.
----------------------------------------------------------------------
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
participants (4)
-
Ethan Smith -
Guido van Rossum -
Kyle Stanley -
Steve Dower