python-committers

Download

python-committers@python.org

April 2020

24 participants
12 discussions

The Night’s Watch is Fixing the CIs in the Darkness for You
by Victor Stinner April 7, 2020

April 7, 2020

Hi, tl; dr Maybe the 1090 pending pull requests will have to be rebased on master to be able to be merged, unless we enhance our CI to always test PRs after rebasing them on the master branch. I'm not sure at this point. -- FYI the Night’s Watch is still fixing the CIs in the darkness for you :-) In short, most CIs should now be green in all branches (especially CIs running on pull requests). I'm only aware of a Refleak issue on 3.x (test_threading, more about that above). Last days were more busy than usual. Here are issues in a random order. I had to update the OpenSSL version in the CI configuration which lives in the same Git repository than Python code base. Since our CI currently runs on unmodified PRs, maybe the 1090 pending pull requests must now be rebased manually on the master branch to get these CI configuration updates. Otherwise, the CI would fail which prevents to merge a PR. I'm not sure at this point. At least, old PRs with a green CI can be merged. But problems are likely to arise if a reviewer requires changes and the CI is re-run on the update PR. The PR may pick the old CI configuration (old OpenSSL version) and fail badly... Having to rebase 1090 PRs would be very unfortunate... Maybe one solution would be to redesign CIs running on PRs to always rebase them on the up-to-date master branch. There are services like https://mergify.io/ which implement such workflow. Currently, we test "outdated" master: there is a risk that tests fail once the change is rebased on top of the up-to-date master, whereas tests passed on the old master... Note: fixing multitestssl.py to update the URL of old OpenSSL tarballs doesn't work: again, old PRs don't get magically fixes pushed to master. -- concurrent.futures no longer uses daemon threads. It causes a reference leak in test_asyncio. Kyle Stanley managed to fix it in test_asyncio: https://bugs.python.org/issue40115 -- Adding a module state to the _abc module triggered a reference leak in test_threading: https://bugs.python.org/issue40149 This leak is very unlikely to be a new bug. It's just that previously it was hidden by the fact that all interpreters used the same module instance. regrtest -R 3:3 ignores the 18550 references that Python leaks at exit: it only checks if the total references number changes before/after a test run. When we *fix* modules to get isolated module instances, we start to see old leaks as "new" leaks. I have a pending workaround, but I'm first trying to investigate why two garbage collections are needed to fix the leak... -- The URL of the OpenSSL 1.1.1d moved which broke GitHub actions jobs, Azure Pipelines and Travis CI. Not all CIs failed at once because of different caches with different lifetime. * https://bugs.python.org/issue40146 Azure * https://bugs.python.org/issue40162 Travis CI * https://bugs.python.org/issue40125 multissltests I also proposed to modify multissltests to attempt to get the tarball from a different URL. Christian Heimes proposed a fix: https://bugs.python.org/issue40163 -- I enhanced tests to detect when a child process does crash. Previously, only FreeBSD *indirectly* detected such crash by the creation of coredump files. test_builtin started to fail on Solaris because the main test_builtin process was killed by SIGHUP. I already fixed a similar bug in test_pty last December by registering a signal handle to ignore SIGHUP. I copied the fix to test_builtin: https://bugs.python.org/issue40140 The commit also fix an AIX issue in the same test. -- After Python 3.9.0 alpha 5 release, I fixed a reference leak in test_importlib which was introduced by the recent work on converting C extension modules to multiphase initialization (PEP 489): https://bugs.python.org/issue40050 It's a subtle issue between importlib and _weakref. See the issue for the technical analysis. I also fixed os.getgrouplist(): https://bugs.python.org/issue40014 The macOS job on GitHub PRs failed because of this issue. In fact, the bug existed for a long time, but macOS 10.15 (Catalina) added more system groups to users which triggered the bug. os.getgrouplist() only failed if the user has more than ... 17 groups :-) Funny issue. -- To be able to backport security fixes to Pyhon 3.5 and Python 3.6, I had to fix the "Codecov patch" job which failed: https://bugs.python.org/issue40156 I copied the Codecov configuration from master. Sadly, all 3.5 and 3.6 PRs have to be rebased to retrieve these fixes, otherwise, they cannot be merged. -- To finish, you can find general info about Python CIs at: https://pythondev.readthedocs.io/ci.html Victor -- Night gathers, and now my watch begins. It shall not end until my death.

5 6

Vote to promote Dong-hee Na
by Victor Stinner April 6, 2020

April 6, 2020

https://discuss.python.org/t/vote-to-promote-dong-hee-na/3794 Victor -- Night gathers, and now my watch begins. It shall not end until my death.

1 2