Python-Dev May 2018

python-dev@python.org

103 participants
88 discussions

by Serhiy Storchaka

Does anyone has the full copy of the PyXML repository, with the complete history? This library was included in Python 2.1 as the xml package and is not maintained as a separate project since 2004. It's home on SourceForge was removed. I have found sources of the last PyXML version (0.8.4), but without history. I'm trying to figure out some intentions and fix possible bugs in the xml package. The history of all commits could help.

2 years, 7 months

How to watch buildbots?

by Victor Stinner

Hi, I would like to delegate the maintenance task "watch buildbots", since I'm already very busy with many other maintenance tasks. I'm looking for volunteers to handle incoming emails on buildbot-status. I already started to explain to Pablo Galindo Salgado how to do that, but it would be great to have at least two people doing this task. Otherwise, Pablo wouldn't be able to take holiday or just make a break for any reason. Buildbots are evil beast which require care every day. Otherwise, they quickly turn red and become less useful :-( It seems like the first blocker issue is that we have no explicit documentation "how to deal with buildbots?" (the devguide documentation is incomplete, it doesn't explain what I'm explaining below). Let me start with a few notes of how I watch buildbots. I'm getting buildbot notifications on IRC (#python-dev on Freenode) and on the buildbot-status mailing list: https://mail.python.org/mm3/mailman3/lists/buildbot-status.python.org/ When a buildbot fails, I look at tests logs and I try to check if an issue has already been reported. For example, search for the test method in title (ex: "test_complex" for test_complex() method). If no result, search using the test filename (ex: "test_os" for Lib/test/test_os.py). If there is no result, repeat with full text searchs ("All Text"). If you cannot find any open bug, create a new one: * The title should contain the test name, test method and the buildbot name. Example: " test_posix: TestPosixSpawn fails on PPC64 Fedora 3.x". * The description should contain the link to the buildbot failure. Try to identify useful parts of tests log and copy them in the description. * Fill the Python version field (ex: "3.8" for 3.x buildbots) * Select at least the "Tests" Component. You may select additional Components depending on the bug. If a bug was already open, you may add a comment to mention that there is a new failure: add at least a link to buildbot name and a link to the failure. And that's all! Simple, isn't it? At this stage, there is no need to investigate the test failure. To finish, reply to the failure notification on the mailing list with a very short email: add a link to the existing or the freshly created issue, maybe copy one line of the failure and/or the issue title. Recent bug example: https://bugs.python.org/issue33630 -- Later, you may want to analyze these failures, but I consider that it's a different job (different "maintenance task"). If you don't feel able to analyze the bug, you may try to find someone who knows more than you about the failure. For better bug reports, you can look at the [Changes] tab of a build failure, and try to identify which recent change introduced the regression. This task requires to follow recent commits, since sometimes the failure is old, it's just that the test fails randomly depending on network issues, system load, or anything else. Sometimes, previous tests have side effects. Or the buildbot owner made a change on the system. There are many different explanation, it's hard to write a complete list. It's really on a case by case basis. Hopefully, it's now more common that a buildbot failure is obvious and caused by a very specific recent changes which can be found in the [Changes] tab. -- If you are interested to help me on watching our CIs: please come on the python-buildbot(a)python.org mailing list! Introduce yourself and explain how do you plan to help. I may propose to mentor you to assist you the first weeks. As I wrote, maybe a first step would be to write down a documentation how to deal with buildbots and/or update and complete existing documentations. https://devguide.python.org/buildbots/ Victor

2 years, 7 months

[RELEASE] Python 3.7.0b5 bonus beta!

by Ned Deily

A 3.7 update: Python 3.7.0b5 is now the final beta preview of Python 3.7, the next feature release of Python. 3.7.0b4 was intended to be the final beta but, due to some unexpected compatibility issues discovered during beta testing of third-party packages, we decided to revert some changes in how Python's 3.7 Abstract Syntax Tree parser deals with docstrings; 3.7.0b5 now behaves like 3.6.x and previous releases (refer to the 3.7.0b5 changelog for more information). **If your code makes use of the ast module, you are strongly encouraged to test (or retest) that code with 3.7.0b5, especially if you previously made changes to work with earlier preview versons of 3.7.0.** As always, please report issues found to bugs.python.org as soon as possible. Please keep in mind that this is a preview release and its use is not recommended for production environments. Attention macOS users: there is now a new installer variant for macOS 10.9+ that includes a built-in version of Tcl/Tk 8.6. This variant is expected to become the default version when 3.7.0 releases. Check it out! The next (and final, we hope!) preview release will be the release candidate which is now planned for 2018-06-11, followed by the official release of 3.7.0, now planned for 2018-06-27. You can find Python 3.7.0b5 and more information here: https://www.python.org/downloads/release/python-370b5/ -- Ned Deily nad(a)python.org -- []

2 years, 7 months

Forward of moderated message

by python-list-bounces＠python.org

2 years, 7 months

Re: [Python-Dev] Compact GC Header

by Serhiy Storchaka

29.05.18 17:15, Steve Dower пише: > Looks like it breaks the 3.7 ABI, which is certainly not allowed at this > time. But it’s not a limited API structure, so no problem for 3.8. Looks like it breaks only extensions that use private macros _PyObject_GC_TRACK, _PyObject_GC_UNTRACK and _PyObject_GC_IS_TRACKED. Those that use only public functions PyObject_GC_Track() and PyObject_GC_UnTrack() shouldn't be affected.

2 years, 7 months

Troubles to merge changes in the 2.7 branch: PR "out-of-date" branch

by Victor Stinner

Hi, Since one or two weeks, I noticed that it's difficult to merge pull requests into the 2.7 branch. If a different commit is pushed in the meanwhile (if a different PR has been merged), the 2.7 branch diverges and the PR is immediately marked as "This branch is out-of-date with the base branch" and the "Squash and Merge" button is disabled (grey). For example my PR https://github.com/python/cpython/pull/7120 which changes Lib/test/regrtest.py cannot be merged because a commit touching the documentation (Doc/ directory) has been merged after I posted my PR and before the CI completed. I don't see the same behavior on the master branch. Is the 2.7 branch configured as more strict? Victor

2 years, 7 months

Python 3.7.0 updated schedule: beta 5 cutoff in 24 hours

by Ned Deily

Here's an update on the 3.7.0 endgame. As announced several days ago, we made the difficult decision to hold back on 3.7.0rc1 due primarily to some unexpected difficulties being seen downstream due to changes in how docstrings were handled in 3.7.0 (details below). After some discussions about various approaches, we agreed on a solution that should minimize downstream impact without losing all the benefits of the existing 3.7 changes. Thanks to a lot of work over the long weekend by a number of people that solution is now merged in the 3.7 branch. In parallel with that, a number of people spent a lot of time looking at CI and buildbot test failures, mostly intermittent ones. As a result, a number of actual bugs were fixed and also problems with a number of tests were fixed which should make the test suite more robust. All this is good news. Primarily because of the important user-facing changes made with the AST docstring API, I feel we need to do one more beta release before we are ready for the release candidate. About 24 hours from now, approximately 2018-05-30 18:00UTC, I plan to tag and start manufacturing 3.7.0b5. This will be a short beta cycle, aimed mainly at users of the AST API so they can recheck that their packages with 3.7.0. Assuming all goes well, we will then plan to tag 3.7.0rc1 on 2018-06-11 and 3.7.0 final on 2017-06-27. I am also rescheduling 3.6.6rc1 and 3.6.6 final to match the new 3.7.0 dates. All fixes that have been merged into the 3.7 branch as of cutoff tomorrow will be in 3.7.0b5 and fixes merged afterwards will be in 3.7.0rc1 up to its cutoff point. After 3.7.0rc1 cutoff, 3.7 merges will appear in 3.7.1. Please continue to exercise diligence when deciding whether a change is appropriate for 3.7; as a rule of thumb, treat the 3.7 branch as if it were already released and in maintenance mode. Please also pay attention to CI test failures and buildbot test failures and see if you can help resolve them. I want to thank everyone who has been involved so far in helping us through this endgame and who have given up their personal time to work on making Python better. I, for one, am deeply grateful. 2018-05-30 3.7.0b5 2018-06-11 3.7.0rc1 & 3.6.6rc1 2018-06-27 3.7.0final & 3.6.6final --Ned On May 25, 2018, at 01:33, Ned Deily <nad(a)python.org> wrote: > On May 24, 2018, at 03:23, Ned Deily <nad(a)python.org> wrote: >> On May 23, 2018, at 09:13, Ned Deily <nad(a)python.org> wrote: >>> On May 23, 2018, at 07:45, Serhiy Storchaka <storchaka(a)gmail.com> wrote: >>>> Is it possible to add yet one beta instead? >>>> CI was broken for few latest days, tests are not passed on my computer still (and fail on some buildbots), updating What's New exposed new features which need additional testing (and maybe fixing or reverting), and I'm not comfortable about some changes which would be harder to fix after the release. >>> it is possible but there's no point in doing either another beta or a release candidate until we understand and address the current blocking issues, like the major buildbot failures. We have another 24 hours until rc1 was planned to be tagged. Let's keep working on the known issues and we will make a decision then. >> An update: thanks to a lot of effort over the past day by a number of >> people (including Victor, Serhiy, Christian, Zach, and others I'm sure >> I'm forgetting - my apologies), we have addressed all of the "release >> blocker" issues and all but one of the persistent failures on the 3.7 >> stable buildbots. We should have the couple of remaining "deferred >> blockers" including the remaining stable buildbots in green status by >> later today. At that point, we will be ready to tag 3.7.0rc1 and begin >> producing the release candidate artifacts. > Further update: some good news and some changes. > > The good news is that we have resolutions for all of the previous release and deferred blockers. Thanks to a number of people for continuing to help get the remaining stable buildbot issues taken care of along with some lingering bugs. > > The not-quite-as-good news is that we have had more discussions about some unexpected incompatibilities that have shown up with downstream user testing with the AST docstrings changes in place (see bpo-32911). We have had some previous discussions about the expected user impact and, earlier in the beta phase, I encouraged us to stay the course with the feature as implemented. But I am now persuaded that we owe it to our users to take one more look at this to make sure we do not force them to make changes for 3.7 and then once again for 3.8. More details are in the bug tracker issue; I strongly encourage those of us who have been involved with this to "vote" there on the proposal to either (A) proceed with the release of the current implementation in 3.7.0 or (B) revert the feature in 3.7.0 and retarget for 3.8. Should the consensus be to revert (B), we will plan to have one more fast-track beta release (b5) prior to the release candidate, in order to allow downstream users to tes > t their projects with the removal. PLEASE, keep the discussion about this on the bug tracker (and not here!) and keep it brief so we can move forward quickly. Because of the upcoming 3-day holiday weekend in some countries, I have set Tue 2018-05-29 18:00 UTC as a cutoff for "voting" but, if a clear consensus emerges earlier, we will likely cut the discussion short. So chime in now on the bug tracker if you have a stake in this issue. > > https://bugs.python.org/issue32911 > > This does mean that yesterday's "last chance" has been extended a bit, at most a few days. I will let you know as soon as we have made a decision about the feature and will provide updated 3.7.0 schedule info at that time. -- Ned Deily nad(a)python.org -- []

2 years, 7 months

Compact GC Header

by INADA Naoki

Hi, all. I hacked GC module and managed to slim PyGC_Head down from 3 words to 2 words. It passes test suite, while some comments and code cleanup is needed before merge. * https://bugs.python.org/issue33597 * https://github.com/python/cpython/pull/7043 I want to merge it after 3.7.0rc1 and buildbots are stable, if Antoine or other GC expert accept it. I estimate it reduces 5% memory usage (RSS) and negligible performance difference. If someone interested in it, please test and benchmark it on GC heavy application. Regards, -- INADA Naoki <songofacandy(a)gmail.com>

2 years, 7 months

When tp_clear returns non-zero?

by Serhiy Storchaka

The tp_clear field of PyTypeObject has type inquiry. It is a pointer to a function that takes PyObject * and return int. typedef int (*inquiry)(PyObject *); I'm interesting what the result of this function means. In what cases it can return non-zero, and can it set an exception? Currently tp_clear() is called in a single place, and its result is ignored. All tp_clear implementations in the stdlib always return 0.

2 years, 7 months

macOS: minimum supported version?

by Victor Stinner

Hi, Ned Deily closed old bugs reported on the macOS Tiger buildbot, since this buildbot has been retired 3 months ago (the builders are still visible online, but last builds were 3 months ago). It seems like the oldest macOS buildbot is now macOS El Capitan (macOS 10.11, 2015). Does it mean that the minimum officially supported macOS version is now macOS 10.11 El Capitain? For me, to get an official "full" support, we need a buildbot. Without buildbot, we can only provide a weaker "best-effort" support. Otherwise, the risk of regression is too high. I failed to find any official and obvious list of CPython supported platforms, so I wrote my own list: http://vstinner.readthedocs.io/cpython.html#supported-platforms My first motivation for this list was to get a simple list of supported Windows versions, because I'm unable to follow Windows lifecycle (the PEP 11 has a vague statement about Windows which requires to follow Windows end of life for each Windows release). Victor

2 years, 7 months

← Newer
1
2
3
4
5
6
7
8
9
Older →

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev May 2018