Mailman 3 January 2014 - python-committers

Code review tool (rietveld) bug
by Yury Selivanov 29 Jan '14

29 Jan '14

Hello, I'm having difficulty with replying to a message in rietveld. Here's a screenshot of the exception: http://goo.gl/70iQ5v (AttributeError at /review/20356/publish) Is this a known issue? Maybe I'm doing something wrong? Yury

6 14

[RELEASED] Python 3.3.4 release candidate 1
by Georg Brandl 29 Jan '14

29 Jan '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I'm reasonably happy to announce the Python 3.3.4 release candidate 1. Python 3.3.4 includes several security fixes and over 120 bug fixes compared to the Python 3.3.3 release. This release fully supports OS X 10.9 Mavericks. In particular, this release fixes an issue that could cause previous versions of Python to crash when typing in interactive mode on OS X 10.9. Python 3.3 includes a range of improvements of the 3.x series, as well as easier porting between 2.x and 3.x. In total, almost 500 API items are new or improved in Python 3.3. For a more extensive list of changes in the 3.3 series, see http://docs.python.org/3.3/whatsnew/3.3.html and for the detailed changelog of 3.3.4, see http://docs.python.org/3.3/whatsnew/changelog.html To download Python 3.3.4 rc1 visit: http://www.python.org/download/releases/3.3.4/ This is a preview release, please report any bugs to http://bugs.python.org/ The final version is scheduled to be released in two weeks' time, on or about the 10th of February. Enjoy! - -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and 3.3's contributors) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlLmDI4ACgkQN9GcIYhpnLAr6wCePRbHF80k5goV4RUDBA5FfkwF rLUAnRg0RpL/b6apv+Dt2/sgnUd3hTPA =Z4Ss -----END PGP SIGNATURE-----

3 2

[RELEASED] Python 3.4.0b3
by Larry Hastings 27 Jan '14

27 Jan '14

On behalf of the Python development team, I'm quite pleased to announce the third beta release of Python 3.4. This is a preview release, and its use is not recommended for production settings. Python 3.4 includes a range of improvements of the 3.x series, including hundreds of small improvements and bug fixes. Major new features and changes in the 3.4 release series include: * PEP 428, a "pathlib" module providing object-oriented filesystem paths * PEP 435, a standardized "enum" module * PEP 436, a build enhancement that will help generate introspection information for builtins * PEP 442, improved semantics for object finalization * PEP 443, adding single-dispatch generic functions to the standard library * PEP 445, a new C API for implementing custom memory allocators * PEP 446, changing file descriptors to not be inherited by default in subprocesses * PEP 450, a new "statistics" module * PEP 451, standardizing module metadata for Python's module import system * PEP 453, a bundled installer for the *pip* package manager * PEP 454, a new "tracemalloc" module for tracing Python memory allocations * PEP 456, a new hash algorithm for Python strings and binary data * PEP 3154, a new and improved protocol for pickled objects * PEP 3156, a new "asyncio" module, a new framework for asynchronous I/O Python 3.4 is now in "feature freeze", meaning that no new features will be added. The final release is projected for mid-March 2014. To download Python 3.4.0b3 visit: http://www.python.org/download/releases/3.4.0/ Please consider trying Python 3.4.0b3 with your code and reporting any new issues you notice to: http://bugs.python.org/ Enjoy! -- Larry Hastings, Release Manager larry at hastings.org (on behalf of the entire python-dev team and 3.4's contributors)

1 0

PEP 462: Workflow automation for CPython
by Nick Coghlan 27 Jan '14

27 Jan '14

Hey all, Rather than leaving my ideas undocumented until the language summit in April, I wrote up what I see as the critical issues in our current workflow and how I believe Zuul could help us resolve them as a PEP: http://www.python.org/dev/peps/pep-0462/ I don't think we should *do* anything about this until after PyCon US, but wanted to publish something that clearly explained my thinking rather than surprising people with it at the summit. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

8 17

Status of the Derby, and request for another slip
by Larry Hastings 27 Jan '14

27 Jan '14

The Great Argument Clinic Conversion Derby has been underway for about 2.5 weeks. Here's a status update. I'll try to keep this short. It's taking a lot longer than I expected it to. This has been due to * bugs and flaws in Argument Clinic itself (a "flaw" is an "I didn't foresee that" design error), * having to make changes to CPython itself, which required *very* careful work and was therefore quite time consuming, and * my time being a huge bottleneck. I'd like to extend the Derby by two more weeks and add a fourth beta. -- I've made huge improvements to the implementation of Argument Clinic, in response to bugs filed and crippling-missing-features requested. Argument Clinic is growing more sophisticated by the day. I've also, regrettably, made some modifications to CPython, to better support signatures in builtins. In particular, I had to modify four additional callable types to add support for "__text_signature__", including PyTypeObject. These changes were reviewed by Nick and Guido, so I have a high confidence that the changes will not destabilize Python 3.4. (If there is demand, I can compile a list of all such arguably-not-a-bugfix post-feature-freeze changes to Python 3.4 if there's interest.) The main problem with running the Derby: everything is gated on me. Until just recently, I've had to review every patch, and I've had to make every modification to clinic.py. I've been spending all day, every day, with very little time off, on the Derby, and still it's nowhere near enough. This is changing, slowly. I now have a handful of people who make small changes to clinic.py or review patches. But major brain surgery on the tool is still up to me, and I still have a half-week's worth of work on my to-do list. I've been prioritizing bug fixes and crippling missing features over reviewing patches. (I figured that would maximize the productivity of the Derby participants.) As a result, I have a patch backlog that would makes you cry. I've been reviewing patches FIFO, and the patch waiting at the front of my queue is ten days old. This means that a lot of the conversion work done for the Derby has not been checked in yet. To be honest I have absolutely no idea what % of files have been converted so far. Figuring it out would use time that would be better spent catching up on my to-do list. -- I'd like to continue the Derby for two more weeks and insert a fourth beta. That would put the release date for Python 3.4 final on March 31st. With the new conversion policy in place (see my recent posting to python-dev), the number of new surprises that I would have to spend time on should drop to about zero. (My new answer: "That will have to wait until 3.5.") I have about a half-week's worth of bug fixes / missing features work for clinic.py right now; once that's done I'll be able to spend nearly all of my time on code reviews. (If I catch up, I might even start doing conversions myself!) So I expect the pace of the Derby to ramp up in the coming week. I *would* ask for three more weeks, not two, just to be dead certain we finish before rc1. But then the release date for 3.4 final would be April 7th, two days before the start of PyCon US 2014, and I don't want to shave it that close. If we had to slip again for some reason, we'd be cutting the release *during* PyCon US, and I want to avoid *that* at all costs, particularly as that may be impractical for the platform release manager (their build environment may not be portable). I want 3.4 out before PyCon US so that trunk is open for 3.5 during the dev sprints. The alternative would be: stop all submissions to the Derby right now, iterate (quickly!) on the existing patches, check them in, and proceed to rc1. In theory, as release manager I could simply announce we were doing this. However, I only want to continue doing this if I have approval from the core dev community. If slipping the schedule again means a majority of people withdraw their support for the Derby, then it would probably be best to cut our losses now. Please vote for either "continue the Derby" (which also means slipping the schedule and adding a fourth beta) or "stop the Derby". Thank you for your attention, //arry/

13 19

introduction
by Yury Selivanov 24 Jan '14

24 Jan '14

Hello, Thank you very much for accepting me as a member of your team! Python is the most beautiful, simple and complex language I ever knew, and it's an honor for me to help to make it better. Just to give you an idea of how passionate I am about python, I'd like to tell you a short story. When I just started to work on PEP 362 with Brett and Larry, we had a few discussions of how things should be going and I promised them to draft a first version of the PEP and implementation by the end of the week. On Friday, I decided to make a surprise for my girlfriend and booked a weekend in Montreal, and in the morning of Saturday I woke up with a fever of 37 degrees. Well, it's just 37, I'll be OK by the time we drive there. Hell I was wrong. For two days, the only thing I was romantically attached to, was my bed in the hotel room. And midday of Sunday, I realized that I have a promise to fulfill. So here I was, sitting on the bed, with a fever of 39 degrees, with one eye closed, and the other one so filled with tears so I had to squish it to have a clear spot on the screen, working on that draft and writing unittests. All because I knew that while it'd be OK to wait a few days, we didn't actually have time for that, because the release was just around the corner, and every day was important. So, again, thank you very much! I now have to break the buildbot I guess.. Yury

6 5

Commit access for Yury Selivanov?
by Nick Coghlan 23 Jan '14

23 Jan '14

Hi, I'd like to suggest granting commit access to Yury Selivanov, primarily to assist with maintenance of the inspect module. That's currently an orphaned module in the experts index, and Yury was a driving force behind getting PEP 362 (the new introspection API) accepted for Python 3.3, and has also picked up on a number of introspection support issues we missed when adding other features to Python 3.4 (like inspect.signature not handling functools.partialmethod correctly - it simply didn't occur to me or Alon to add test cases for that). He's also created a reimplementation of inspect.getfullargspec for Python 3.4 (not yet merged, but close to being so) that will allow almost all existing introspection code to benefit from the Argument Clinic changes, not just the code that has been ported to the new PEP 362 introspection API. Yury's interested in the idea of commit access, and is comfortable with our approach to code review and automated testing. As usual when nominating someone, I'm happy to handle the mentoring period and addressing any questions Yury may have about the mechanics of actually pushing changes rather than having to wait for me or Larry or someone else to merge them on his behalf. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 9

Commit right for Vajrasky Kok?
by Victor Stinner 21 Jan '14

21 Jan '14

Hi, I noticed that Vajrasky Kok is very active on bugs.python.org. He produced many patches and contribued to various modules written in Python and C. Search "Vajrasky Kok" in the Mercurial history to see how many contributions he made recently. On http://bugs.python.org his nickname is "vajrasky". He knows the process of review and update his patch when he got remarks. If you consider that he needs a mentor, I can be his mentor. Victor

5 6

Re: [python-committers] LCA2014 presentation on Zuul, OpenStack's merge gating system
by Nick Coghlan 20 Jan '14

20 Jan '14

On 21 Jan 2014 01:19, "Meador Inge" <meadori(a)gmail.com> wrote: > > On Sat, Jan 18, 2014 at 11:34 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > >> If people find this idea interesting, I would like to invite some of >> the Mercurial devs and OpenStack infrastructure folks (i.e. Zuul devs) >> to the language summit (leaving it up to them if they stay for the >> whole day, or just this part). I believe we already have enough >> Reitveld, Roundup and Buildbot experience amongst the core development >> team to assess the feasibility of the idea from that side of things. > > > I find the idea interesting and plan to watch the linked video and > read up on this. Is it an all or nothing thing (i.e. either all devs > operate using the new Zull-based method or no one does)? All-or-nothing - with merge gating, *only* Zuul commits to the main branches/clone (so we'd also be in for some interesting discussions with the release managers and installer buiders - we may need to provide them with the ability to override the normal gating process). The underlying philosophy of the approach is reflected in the fact that OpenStack doesn't have core committers or even core developers - they have core reviewers instead. Cheers, Nick. > > -- > # Meador

1 0

LCA2014 presentation on Zuul, OpenStack's merge gating system
by Nick Coghlan 19 Jan '14

19 Jan '14

One of the items I have on the language summit agenda is the core reviewer workflow used by the OpenStack project, specifically Zuul, the merge gating system that powers that workflow. I know some other folks here are working on OpenStack these days so this will already be familiar to them, but many aren't. I'm not either, but my day job involves developing CI workflow automation tools for Red Hat, so I'm paying very close attention to what the OpenStack infrastructure team are up to. We don't necessarily have to wait until the language summit to discuss it though, so I figured I'd pass along a link to James Blair's presentation at linux.conf.au 2014: Stream: https://www.youtube.com/watch?v=sLD9LHc1QFM Download: http://mirror.linux.org.au/linux.conf.au/2014/Friday/106-How_OpenStack_Impr… The Zuul docs are at: http://ci.openstack.org/zuul/ The OpenStack Zuul dashboard is at: http://status.openstack.org/zuul/ The key difference, relative to our current workflow, is to take the human out of the loop for the final pre-merge test run. Instead, once a core reviewer gives a patch a +2, the process of taking that reviewed patch, checking it still merges correctly, checking it passes all the tests on the stable buildbots and then merging it into source control would all be handled automatically by Zuul. This will require and/or enable eliminating several of the current annoyances in the core development workflow: - the annoyingly manual "download patch, apply patch, run tests locally, commit change, push change, watch the buildbots for problems" approach goes away entirely (while what we have now is substantially better than what we had a few years ago, times move on and projects like OpenStack raise the bar when it comes to figuring out how to make the most effective of scarce contributor time) - spurious conflicts on NEWS will go away (as we'll have to finally address this in order to avoid breaking Zuul's gating) - push races will go away (since merging would now be handled by Zuul, non conflicting changes would be rebased automatically, and only conflicting changes would be bounced back to the submitter for updates). Critically, conference sprints might create a backlog in Zuul, but they wouldn't completely break a developer's workflow through incessant push races relating to non-conflicting changes. - POSIX developers breaking the Windows buildbots and vice-versa (since Zuul would ensure at least the stable buildbots remain green by blocking such changes before they hit the main branches). This means even when we get such things wrong, we will no longer have the time pressure of needing to unbreak other people's builds. - approving and merging pure docs patches should become largely trivial, as it should be possible to configure Zuul to only check that the docs build cleanly for those, rather than running the full test suite across all the stable buildbots. - checking for a contributor licensing agreement in Roundup before processing a patch could also be automated, rather than requiring core developers to check for it manually. Zuul, as OpenStack use it, already has plugins for the Gerrit code review/git repo management system (at least as customised by the OpenStack infrastructure folks), as well as for Jenkins to run the automated CI tests. Rather than suggesting wholesale changes to our own infrastructure, what I am suggesting we consider is devoting time (and potentially PSF funding) to getting Zuul to play nice with Roundup, Reitveld, BuildBot and Mercurial. (Like the rest of our existing infrastructure, Zuul is a web service implemented in Python). The main technical challenges I foresee are: * dealing with maintenance branches (especially for patches which don't merge forward cleanly), since OpenStack currently appear to "handle" that limitation by just not providing upstream maintenance branches at all, leaving downstream vendors to fill the gap * finally cleaning up the way we manage the NEWS file (see http://bugs.python.org/issue18967 for discussion) * replicating Gerrit's patch approval/voting mechanism in Reitveld * replicating Gerrit's merge/rebase capabilities in Mercurial (I'm sure Mercurial is functionally capable of this, I just don't know the best way to model it). * actually writing the new Zuul plugins to talk to the services used by the CPython project, rather than those used by OpenStack There would also be a non-trivial update to the developer guide involved in such a change, since it would heavily impact the core developer workflows, as well as the way external contributors interact with the core development team. If people find this idea interesting, I would like to invite some of the Mercurial devs and OpenStack infrastructure folks (i.e. Zuul devs) to the language summit (leaving it up to them if they stay for the whole day, or just this part). I believe we already have enough Reitveld, Roundup and Buildbot experience amongst the core development team to assess the feasibility of the idea from that side of things. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

3 3