Mailman 3 August 2015 - python-committers

PSA: replace your DSA keys for SSH
by Georg Brandl 21 Oct '15

21 Oct '15

Hi all, newer OpenSSH versions (7.0+) default to not allowing ssh-dss keys for public key authentication. If you experience "permission denied" errors, this (currently) comes from the client side only and hg.python.org will accept these keys if you enable them using the PubkeyAcceptedKeyTypes option in your SSH config file. Of course ssh-dss is being phased out for a reason; we'd like to invite everybody who has only DSA keys submitted for hg.python.org access to send an RSA (min. 1024 bits) or ED25519 key to hgaccounts(a)python.org. cheers, Georg

9 17

How To Forward-Merge Your Change After Your Pull Request Is Accepted Into Python 3.5.0rcX
by Larry Hastings 27 Aug '15

27 Aug '15

Now that we're in the "release candidate" phase of Python 3.5.0, the workflow has changed a little. We're trying an experiment using Bitbucket and pull requests. You can read about that workflow, here: https://mail.python.org/pipermail/python-dev/2015-August/141167.html But the instructions for that workflow are pretty hazy on what you do after your pull request is accepted. This message is an addendum to those instructions, describing exactly what you should do after your pull request is accepted. To save wear and tear on my hands (and your eyes), for the rest of these instructions, I'm going to refer to each place-you-can-check-things-in-to by version number as follows: 3.4 : hg.python.org/cpython (branch "3.4") 3.5.0 : https://bitbucket.org/larry/cpython350 (branch "3.5") 3.5.1 : hg.python.org/cpython (branch "3.5") 3.6 : hg.python.org/cpython (branch "default") With that nomenclature established I can now precisely say: when your pull request is accepted into 3.5.0, you need to merge from 3.5.0 into 3.5.1, and then from 3.5.1 into 3.6. Doing this is much like the existing workflow. You use "hg merge" to merge your changes from previous versions into subsequent versions (what I call "forward merging"). What complicates matters is the fact that the 3.5.0 release candidates don't live in the normal repo--they lives in a repo on Bitbucket which is only writeable by me. In order to keep a tight lid on the changes checked in to the 3.5.0 release candidates, I won't pull revisions from the normal CPython repo. (If I did, I'd also pull in changes intended for 3.5.1, and... it'd be a mess.) So here come the instructions. They look long, but that's just because I go into a lot of detail to try and make them as foolproof as possible. They aren't really much longer or more complicated than the steps you already follow to perform forward-merges. Note that these are easy, guaranteed-clean instructions on how to perform the merge. Are there shortcuts you could take that might speed things up? Yes. But I encourage you to skip those shortcuts and stick to my instructions. Working with multiple branches is complicated enough, and this external repo makes things even more complicated. It's all too easy to make a mistake. HOW TO FORWARD-MERGE FROM 3.5.0 TO 3.5.1 ---------------------------------------- 1: Wait until your pull request has been accepted. 2: Make a *clean* local clone of the CPython tree, updated to the "3.5" branch. In my instructions I'll call the clone "cpython351-merge": % hg clone ssh://hg@hg.python.org/cpython -u 3.5 cpython351-merge % cd cpython351-merge 3: Confirm that you're in the correct branch. You should be in the "3.5" branch. Run this command: % hg id Let's assume that the current head in the "3.5" branch has changeset ID "7890abcdef". If that were true, the output of "hg id" would look like this: 7890abcdef (3.5) It might also say "tip" on the end, like this: 7890abcdef (3.5) tip If it doesn't say "3.5", switch to the 3.5 branch: % hg up -r 3.5 and repeat this step. 4: Pull from the 3.5.0 repo into your "cpython351-merge" directory. % hg pull ssh://hg@bitbucket.org/larry/cpython350 You should now have two "heads" in the 3.5 branch; the existing head you saw in the previous step, and the new head you just pulled in, which should be the changeset from your pull request. 5: As an optional step: confirm you have the correct two heads. This command will print a list of all the heads in the current repo: % hg heads Again, you should have exactly two identified as being on the "3.5" branch; one should have the changeset ID shown by "hg id" in step 3, the other should be your change from the pull request. 6: Merge the two heads together: % hg merge If there are merge conflicts, Mercurial will guide you through the conflict resolution process as normal. 7: Make sure that all your changes merged properly and you didn't merge anything else by accident. I run these two commands: % hg stat % hg diff and read all the output. 8: Make sure Misc/NEWS has your update in the right spot. (See below.) 9: Check in. The checkin comment should be something like Merge from 3.5.0 to 3.5.1. 10: Push your changes back into the main CPython repo. % hg push 11: Now forward-merge your change to 3.6 as normal, following the CPython Dev Guide instructions: https://docs.python.org/devguide/committing.html#merging-between-different-… FREQUENTLY ASKED QUESTIONS -------------------------- Q: I screwed something up! What do I do now? If you haven't pushed your changes out, it's no problem. Just delete your repo and start over. If you *have* pushed your changes out, obviously we'll need to fix the mistake. If you're not sure how to fix the problem, I suggest logging in to the #python-dev IRC channel and asking for help. Q: What do I need to do about Misc/NEWS? I'm glad you asked! First, you *must* put your Misc/NEWS update into the correct section. If you're creating a pull request for Python 3.5.0 rc-something, put it in the 3.5.0 rc-something section. If you're checking in to 3.5.1, put it in the 3.5.1 section. If you're just checking into 3.6, put it in the 3.6.0 alpha 1 section. Second, when you merge forward, make sure the merge tool puts your Misc/NEWS entry in the right section. The merge tool I seem to use isn't particularly smart about this, so I've had to manually edit Misc/NEWS many times to fix it. (When I released 3.5.0rc2, I had to do a lot of cleanup on Misc/NEWS, and again in the 3.5.1 branch, and again in 3.6.) Every time you merge forward, make sure your Misc/NEWS entry is in the right spot. Q: What if a second pull request is accepted before I get around to doing the merge? Well, *someone* needs to merge, and they're going to have to merge *both* changes. I can't come up with a good general policy here. Hopefully this won't happen often; for now let's just handle it on a case-by-case basis. Q: What if I have a bugfix for 3.4 that I want to ship with 3.5.0? You have to check in twice, and merge-forward twice. First, check in to 3.4, then merge forward into 3.5.1 and 3.6. Then, once your pull request is accepted into 3.5.0, do a "null merge" (a merge where no files are changed) from 3.5.0 into 3.5.1 and 3.6. Q: What if my pull request is turned down? If your bug fix isn't critical enough to merit shipping with 3.5.0, just check it into the normal 3.5 branch on hg.python.org and it'll ship with 3.5.1. (And, naturally, forward-merge it into 3.6.) //arry/

2 1

[RELEASED] Python 3.5.0rc2 is now available
by Larry Hastings 25 Aug '15

25 Aug '15

On behalf of the Python development community and the Python 3.5 release team, I'm relieved to announce the availability of Python 3.5.0rc2, also known as Python 3.5.0 Release Candidate 2. Python 3.5 has now entered "feature freeze". By default new features may no longer be added to Python 3.5. This is a preview release, and its use is not recommended for production settings. You can find Python 3.5.0rc2 here: https://www.python.org/downloads/release/python-350rc2/ Windows and Mac users: please read the important platform-specific "Notes on this release" section near the end of that page. Happy hacking, //arry/

1 0

How are we merging forward from the Bitbucket 3.5 repo?
by Larry Hastings 24 Aug '15

24 Aug '15

So far I've accepted two pull requests into bitbucket.com/larry/cpython350 in the 3.5 branch, what will become 3.5.0rc2. As usual, it's the contributor's responsibility to merge forward; if their checkin goes in to 3.5, it's their responsibility to also merge it into the hg.python.org/cpython 3.5 branch (which will be 3.5.1) and default branch (which right now is 3.6). But... what's the plan here? I didn't outline anything specific, I just assumed we'd do the normal thing, pulling from 3.5.0 into 3.5.1 and merging. But of the two pull requests so far accepted, one was merged this way, though it cherry-picked the revision (skipping the pull request merge revision Bitbucket created), and one was checked in to 3.5.1 directly (no merging). I suppose we can do whatever we like. But it'd be helpful if we were consistent. I can suggest three approaches: 1. After your push request is merged, you cherry-pick your revision from bitbucket.com/larry/cpython350 into hg.python.org/cpython and merge. After 3.5.0 final is released I do a big null merge from bitbucket.com/larry/cpython350 into hg.python.org/cpython. 2. After your push request is merged, you manually check in a new equivalent revision into hg.python.org/cpython in the 3.5 branch. No merging necessary because from Mercurial's perspective it's unrelated to the revision I accepted. After 3.5.0 final is released I do a big null merge from bitbucket.com/larry/cpython350 into hg.python.org/cpython. 3. After your push request is merged, you pull from bitbucket.com/larry/cpython350 into hg.python.org/cpython and merge into 3.5. In this version I don't have to do a final null merge! I'd prefer 3; that's what we normally do, and that's what I was expecting. So far people have done 1 and 2. Can we pick one approach and stick with it? Pretty-please? //arry/

6 9

[low-pri] Changing my email address
by Steve Dower 12 Aug '15

12 Aug '15

Hi all Just a heads-up that I'll be switching to an alternate email address for all of my Python communications, due to what I'm sure are very sensible corporate security policies that nonetheless corrupt code snippets and URLs in my incoming email. I will henceforth be known as steve.dower(a)python.org (which is forwarding to python(a)stevedower.id.au, for the security conscious who will no doubt spot that address in headers). I can obviously still be reached at my old address, just don't send me URLs or text with full stops in it please :) Cheers, Steve

1 0

Sorry folks, minor hiccup for Python 3.5.0rc1
by Larry Hastings 12 Aug '15

12 Aug '15

I built the source tarballs with a slightly-out-of-date tree. We slipped the release by a day to get two fixes in, but the tree I built from didn't have those two fixes. I yanked the tarballs off the release page as soon as I suspected something. I'm rebuilding the tarballs and the docs now. If you grabbed the tarball as soon as it appeared, it's slightly out of date, please re-grab. Sorry for the palaver, //arry/

2 3

A possible solution to the Misc/NEWS merging problem
by Brett Cannon 11 Aug '15

11 Aug '15

We have all been there: you fix something in some maintenance branch, do your `hg pull` into the default branch, and everything *but* Misc/NEWS merges cleanly. You typically revert Misc/NEWS, commit your forward-ported change to default, and then move on with your life. It would be much easier if Misc/NEWS was never an issue. In preparation for choosing a workflow that allows us to do patch/PR merging through the browser, we have been thinking about this Misc/NEWS "problem". The solution we came up with was to put the changelog message in Roundup. The idea is that Roundup grows a changelog field to enter a changelog messge and a comma-separated list of commit revisions pertaining to the issue. We continue to do commit messages as we do, but when Roundup adds the commit message to the issue it will also append the revision number to the revision(s) field. Any relevant changelog entry for an issue will be added to the issue itself -- by hand to start, we can talk about automation later -- and not a file that could have merge conflicts. When it comes time to cut a release we will have a tool that will go through Roundup, collect all the relevant issues for the release, and create the changelog file based on what is in Roundup. This will buy us several things. One is no more merge conflicts for Misc/NEWS since it won't exist as-is (we might have the release manager check in a generated file per version, but that can be discussed later). Two is no more extra commits to add a missing changelog entry in Misc/NEWS since you just need to update the issue in Roundup instead. Three is it should lead to a more rich changelog as we will have the issue # and all related commits tied to the changelog message itself, so they can contain whatever backlinks we want in the changelog. The only potential downside is any change warranting a changelog entry will now also require an issue. Now I personally think -- and others on the core-workflow list seem to agree -- this is actually a good thing to help track changes that are made and tie all relevant information together from across the issue tracker and code repo. Basically if something is important enough to be listed in the changelog it should be important enough to have an issue tracking it (we could even require the changelog entry be filled in before you're allowed to set an issue to "fixed"). The whole point of this email is to let everyone know that this is the plan and should be coming in a couple of months (I'm also hoping to make a decision about a new overall workflow by 2016, but I need to hear back from Donald first about whether the proposed timeline for a test instance will work for him). If you hate this idea so much that it will cause you to stop contributing to Python then let me know, otherwise I'm going to assume people are either happy with this approach or will begrudgingly accept it.

3 2

Instructions on the new "push request" workflow for 3.5.0rc1+ through 3.5.0 final
by Larry Hastings 11 Aug '15

11 Aug '15

As of Python 3.5.0rc1, the canonical repository for Python 3.5.0 is *no longer* on hg.python.org. Instead, it's hosted on Bitbucket on my personal account, here: https://bitbucket.org/larry/cpython350 Since 3.5.0rc1 isn't out yet I'm keeping the repository private for now. Once 3.5.0 rc1 is released (hopefully Monday) I'll flip the switch and make the repository public. (I'll email python-dev and python-committers when that happens.) Putting it succinctly, here's a table of versions and where you'd check in for your change to go there: 3.5.0 : https://bitbucket.org/larry/cpython350 (branch "3.5") 3.5.1 : hg.python.org/cpython (branch "3.5") 3.6.0 : hg.python.org/cpython (branch "default") You'll notice nobody but myself has checkin permissions for my 3.5.0 repo on Bitbucket. That's on purpose. The only way you can get changes in to 3.5.0 now is by sending me a Bitbucket "pull request". This is a workflow experiment, to see if we as a community like this sort of new-fangled gizmo. For now, we're only using Bitbucket for the actual final checking-in stage. Requests for fixes to be accepted into 3.5.0 and code review will all still happen on the Python issue tracker. Also, I'm officially now asking you folks to do the forward-merge into 3.5.1 and 3.6.0 yourselves. Here's how to get a fix checked in for 3.5.0, starting with 3.5.0rc1+ and continuing through until 3.5.0 final. Pre-requisites: * You must have a Bitbucket account. * You must have commit rights to the CPython repository. 1. Create an issue on the Python issue tracker for the problem. 2. Submit a patch that fixes the problem. 3. Add me to the issue and get me to agree that it needs fixing in 3.5.0. (You can attempt this step before 2 if you prefer.) 4. Fork my repository into your Bitbucket account using their web GUI. To do that, go to Bitbucket, log in, then go to my 3.5.0 repo: https://bitbucket.org/larry/cpython350 and press the "Fork" link in the left column. Bitbucket has a tutorial on how to do this, here: https://confluence.atlassian.com/display/BITBUCKET/Fork+a+teammate%27s+repo… Note: DO NOT start with a conventional CPython trunk cloned from hg.python.org. The 3.5 branch in my repo and the 3.5 branch in normal CPython trunk have intentionally diverged and *need* to stay out-of-sync. 5. Make a local clone of your fork on your computer. Bitbucket has a tutorial on how to do that, here: https://confluence.atlassian.com/display/BITBUCKET/Copy+your+Mercurial+repo… Reminder: switch to the 3.5 branch! 6. Apply your change to the 3.5 branch and check in. Reminder: check in to the 3.5 branch! 7. Make sure you checked in your change to the 3.5 branch. Reminder: Seriously. I keep messing this up. I say, the more reminders, the better. 8. Push your change back to *your* fork on *your* Bitbucket account. Just normal "hg push" should work here. In case it helps, I recommend using the "https" protocol for this step, as it sidesteps ssh authentication and prompts you for your Bitbucket username and password. 9. Create a pull request using Bitbucket's web GUI. Bitbucket has a tutorial on how to create a pull request, here: https://confluence.atlassian.com/display/BITBUCKET/Create+a+pull+request On the "Create pull request" web GUI, make sure that you specify branch "3.5" for *both* your repo *and* my repo. Also, make sure you *don't* check the "Close 3.5 after the pull request is merged" check box. (If you use the "Compare" page, you also need to select "3.5" in both drop-down lists--one for my repo, and one for yours.) 10. Paste a link to the pull request into the issue tracker issue for this change request. 11. Wait for confirmation that I've accepted your pull request into the 3.5.0 repo. 12. Pull your accepted change from your local Bitbucket fork repo into a normal hg.cpython.org CPython repo, merge into 3.5, then merge into 3.6, then push. For the record, here's what *my* workflow looks like when I accept your pull request: 1. Click on the URL you pasted into the pull request. 2. Visually check that the diff matches the approved diff in the issue on the issue tracker. 3. Click on the "Merge" button. Frequently Asked Questions ========================== Q: What if someone sends me a "pull request" for a change that doesn't merge cleanly? A: Then I'll decline it, and ask you on the issue tracker to rebase and resubmit. Q: What if someone sends me a "pull request" but they don't have commit rights to CPython? A: I'll ignore it. I'll only pay attention to pull requests pasted into the issue tracker by someone with commit rights. Q: Whose name goes on the commit? A: It gets the name the checkin was made with. Don't worry, your name will stay on your commit. Q: This seems like a lot more work than the old way. A: For you guys, yes. But notice how little work it is for *me*! Seriously. Q: Can I reuse my fork / my local copy? Or do I have to create a fresh one each time? A: I don't care either way. All I care about are clean pull requests. If you're careful you should have no trouble reusing forks and local checkouts. If it were me, I'd probably use a fresh fork each time. Forks are cheap and this way is cleaner. I'll add these instructions to the Python Dev Guide in the next day or two. /arry p.s. Remember to use the 3.5 branch!

2 2

[RELEASED] Python 3.5.0rc1 is now available
by Larry Hastings 11 Aug '15

11 Aug '15

On behalf of the Python development community and the Python 3.5 release team, I'm relieved to announce the availability of Python 3.5.0rc1, also known as Python 3.5.0 Release Candidate 1. Python 3.5 has now entered "feature freeze". By default new features may no longer be added to Python 3.5. This is a preview release, and its use is not recommended for production settings. You can find Python 3.5.0rc1 here: https://www.python.org/downloads/release/python-350rc1/ Windows and Mac users: please read the important platform-specific "Notes on this release" section near the end of that page. Happy hacking, /arry

1 0

Python 3.5.0rc1 is delayed by a day
by Larry Hastings 10 Aug '15

10 Aug '15

We retagged Python 3.5.0rc1 today to fix two bugs that popped up late in the process. Release candidates are supposed to be software you genuinely would release, and I couldn't release Python with both those bugs. This delay rippled through the whole process, so it just isn't going out tonight (late Sunday / early Monday in my timezone). I have every expectation it'll go out Monday. In case you're interested, the bugs are (were!): http://bugs.python.org/issue24745 http://bugs.python.org/issue24835 My thanks to the folks who stepped up and fixed the bugs on short notice, and my apologies to the community for the delay. We're just trying to make the best Python we can, for yooooooooou! See you tomorrow, //arry/

1 0

Having issues updating a checkout
by Brett Cannon 10 Aug '15

10 Aug '15

Anyone else seeing this? pulling from ssh://hg@hg.python.org/cpython remote: ssh_exchange_identification: Connection closed by remote host abort: no suitable response from remote hg! My laptop is fairly new so I'm trying to eliminate any way it might be my setup instead of the servers.

3 3

Reminder: the "3.5" branch in CPython trunk is now 3.5.1
by Larry Hastings 09 Aug '15

09 Aug '15

As I write this email I'm tagging Python 3.5.0 release candidate 1. This is the moment that we switch over to our new experimental workflow, where we use Bitbucket and pull requests for all future changesets that will get applied to 3.5.0. The Bitbucket repository isn't ready yet, and I'm still putting the final touches on the documentation for the process. I'll have everything ready no later than a day from now. It'll be posted here, and will also go into the Python Dev Guide. For now changes for 3.5.0rc2 will just have to wait. Again: any revisions you check in to the "3.5" branch on hg.python.org/cpython right now will go automatically into 3.5.1. They will *not* automatically go in to 3.5.0. The only way to get changes into 3.5.0 from this moment forward is by creating a pull request on Bitbucket which you convince me to accept. //arry/ p.s. In case you're curious, the last revision to make it in to 3.5.0rc1 (apart from the revisions I generate as part of my build engineering work) was 202a7aabd4fe.

1 0

erykun
by R. David Murray 07 Aug '15

07 Aug '15

FYI I just granted Developer privs on the tracker to eryksun (real name unknown), after wondering why s/he hadn't just closed an issue s/he commented on and thereby finding s/he didn't have them yet. Someone to keep an eye on, IMO. --David

1 0

getting help with the hgaccounts alias
by Brett Cannon 03 Aug '15

03 Aug '15

I think the only people on it are Antoine, Georg, and myself. Antoine has said he has left python-dev, Georg is often busy with school, and I'm on a temp machine for a few weeks so I can't add any SSH keys for a while (heck I'll probably need new ones installed on my behalf once I have a permanent machine). Is anyone up for helping with adding SSH keys for people? It's basically taking someone's key as an attachment in an email, making sure it's RSA and not DSA, and then either creating a new file in a specific repo for them or appending it to their existing key file. P.S.: And I don't even remember how to get people added to the group of people who can add keys, so I probably need to find that out as well if anyone steps forward. =)

8 18