On Sun, Feb 7, 2010 at 22:51, Benjamin Peterson wrote:

How about a week after, so we have more time to adjust release procedures?

Sounds fine to me.

Will you do test conversions of the sandbox projects, too?

Got any particular projects in mind?

Also I think we should have some document (perhaps the dev FAQ) explaining exactly how to do common tasks in mercurial. For example - A bug fix, which needs to be in 4 branches. - A bug fix, which only belongs in 2.7 and 2.6 or 3.2 and 3.1. - Which way do we merge (What's a subset of what?)

Yes, writing lots of docs is part of the plan. Cheers, Dirkjan

Benjamin Peterson wrote:

2010/2/8 Dirkjan Ochtman :

...

On Sun, Feb 7, 2010 at 22:51, Benjamin Peterson wrote:

...

Will you do test conversions of the sandbox projects, too? Got any particular projects in mind?

2to3.

Does Mercurial even support merge tracking the way we are doing it for 2to3 right now? Regards, Martin

On Tue, Feb 9, 2010 at 04:47, Benjamin Peterson wrote:

I don't believe so. My plan was to manually sync updates or use subrepos.

Using subrepos should work well for this. It turned out that my local copy of the Subversion repository contained the Python dir only, so I'm now syncing a full copy so that I can convert other parts. I believe 2to3 might be a little tricky because it was moved at some point, but I can look at getting that right (and this will help in converting other parts of the larger Python repository). Cheers, Dirkjan

On Wed, Feb 10, 2010 at 02:03, Benjamin Peterson wrote:

What do you mean by moved? I don't it has ever moved around in the sandbox.

IIRC it was moved into the sandbox from some other location at some point? Cheers, Dirkjan

On Wed, Feb 10, 2010 at 13:59, Benjamin Peterson wrote:

The only moving was moving a lot of the files into a lib2to3 directory. It would be nice if the hg history could be preserved for those files.

Please see if hg.python.org/2to3 would satisfy your needs. Cheers, Dirkjan

Why even keep 2to3 in the sandbox? It should be mature enough now to be maintained directly in the tree.

I think the original plan was to make standalone releases, so that people could upgrade their installation from a newer release of 2to3. IMO, it is realistic to predict that this will not actually happen. If we can agree to give up the 2to3 sandbox, we should incorporate find_pattern into the tree, and perhaps test.py as well. Regards, Martin

Brett Cannon wrote:

On Fri, Feb 12, 2010 at 11:17, "Martin v. Löwis" wrote:

...

IMO, it is realistic to predict that this will not actually happen. If we can agree to give up the 2to3 sandbox, we should incorporate find_pattern into the tree, and perhaps test.py as well.

I vote on giving up the 2to3 sandbox.

Besides, if we're using hg, it should make it much easier for someone else to branch that part of the stdlib and create a standalone 2to3 release from it if they really want to. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

I personally like 2to3 in a separate repo because it fits well with my view that 2to3 is an extra application that happens to also be distributed with python.

But isn't that just a theoretical property? I know that's how 2to3 started, but who, other than the committers, actually accesses the 2to3 repo? I would be much more supportive of that view if there had been a single release of 2to3 at any point in time (e.g. to PyPI). Alas, partially due to me creating lib2to3, you actually couldn't release it as an extra application and run it on 2.6 or 2.7, as the builtin lib2to3 would take precedence over the lib2to3 bundled with the application. Regards, Martin

...

But isn't that just a theoretical property? I know that's how 2to3 started, but who, other than the committers, actually accesses the 2to3 repo?

It's used in 3to2 for example.

That doesn't really seem to be the case. AFAICT, 3to2 is a hg repository, with no inherent connection to the 2to3 svn sandbox. It does use lib2to3, but that could come either from an installed Python, from a trunk/3k checkout, or from the sandbox. Correct? So if the 2.x trunk became the official master for (lib)2to3, nothing would really change for 3to3, right? (except for the comment in the readme that you should get 2to3 from the sandbox if the trunk copy doesn't work; this comment would become obsolete as changes *would* propagate immediately into the Python trunk).

...

I would be much more supportive of that view if there had been a single release of 2to3 at any point in time (e.g. to PyPI). Alas, partially due to me creating lib2to3, you actually couldn't release it as an extra application and run it on 2.6 or 2.7, as the builtin lib2to3 would take precedence over the lib2to3 bundled with the application.

It could be distributed under another name or provide a way to override the stdlib version.

Sure. However, I'm still claiming that this is theoretical. The only person who has shown a slight interest in having this as a separate project (since Collin Winter left) is you, and so far, you haven't made any efforts to produce a stand-alone release. I don't blame you at all for that, in fact, I think Python is better off with the status quo (i.e. changes to 2to3 get liberally released even with bug fix releases, basically in an exemption from the "no new features" policy - similar to -3 warnings). I still think that the best approach for projects to use 2to3 is to run 2to3 at install time from a single-source release. For that, projects will have to adjust to whatever bugs certain 2to3 releases have, rather than requiring users to download a newer version of 2to3 that fixes them. For this use case, a tightly-integrated lib2to3 (with that name and sole purpose) is the best thing. Regards, Martin

The other thing is that we will loose some vcs history and some history granularity by switching development to the trunk version, since just the svnmerged revisions will be converted.

I suppose it might be possible to fake the history of Lib/lib2to3 with commits that didn't actually happen, although this is probably a cure worse than the disease. We are not going to throw away the subversion repository, so it would always be possible to go back and look at the actual history. Regards, Martin

...

The other thing is that we will loose some vcs history and some history granularity by switching development to the trunk version, since just the svnmerged revisions will be converted.

So the consensus is that 2to3 should be pulled out of the main Python tree?

Not sure what you mean by "pull out"; I had expect that the right verb should be "pull into": 2to3 should be pulled into the main Python tree.

Should the 2to3 hg repository be deleted, then?

Which one? To my knowledge, there is no official 2to3 repository yet. When the switchover happens, 2to3 should not be converted to its own hg repository, yes. Regards, Martin

...

...

Should the 2to3 hg repository be deleted, then? Which one? To my knowledge, there is no official 2to3 repository yet. When the switchover happens, 2to3 should not be converted to its own hg repository, yes.

This one: http://hg.python.org/2to3

Ah, this shouldn't be used at all for anything (except for studying how Mercurial works). Along with the cpython repository, it is Dirkjan's test conversion. Even if it survived the ultimate migration (which it probably won't), it would get regenerated from scratch, probably changing all revision numbers. Regards, Martin

On Tue, Feb 23, 2010 at 00:55, "Martin v. Löwis" wrote:

...

Ah, this shouldn't be used at all for anything (except for studying how Mercurial works). Along with the cpython repository, it is Dirkjan's test conversion. Even if it survived the ultimate migration (which it probably won't), it would get regenerated from scratch, probably changing all revision numbers.

Actually, since this one is much simpler, it's more or less ready to be the canonical repository, if Benjamin would like that. It's very different from the cpython repository in that respect.

I thought we decided not to have a 2to3 repository at all, but let this live in the Python trunk exclusively. Regards, Martin

2010/2/23 Dirkjan Ochtman :

On Tue, Feb 23, 2010 at 01:06, "Martin v. Löwis" wrote:

...

I thought we decided not to have a 2to3 repository at all, but let this live in the Python trunk exclusively.

That would be fine with me, I just remembered that Benjamin would like to start using hg sooner and having it as a separate repo was okay.

I would :), but Martin is correct in that we agreed to start developing it as part of the trunk when the hg transition takes place. -- Regards, Benjamin

Dirkjan Ochtman wrote:

On Mon, Feb 22, 2010 at 16:09, Collin Winter wrote:

...

So the consensus is that 2to3 should be pulled out of the main Python tree? Should the 2to3 hg repository be deleted, then?

Wouldn't the former be reason to officialize the hg repository, instead of deleting it?

I think the difference between "pull out" and "pull from" is causing confusion here (and no, I'm not sure which of those Collin actually meant either). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Mon, Feb 22, 2010 at 17:05, Collin Winter wrote:

Sorry, I meant "pull from". I want an updated snapshot of 2to3 for the benchmark suite, and I'm looking for the best place to grab it from.

Well, the server that has all the stuff for doing the conversions has annoyingly been down for about 24 hours now. I've got a friend coming in who should hopefully be fixing this tomorrow in the afternoon (Atlanta time), after which I should be able to make my conversion stuff update the hg.p.o repositories with more regularity. Cheers, Dirkjan

On Sat, Feb 13, 2010 at 12:14 PM, "Martin v. Löwis" wrote:

...

...

But isn't that just a theoretical property? I know that's how 2to3 started, but who, other than the committers, actually accesses the 2to3 repo?

It's used in 3to2 for example.

That doesn't really seem to be the case. AFAICT, 3to2 is a hg repository, with no inherent connection to the 2to3 svn sandbox. It does use lib2to3, but that could come either from an installed Python, from a trunk/3k checkout, or from the sandbox. Correct?

So if the 2.x trunk became the official master for (lib)2to3, nothing would really change for 3to3, right? (except for the comment in the readme that you should get 2to3 from the sandbox if the trunk copy doesn't work; this comment would become obsolete as changes *would* propagate immediately into the Python trunk).

...

...

I would be much more supportive of that view if there had been a single release of 2to3 at any point in time (e.g. to PyPI). Alas, partially due to me creating lib2to3, you actually couldn't release it as an extra application and run it on 2.6 or 2.7, as the builtin lib2to3 would take precedence over the lib2to3 bundled with the application.

It could be distributed under another name or provide a way to override the stdlib version.

Sure. However, I'm still claiming that this is theoretical. The only person who has shown a slight interest in having this as a separate project (since Collin Winter left) is you, and so far, you haven't made any efforts to produce a stand-alone release. I don't blame you at all for that, in fact, I think Python is better off with the status quo (i.e. changes to 2to3 get liberally released even with bug fix releases, basically in an exemption from the "no new features" policy - similar to -3 warnings).

I still think that the best approach for projects to use 2to3 is to run 2to3 at install time from a single-source release. For that, projects will have to adjust to whatever bugs certain 2to3 releases have, rather than requiring users to download a newer version of 2to3 that fixes them. For this use case, a tightly-integrated lib2to3 (with that name and sole purpose) is the best thing.

Regards, Martin

Yes, if the trunk were the official master for lib2to3, then 3to2 would not change at all. If fixes to lib2to3 were immediately propagated to the trunk, 3to2 would benefit from that. I support lib2to3's integration with the trunk... it's too confusing otherwise and kind of defeats the idea of "trunk": if lib2to3 is provided with Python, then shouldn't its latest version be in Python's trunk? --Joe Amenta

On Feb 13, 2010, at 1:31 AM, Martin v. Löwis wrote:

...

...

On Fri, Feb 12, 2010 at 11:17, "Martin v. Löwis" wrote:

...

IMO, it is realistic to predict that this will not actually happen. If we can agree to give up the 2to3 sandbox, we should incorporate find_pattern into the tree, and perhaps test.py as well. I vote on giving up the 2to3 sandbox.

Besides, if we're using hg, it should make it much easier for someone else to branch that part of the stdlib

Actually - no: hg doesn't support branching of parts of a repository. You would need to branch all of Python. Then, there wouldn't be a straight-forward place to setup.py and any other top-level files (although you could hack them into Lib, and work with a distutils manifest).

Does hg support an equivalent of 'bzr split'? % bzr split --help Purpose: Split a subdirectory of a tree into a separate tree. Usage: bzr split TREE Options: --usage Show usage message and options. -v, --verbose Display more information. -q, --quiet Only display errors and warnings. -h, --help Show help message. Description: This command will produce a target tree in a format that supports rich roots, like 'rich-root' or 'rich-root-pack'. These formats cannot be converted into earlier formats like 'dirstate-tags'. The TREE argument should be a subdirectory of a working tree. That subdirectory will be converted into an independent tree, with its own branch. Commits in the top-level tree will not apply to the new subtree. See also: join -Barry

2010/2/13 Dirkjan Ochtman :

On Sat, Feb 13, 2010 at 17:14, Barry Warsaw wrote:

...

Does hg support an equivalent of 'bzr split'?

% bzr split --help Purpose: Split a subdirectory of a tree into a separate tree. Usage:   bzr split TREE

Options:  --usage        Show usage message and options.  -v, --verbose  Display more information.  -q, --quiet    Only display errors and warnings.  -h, --help     Show help message.

Description:  This command will produce a target tree in a format that supports  rich roots, like 'rich-root' or 'rich-root-pack'.  These formats cannot be  converted into earlier formats like 'dirstate-tags'.

 The TREE argument should be a subdirectory of a working tree.  That  subdirectory will be converted into an independent tree, with its own  branch.  Commits in the top-level tree will not apply to the new subtree.

Is that like a clone/branch of a subdir of the original repository? We don't have what we usually call "narrow clones" yet (nor "shallow clones", the other potentially useful form of partial clones). We do have the convert extension, which allows you to create a new repository from a subtree of an old repository, but it changes all the hashes (and I don't know if we have a way to splice them back together).

It is not a partial clone, but rather similar to what you are referring to with the convert extension. -- Regards, Benjamin

Brett Cannon wrote:

On Fri, Feb 12, 2010 at 11:17, "Martin v. Löwis" wrote: I vote on giving up the 2to3 sandbox.

One other point - is there a Python 2.6 backwards compatibility restriction on 2to3 at the moment? If there isn't, should there be? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Benjamin Peterson wrote:

2010/2/12 Nick Coghlan :

...

Brett Cannon wrote:

...

On Fri, Feb 12, 2010 at 11:17, "Martin v. Löwis" wrote: I vote on giving up the 2to3 sandbox. One other point - is there a Python 2.6 backwards compatibility restriction on 2to3 at the moment? If there isn't, should there be?

I try to keep it compatible with 2.6, since we have to backport changes.

With 2.7 just around the corner, it should probably be listed in PEP 291 on that basis. Of course, PEP 291 could do with a list of 2.5 and 2.6 specific features first... Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Benjamin Peterson wrote:

2010/2/12 Nick Coghlan :

...

Of course, PEP 291 could do with a list of 2.5 and 2.6 specific features first...

I think that section is rather pointless to keep updated, since a good list can be found in the what's new documents. What people really need to do is run the unittests on all supported versions.

It's handy as a list of big ticket items to avoid, especially those that can significantly affect the way you structure code. Agreed that the main enforcement should be to run those tests on the relevant older versions. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Fri, Feb 12, 2010 at 09:39, Georg Brandl wrote:

No, it does not.  This is also a concern for the Python 2 -> Python 3 merging, where (I think) we decided not to have shared history.  Transplant already

I don't think this is similar to 2 vs. 3, because 2 vs. 3 are full branching (so you could still use "normal" hg merge tracking there). Since hg doesn't do merge tracking on the directory level, you couldn't use Mercurial merges (or transplant, AFAICS) to do what you want here.

does most of the job (including recording the source hash of transplanted changesets), but it lacks blocking and consistent rejection of already-merged changesets (it does not read the source hashes it records, but keeps a local cache of such hashes instead, which obviously doesn't do anything across repositories.)  I think it should be possible to have transplant regenerate and update that cache automatically on clone/pull/etc.

I guess this is a relatively simple task for a Mercurial hacker, and if it's decided to use this workflow "someone" ;) could address it at the PyCon sprint.

Yes, we should figure out some workflow issues soon. Cheers, Dirkjan

On Sun, Feb 7, 2010 at 22:58, Mark Hammond wrote:

Isn't setting a date premature while outstanding issues remain without a timetable for their resolution?

If we set a date, that would imply a timetable for their resolution.

See http://mercurial.selenic.com/wiki/EOLTranslationPlan#TODO - of particular note:

* There are transient errors in the tests which Martin is yet to identify.  These tests do not require windows to reproduce or fix.

* The mercurial tests do not run on Windows.

Given the above, most sane Windows developers would hold off on "live" testing of the extension until at least the first issue is resolved - but the second issue makes it very difficult for them to help resolve that.

The Mercurial tests can actually run on Windows -- and I've updated the page to that effect. They require something called pysh, though. I've also asked Patrick Mezard to include the eol extension in his nightly test run on Windows. I guess since some of the test errors do not require Windows to reproduce or fix, I'd invite anyone to jump in and help fix these issues. Cheers, Dirkjan

On Mon, Feb 22, 2010 at 18:38, Martin Geisler wrote:

My dissertation is due this Friday(!), so I will not have much time to look at EOL issues this week (as usual). But please give it a spin anyway and let us hear what you think!

I've got about 48 more hours of PyCon sprints ahead of me, so if anyone comes up with bugs (preferably concrete and reproducible) in that time frame, I can look into them more or less directly. Cheers, Dirkjan