Mercurial migration: progress report (PEP 385)
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though. - First of all, I've got the basic conversion down, I've done it a few times now, with progressively better results. You can view some results at http://hg.python.org/, which has a preliminary cpython repository. *** The changeset hashes for that repo will change, so you won't be able to commit or pull from it in the future.*** - Second of all, some planning. I've thought about it a bit, and I think we should aim for going live with hg on August 1. Given that I'm on vacation from 8-18 July (and I'm not sure whether I'll be able to actually work on it during that time, though I imagine I'll be able to spend some time on it at least), that's quite ambitious, so I'm going to say it's okay if it slips by a few days. Putting a deadline out there is a good thing, anyway. - Third of all, to make this possible, it would be helpful if I got more feedback on the PEP. Last time I raised it, there was virtually nothing. This time, I'll include it inline so there's hopefully less of a barrier to reviewing it. - Fourth, Mercurial 1.3 was just released! Bet you didn't see that coming. It's looking like a pretty good release, with an experimental version of the much-coveted subrepository support (like svn:externals). This also means that the latest version of hgsubversion, the tool I used for the conversion, will be more accessible for converting other projects. You know you want to! - Fifth, here's a list of things, off the top of my head, that still need doing: * Get agreement on branch strategy and branch processing (list of branches + proposed handling at http://hg.python.org/pymigr/file/tip/all-branches.txt) <--- PLEASE REVIEW * Get agreement on tag processing (first come up with a list) * Set up hg-ssh infra (should be easy) * Set up hooks (should be mostly straightforward) * Set up roundup integration (should be made easier by quick revision map hgweb extension) * Write docs - Sixth (this is the good part), less obvious things that have been done or don't need doing: * .hgignore generation (I've been convinced it's too hard, the current version will do) * revlog reordering (it's painless and a big win) I'll get through all of these myself, but obviously any help would be welcome. For any hg users, writing docs should be an easy start. For others, please review the PEP (below), the branch map in http://hg.python.org/pymigr/file/tip/all-branches.txt and the author map at http://hg.python.org/pymigr/file/tip/author-map (not that much has changed since the start, so if you've looked at it already, feel free to skip this part). Right now I'm a little stuck on branch processing, because it's a long running script that needs a bunch of debugging, but I'll get going on that again. I think that's all I can think of for now, I'll update the PEP with new bits soon. Here it is, ready for your review: ============================================================== Motivation After having decided to switch to the Mercurial DVCS, the actual migration still has to be performed. In the case of an important piece of infrastructure like the version control system for a large, distributed project like Python, this is a significant effort. This PEP is an attempt to describe the steps that must be taken for further discussion. It's somewhat similar to PEP 347 [1], which discussed the migration to SVN. To make the most of hg, I (Dirkjan) would like to make a high-fidelity conversion, such that (a) as much of the svn metadata as possible is retained, and (b) all metadata is converted to formats that are common in Mercurial. This way, tools written for Mercurial can be optimally used. In order to do this, I want to use the hgsubversion [2] software to do an initial conversion. This hg extension is focused on providing high-quality conversion from Subversion to Mercurial for use in two-way correspondence, meaning it doesn't throw away as much available metadata as other solutions. Such a conversion also seems like a good time to reconsider the contents of the repository and determine if some things are still valuable. In this spirit, the following sections also propose discarding some of the older metadata. Timeline TBD; needs fully working hgsubversion and consensus on this document. Transition plan Branch strategy Mercurial has two basic ways of using branches: cloned branches, where each branch is kept in a separate repository, and named branches, where each revision keeps metadata to note on which branch it belongs. The former makes it easier to distinguish branches, at the expense of requiring more disk space on the client. The latter makes it a little easier to switch between branches, but often has somewhat unintuitive results for people (though this has been getting better in recent versions of Mercurial). I'm still a bit on the fence about whether Python should adopt cloned branches and named branches. Since it usually makes more sense to tag releases on the maintenance branch, for example, mainline history would not contain release tags if we used cloned branches. Also, Mercurial 1.2 and 1.3 have the necessary tools to make named branches less painful (because they can be properly closed and closed heads are no longer considered in relevant cases). A disadvantage might be that the used clones will be a good bit larger (since they essentially contain all other branches as well). This can me mitigated by keeping non-release (feature) branches in separate clones. Also note that it's still possible to clone a single named branch from a combined clone, by specifying the branch as in hg clone http://hg.python.org/main/#2.6-maint. Keeping the py3k history in a separate clone problably also makes sense. XXX To do: size comparison for selected separation scenarios. Converting branches There are quite a lot of branches in SVN's branches directory. I propose to clean this up a bit, by employing the following the strategy: * Keep all release (maintenance) branches * Discard branches that haven't been touched in 18 months, unless somone indicates there's still interest in such a branch * Keep branches that have been touched in the last 18 months, unless someone indicates the branch can be deprecated Converting tags The SVN tags directory contains a lot of old stuff. Some of these are not, in fact, full tags, but contain only a smaller subset of the repository. I think we should keep all release tags, and consider other tags for inclusion based on requests from the developer community. I'd like to consider unifying the release tag naming scheme to make some things more consistent, if people feel that won't create too many problems. For example, Mercurial itself just uses '1.2.1' as a tag, where CPython would currently use r121. Author map In order to provide user names the way they are common in hg (in the 'First Last <user@example.org>' format), we need an author map to map cvs and svn user names to real names and their email addresses. I have a complete version of such a map in my migration tools repository [3]. The email addresses in it might be out of date; that's bound to happen, although it would be nice to try and have as many people as possible review it for addresses that are out of date. The current version also still seems to contain some encoding problems. Generating .hgignore The .hgignore file can be used in Mercurial repositories to help ignore files that are not eligible for version control. It does this by employing several possible forms of pattern matching. The current Python repository already includes a rudimentary .hgignore file to help with using the hg mirrors. It might be useful to have the .hgignore be generated automatically from svn:ignore properties. This would make sure all historic revisions also have useful ignore information (though one could argue ignoring isn't really relevant to just checking out an old revision). Revlog reordering As an optional optimization technique, we should consider trying a reordering pass on the revlogs (internal Mercurial files) resulting from the conversion. In some cases this results in dramatic decreases in on-disk repository size. Other repositories Richard Tew has indicated that he'd like the Stackless repository to also be converted. What other projects in the svn.python.org repository should be converted? Do we want to convert the peps repository? distutils? others? Infrastructure hg-ssh Developers should access the repositories through ssh, similar to the current setup. Public keys can be used to grant people access to a shared hg@ account. A hgwebdir instance should also be set up for easy browsing and read-only access. If we're using ssh, developers should trivially be able to start new clones (for longer-term features that profit from a separate branch). Hooks A number of hooks is currently in use. The hg equivalents for these should be developed and deployed. The following hooks are being used: * check whitespace: a hook to reject commits in case the whitespace doesn't match the rules for the Python codebase. Should be straightforward to re-implement from the current version. We can also offer a whitespace hook for use with client-side repositories that people can use; it could either warn about whitespace issues and/or truncate trailing whitespace from changed lines. Open issue: do we check only the tip after each push, or do we check every commit in a changegroup? * commit mails: we can leverage the notify extension for this * buildbots: both the regular and the community build masters must be notified. Fortunately buildbot includes support for hg. I've also implemented this for Mercurial itself, so I don't expect problems here. * check contributors: in the current setup, all changesets bear the username of committers, who must have signed the contributor agreement. In a DVCS, the committers are not necessarily the same people who push, and so we can't check if the committer is a contributor. We could use a hook to check if the committer is a contributor if we keep a list of registered contributors. hgwebdir A more or less stock hgwebdir installation should be set up. We might want to come up with a style to match the Python website. It may also be useful to build a quick extension to augment the URL rev parser so that it can also take r[0-9]+ args and come up with the matching hg revision. After migration Where to get code It needs to be decided where the hg repositories will live. I'd like to propose to keep the hgwebdir instance at hg.python.org. This is an accepted standard for many organizations, and an easy parallel to svn.python.org. The 2.7 (trunk) repo might live at http://hg.python.org/main/, for example, with py3k at http://hg.python.org/py3k/. For write access, developers will have to use ssh, which could be ssh://hg@hg.python.org/main/. A demo installation will be set up with a preliminary conversion so people can experiment and review; it can live at http://hg.python.org/example/. code.python.org was also proposed as the hostname. Personally, I think that using the VCS name in the hostname is good because it prevents confusion: it should be clear that you can't use svn or bzr for hg.python.org. hgwebdir can already provide tarballs for every changeset. I think this obviates the need for daily snapshots; we can just point users to tip.tar.gz instead, meaning they will get the latest. If desired, we could even use buildbot results to point to the last good changeset. Python-specific documentation hg comes with good built-in documentation (available through hg help) and a wiki [4] that's full of useful information and recipes. In addition to that, the parts of the developer FAQ [5] concerning version control will gain a section on using hg for Python development. Some of the text will be dependent on the outcome of debate about this PEP (for example, the branching strategy). Think first, commit later? In recent history, old versions of Python have been maintained by a select group of people backporting patches from trunk to release branches. While this may not scale so well as the development pace grows, it also runs into some problems with the current crop of distributed versioning tools. These tools (I believe similar problems would exist for either git, bzr, or hg, though some may cope better than others) are based on the idea of a Directed Acyclic Graph (or DAG), meaning they keep track of relations of changesets. Mercurial itself has a stable branch which is a ''strict'' subset of the unstable branch. This means that generally all fixes for the stable branch get committed against the tip of the stable branch, then they get merged into the unstable branch (which already contains the parent of the new cset). This provides a largely frictionless environment for moving changes from stable to unstable branches. Mistakes, where a change that should go on stable goes on unstable first, do happen, but they're usually easy to fix. That can be done by copying the change over to the stable branch, then trivial-merging with unstable -- meaning the merge in fact ignores the parent from the stable branch). This strategy means a little more work for regular committers, because they have to think about whether their change should go on stable or unstable; they may even have to ask someone else (the RM) before committing. But it also relieves a dedicated group of committers of regular backporting duty, in addition to making it easier to work with the tool. Now would be a good time to consider changing strategies in this regard, although it would be relatively easy to switch to such a model later on. The future of Subversion What happens to the Subversion repositories after the migration? Since the svn server contains a bunch of repositories, not just the CPython one, it will probably live on for a bit as not every project may want to migrate or it takes longer for other projects to migrate. To prevent people from staying behind, we may want to remove migrated projects from the repository. Build identification Python currently provides the sys.subversion tuple to allow Python code to find out exactly what version of Python it's running against. The current version looks something like this: * ('CPython', 'tags/r262', '71600') * ('CPython', 'trunk', '73128M') Another value is returned from Py_GetBuildInfo() in the C API, and available to Python code as part of sys.version: * 'r262:71600, Jun 2 2009, 09:58:33' * 'trunk:73128M, Jun 2 2009, 01:24:14' I propose that the revision identifier will be the short version of hg's revision hash, for example 'dd3ebf81af43', augmented with '+' (instead of 'M') if the working directory from which it was built was modified. This mirrors the output of the hg id command, which is intended for this kind of usage. For the tag/branch identifier, I propose that hg will check for tags on the currently checked out revision, use the tag if there is one ('tip' doesn't count), and uses the branch name otherwise. sys.subversion becomes * ('CPython', '2.6.2', 'dd3ebf81af43') * ('CPython', 'default', 'af694c6a888c+') and the build info string becomes * '2.6.2:dd3ebf81af43, Jun 2 2009, 09:58:33' * 'default:af694c6a888c+, Jun 2 2009, 01:24:14' This reflects that the default branch in hg is called 'default' instead of Subversion's 'trunk', and reflects the proposed new tag format. References [1] http://www.python.org/dev/peps/pep-0347/ [2] http://bitbucket.org/durin42/hgsubversion/ [3] http://hg.xavamedia.nl/cpython/pymigr/ [4] http://www.selenic.com/mercurial/wiki/ [5] http://www.python.org/dev/faq/#version-control ===================================================== Cheers, Dirkjan
On 3/07/2009 6:42 AM, Dirkjan Ochtman wrote:
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though.
Although this has come up in the past, I don't recall a resolution. What is your plan to handle svn:eol-style? We have some files in the tree which need that support and it isn't clear to me how that would work with the existing win32text extension provided with current mercurial releases. (I've an outstanding patch to hg which should address some of these issues, but without the 'rules' being versioned I fear that would still fall short.) Even more generally, how will you suggest Windows users work? Will local files, in general, have windows line endings or unix? If the latter, will there be hooks in-place to prevent editors on Windows 'accidently' mixing eol styles? If so, this cycles back to the first question - how would we know which files get treated that way? Thanks, Mark
On Fri, Jul 3, 2009 at 01:01, Mark Hammond<skippy.hammond@gmail.com> wrote:
Although this has come up in the past, I don't recall a resolution.
What is your plan to handle svn:eol-style? We have some files in the tree which need that support and it isn't clear to me how that would work with the existing win32text extension provided with current mercurial releases. (I've an outstanding patch to hg which should address some of these issues, but without the 'rules' being versioned I fear that would still fall short.)
What files would need what? Are there any files that really need to be \r\n on Windows and \n on Unix (and possibly \r on Mac)? I remember one file was discussed separately, but I think the outcome there was that it could just always be \r\n (since it wasn't used at all on non-Windows platforms). Anyway, knowing specific requirements (or where to find them) would help here.
Even more generally, how will you suggest Windows users work? Will local files, in general, have windows line endings or unix? If the latter, will there be hooks in-place to prevent editors on Windows 'accidently' mixing eol styles? If so, this cycles back to the first question - how would we know which files get treated that way?
There will be a server-side hook to check whitespace. People will also be able to install it for commit-time. I think just using \n by default everywhere is a good default (though I almost always use Windows client machine, I do all nearly all of my development through a terminal on several Linux boxen), except where it isn't. People who want to use can set up the win32text stuff to get \r\n on Windows if they feel they need that -- we can provide information about that in the dev FAQ (although it would be nice if someone else who was more familiar with it -- like yourself! -- would write it). Cheers, Dirkjan
On Fri, Jul 3, 2009 at 01:01, Mark Hammond<skippy.hammond@gmail.com> wrote:
Although this has come up in the past, I don't recall a resolution.
What is your plan to handle svn:eol-style? We have some files in the tree which need that support and it isn't clear to me how that would work with the existing win32text extension provided with current mercurial releases. (I've an outstanding patch to hg which should address some of these issues, but without the 'rules' being versioned I fear that would still fall short.)
What files would need what? Are there any files that really need to be \r\n on Windows and \n on Unix (and possibly \r on Mac)? I remember one file was discussed separately, but I think the outcome there was that it could just always be \r\n (since it wasn't used at all on non-Windows platforms). Anyway, knowing specific requirements (or where to find them) would help here.
Even more generally, how will you suggest Windows users work? Will local files, in general, have windows line endings or unix? If the latter, will there be hooks in-place to prevent editors on Windows 'accidently' mixing eol styles? If so, this cycles back to the first question - how would we know which files get treated that way?
There will be a server-side hook to check whitespace. People will also be able to install it for commit-time.
I think just using \n by default everywhere is a good default (though I almost always use Windows client machine, I do all nearly all of my development through a terminal on several Linux boxen), except where it isn't. So we must work without effective EOL support? I fear we will end up
On 3/07/2009 9:28 PM, Dirkjan Ochtman wrote: like the mozilla hg repo with some files in windows line endings and some with linux. While my editing tools are good enough to preserve existing EOL styles, I've found myself accidentally checking in new \r\n terminated files in a repo which otherwise uses \n line endings. IMO, SVN's EOL support was better than no EOL support.
People who want to use can set up the win32text stuff to get \r\n on Windows if they feel they need that -- we can provide information about that in the dev FAQ (although it would be nice if someone else who was more familiar with it -- like yourself! -- would write it).
This is exactly why I was asking for your advice - I can't work out how to work effectively with win32text as it stands myself, so remain stuck accidently checking in files with inappropriate line endings and stuck working out how to move pywin32's CVS repo with abandoning the very primitive EOL safety it offers... Cheers, Mark
On Fri, Jul 3, 2009 at 15:31, Mark Hammond<mhammond@skippinet.com.au> wrote:
So we must work without effective EOL support? I fear we will end up like the mozilla hg repo with some files in windows line endings and some with linux. While my editing tools are good enough to preserve existing EOL styles, I've found myself accidentally checking in new \r\n terminated files in a repo which otherwise uses \n line endings. IMO, SVN's EOL support was better than no EOL support.
This is why we'll have hooks -- to prevent you from pushing changesets with inappropriate, to say the least, and, if you're willing to do a little bit of extra work, to prevent you from committing them.
This is exactly why I was asking for your advice - I can't work out how to work effectively with win32text as it stands myself, so remain stuck accidently checking in files with inappropriate line endings and stuck working out how to move pywin32's CVS repo with abandoning the very primitive EOL safety it offers...
As long as the difference between \r\n- and \n-based files is clear and can be reasoned about, I don't see why having some of both (I'm assuming an overwhelming majority will have one, and only a few the other) is a big problem. But feel free to enlighten me! Cheers, Dirkjan
This is exactly why I was asking for your advice - I can't work out how to work effectively with win32text as it stands myself, so remain stuck accidently checking in files with inappropriate line endings and stuck working out how to move pywin32's CVS repo with abandoning the very primitive EOL safety it offers...
As long as the difference between \r\n- and \n-based files is clear and can be reasoned about, I don't see why having some of both (I'm assuming an overwhelming majority will have one, and only a few the other) is a big problem. But feel free to enlighten me!
If "both" means "both the server side test, and win32text", then Mark already gave the answer: he cannot use win32test because he does not know how to operate it (and doesn't bother studying its source code to understand it in detail). Regards, Martin
On 3/07/2009 11:43 PM, Dirkjan Ochtman wrote:
On Fri, Jul 3, 2009 at 15:31, Mark Hammond<mhammond@skippinet.com.au> wrote:
So we must work without effective EOL support? I fear we will end up like the mozilla hg repo with some files in windows line endings and some with linux. While my editing tools are good enough to preserve existing EOL styles, I've found myself accidentally checking in new \r\n terminated files in a repo which otherwise uses \n line endings. IMO, SVN's EOL support was better than no EOL support.
This is why we'll have hooks -- to prevent you from pushing changesets with inappropriate, to say the least, and, if you're willing to do a little bit of extra work, to prevent you from committing them.
This is exactly why I was asking for your advice - I can't work out how to work effectively with win32text as it stands myself, so remain stuck accidently checking in files with inappropriate line endings and stuck working out how to move pywin32's CVS repo with abandoning the very primitive EOL safety it offers...
As long as the difference between \r\n- and \n-based files is clear and can be reasoned about, I don't see why having some of both (I'm assuming an overwhelming majority will have one, and only a few the other) is a big problem. But feel free to enlighten me!
I'm surprised it isn't obvious. Think about it this way: CVS had basic EOL support and SVN gave slightly better support, and we leveraged this support in both those tools. The end result is a very clean repo with consistent line endings and a distinct lack of developer friction between the platforms. Your position seems to be that we *never* needed EOL conversion, especially in SVN which offered decent hooks, and that having a source repo with a mismatch and unpredictable line endings isn't really a problem at all. Even just at face value, it seems clear we have taken full advantage of that feature for around 1.5 decades, so to suggest we never actually needed it, or at least don't need it now for some reason, seems slightly disingenuous... Cheers, Mark
On 3/07/2009 11:43 PM, Dirkjan Ochtman wrote:
As long as the difference between \r\n- and \n-based files is clear and can be reasoned about, I don't see why having some of both (I'm assuming an overwhelming majority will have one, and only a few the other) is a big problem. But feel free to enlighten me!
In that case, and given that I expect more Windows users to be clueless about EOL issues than users of other operating systems, I propose we check in all files initially with Windows line endings <wink> Mark
Mark Hammond wrote:
So we must work without effective EOL support? I fear we will end up like the mozilla hg repo with some files in windows line endings and some with linux. While my editing tools are good enough to preserve existing EOL styles, I've found myself accidentally checking in new \r\n terminated files in a repo which otherwise uses \n line endings. IMO, SVN's EOL support was better than no EOL support.
If Mercury doesn't have automatic whitespace conversion along the lines of svn:eol-style, then the existing white-space checking script probably needs to be updated to enforce the appropriate line endings for all files. If we default to Unix line endings for most files, then the checking script can be made aware of which files should always have Windows line endings (I believe the various Visual Studio files need them, batch files are probably best left with Windows line endings in the repository, and I expect there are other files in PC and PCbuild that need them as well). However, I expect that would still be painful to work with for Windows developers, even if it prevented any line ending problems from making their way into the main repository. I believe that is where the win32text extensions can help. Looking at the Wiki page for win32text [1], I believe it would be a matter of configuring the extension to encode and decode all files with the extensions: *.py *.pyw *.h *.c *.in *.rst *.asdl That said, I don't see a way to tell win32text to also translate files which don't have an extension at all (e.g. NEWS or ACKS), and there doesn't seem to be a way to tell it to *skip* files matching certain patterns (if there was, we could just tell it to ignore extensions like .bat, .sln, .vcproj, .vsprops, .dps, .dsw, .wse, .ico, .bmp and convert everything else) Cheers, Nick. [1] http://mercurial.selenic.com/wiki/Win32TextExtension -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On 4/07/2009 12:04 AM, Nick Coghlan wrote:
However, I expect that would still be painful to work with for Windows developers, even if it prevented any line ending problems from making their way into the main repository. I believe that is where the win32text extensions can help. Looking at the Wiki page for win32text [1], I believe it would be a matter of configuring the extension to encode and decode all files with the extensions:
*.py *.pyw *.h *.c *.in *.rst *.asdl
That said, I don't see a way to tell win32text to also translate files which don't have an extension at all (e.g. NEWS or ACKS), and there doesn't seem to be a way to tell it to *skip* files matching certain patterns (if there was, we could just tell it to ignore extensions like .bat, .sln, .vcproj, .vsprops, .dps, .dsw, .wse, .ico, .bmp and convert everything else)
The big problem I have with win32text is that the rules are not versioned, meaning we will rely on every (windows only?) developer manually enabling the win32text extension, then manually adding these rules to an unversioned hgrc file. Should the rules change over time, we will need to communicate that to every developer. Contrast this with, say, SVN, where the rule is a versioned property on the file. Further, if a repo *already* has mixed line endings and the file doesn't appear in the rules, win32text makes *lots* of noise regarding existing line endings and to "review your rules before you commit." In effect, it seems to me that win32text will only work effectively if *all* developers using a repo are using it, or none are, or other steps have been put in place to ensure \r\n line endings in the repo are otherwise impossible. I've completely failed to use win32text on the mozilla hg repo for exactly this reason. Cheers, Mark
Mark Hammond wrote:
On 4/07/2009 12:04 AM, Nick Coghlan wrote: The big problem I have with win32text is that the rules are not versioned, meaning we will rely on every (windows only?) developer manually enabling the win32text extension, then manually adding these rules to an unversioned hgrc file. Should the rules change over time, we will need to communicate that to every developer. Contrast this with, say, SVN, where the rule is a versioned property on the file.
Further, if a repo *already* has mixed line endings and the file doesn't appear in the rules, win32text makes *lots* of noise regarding existing line endings and to "review your rules before you commit." In effect, it seems to me that win32text will only work effectively if *all* developers using a repo are using it, or none are, or other steps have been put in place to ensure \r\n line endings in the repo are otherwise impossible. I've completely failed to use win32text on the mozilla hg repo for exactly this reason.
I didn't realise Mercurial didn't have a way for a repository to provide versioned extension settings, but in this case I think using our own server side hook can deal with the problem (either adding it to the existing whitespace hook that checks for tab/space sanity in Python files, or else as a separate hook that just checks line endings). Such a hook should be able to base the determination of which files it checks for \r\n line endings on a normal versioned file in the repository. For everything that *isn't* mentioned in that file, the hook would check that it had \n line endings. I believe that would address both of your concerns: 1. A server side hook means that the rules are enforced on everyone regardless of whether they use win32text or not (e.g no Linux developers corrupting a .dsp file by opening it and saving it with \n line endings) 2. The server side hooks rules will be versioned, allowing people to copy them to their hgrc file so that win32text will correctly handle the line endings for them. Also, on another read of the win32text instructions, I believe the configuration actually *does* support negative filters, it just doesn't explain them very well (it only mentions them in the context of overriding default settings in the config file for a specific repository). I think a .hgrc file along the following lines might do what we want (I'm not developing on Windows though, so I haven't actually tested it): ============= [extensions] hgext.win32text= [encode] # Encode files that don't contain NUL characters. ** = cleverencode: # But leave Windows specific and known binary files alone **.bat = ! **.sln = ! **.vcproj = ! **.vsprops = ! **.dps = ! **.dsw = ! **.wse = ! **.ico = ! **.bmp = ! [decode] # Decode files that don't contain NUL characters. ** = cleverdecode: # But leave Windows specific and known binary files alone **.bat = ! **.sln = ! **.vcproj = ! **.vsprops = ! **.dps = ! **.dsw = ! **.wse = ! **.ico = ! **.bmp = ! ============= Assuming we can find settings that do what we want then a sample HGRC file could easily be provided in the Misc directory alongside things like python-mode.el. An alternative to the above would be to store *everything* (even the Windows specific files) in the repository with \n line endings (still enforced by a server side hook), and then *require* that Windows users run win32text with settings like the following: ============= [extensions] hgext.win32text= [encode] # Encode files that don't contain NUL characters. ** = cleverencode: # But leave known binary files alone **.ico = ! **.bmp = ! [decode] # Decode files that don't contain NUL characters. ** = cleverdecode: # But leave known binary files alone **.ico = ! **.bmp = ! ============= Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On 4/07/2009 11:20 AM, Nick Coghlan wrote:
I didn't realise Mercurial didn't have a way for a repository to provide versioned extension settings, but in this case I think using our own server side hook can deal with the problem (either adding it to the existing whitespace hook that checks for tab/space sanity in Python files, or else as a separate hook that just checks line endings).
Such a hook should be able to base the determination of which files it checks for \r\n line endings on a normal versioned file in the repository. For everything that *isn't* mentioned in that file, the hook would check that it had \n line endings.
I believe that would address both of your concerns: 1. A server side hook means that the rules are enforced on everyone regardless of whether they use win32text or not (e.g no Linux developers corrupting a .dsp file by opening it and saving it with \n line endings) 2. The server side hooks rules will be versioned, allowing people to copy them to their hgrc file so that win32text will correctly handle the line endings for them.
That sounds like it should work - presumably mention of this should go into the PEP. (While that is great for Python, it is a shame I probably can't use the same strategy for other hg repos I care about as I don't have access to server side hooks - but that if off-topic for this list.)
Also, on another read of the win32text instructions, I believe the configuration actually *does* support negative filters, it just doesn't explain them very well (it only mentions them in the context of overriding default settings in the config file for a specific repository). I think a .hgrc file along the following lines might do what we want (I'm not developing on Windows though, so I haven't actually tested it):
This *appears* to be correct at first glance, but in practice it doesn't interact well with wildcards specifications - particularly '**=cleverencode'. I started a thread on the ML and submitted a patch a few months ago to fix this and it was accepted, but sadly it seemed to have dropped off the queue somewhere. The patch now conflicts, and I've promised to resubmit it when I find time. But even with that in place it doesn't address the more general problem that when only *some* developers use the extension - mixed mode files are still very possible, at which point win32text starts reporting enough spurious conflicts to become unusable for me. Eg, doing a clean checkout of mozilla with win32text enabled results in a working tree with hundreds of files reporting every line in the file has changed. It is worth noting that win32text also works on Linux, and that in theory, win32text should be able to allow Linux users to work seamlessly with a \r\n repo, in the same way it promises to allow Windows users to work with \n repos. But in practice, it is simply never used by Linux developers, even in the hg core dev team. Patches or comments regarding win32text are always met with "but no one here ever uses it". If they did, and some of the core hg team tried to experiment with win32text and mixed line ending repos the problems would, I believe, become self-evident... Cheers, Mark
Mark Hammond wrote:
This *appears* to be correct at first glance, but in practice it doesn't interact well with wildcards specifications - particularly '**=cleverencode'. I started a thread on the ML and submitted a patch a few months ago to fix this and it was accepted, but sadly it seemed to have dropped off the queue somewhere. The patch now conflicts, and I've promised to resubmit it when I find time. But even with that in place it doesn't address the more general problem that when only *some* developers use the extension - mixed mode files are still very possible, at which point win32text starts reporting enough spurious conflicts to become unusable for me. Eg, doing a clean checkout of mozilla with win32text enabled results in a working tree with hundreds of files reporting every line in the file has changed.
Ah, the scope of the issues begin to become clear... While a server side hook should be able to deal with the mixed mode file problem, I'm not sure what can be done about the problems with properly configuring win32text. If wildcard specifications don't interact properly with filtered negative specifications then that would appear to rule out that approach. The presence of text files without extensions like NEWS and ACKS in the Python repository appears to rule out the use of positive filters to select only the files that are stored in the repository with \n line endings. I spent some more time exploring the approach recommended on the win32text documentation page as a possible way of handling the situation: 1. Store all text files (even Windows specific ones) in \n format in the repository 2. Apply the win32text precommit hook to forbid the introduction of \r\n line endings 3. Use the recommended settings from the win32text documentation: =============================== [extensions] hgext.win32text= [encode] # Encode files that don't contain NUL characters. ** = cleverencode: [decode] # Decode files that don't contain NUL characters. ** = cleverdecode: =============================== This would be the equivalent of setting "svn:eol-style native" on every file in the repository. However, in running Martin's query (svn pg -R svn:eol-style .|grep CRLF) over 2.x and 3.x checkouts, I found that \r\n line endings are currently enforced for: - .bat files under Tools/buildbot - .dsp and .dsw files under PC/VC6 - Lib/email/test/data/msg_26.txt I believe the rationale for the first two is to allow a source tarball to be prepared on Linux but still be usable on Windows (e.g. see [1]). I'm not clear on the rationale for the explicit CRLF line ending on the email test message, but I would guess it is to ensure that the email module can handle CRLF line endings correctly regardless of platform. Only VC6 files appear on the list because later versions of Visual Studio actually tolerate \n line endings in their project files. Mercurial's heuristic handling of text vs binary and expected line endings fails completely for these use cases. I think there needs to be a solid answer in place for these use cases before the actual migration to Mercurial takes place. A hand-waving "use win32text" isn't enough - it needs to be "use win32text with these exact settings" (with server side hook support to enforce the rules). And since Mercurial doesn't even allow us to say "this is a binary file" the way CVS used to I'm currently not seeing any way for that to happen except for win32text to be updated to correctly handle wild cards in combination with negative filters. Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2006-March/062225.html -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On 4/07/2009 12:30 PM, Nick Coghlan wrote: ...
I think there needs to be a solid answer in place for these use cases before the actual migration to Mercurial takes place. A hand-waving "use win32text" isn't enough - it needs to be "use win32text with these exact settings" (with server side hook support to enforce the rules).
And since Mercurial doesn't even allow us to say "this is a binary file" the way CVS used to I'm currently not seeing any way for that to happen except for win32text to be updated to correctly handle wild cards in combination with negative filters.
I agree with your conclusion. My ruminating on this over the last few months leaves me thinking this would involve: * my older 'accepted but then lost' hg patch to allow an explicit 'none' rule for a single file to override wildcards. * win32text be enhanced to use a normal versioned file in the root of the repo, much like hgingore, where a project can maintain project wide rules. * win32text be enhanced such that all python developers, regardless of platform, are willing to use this extension, even if the majority of files happen to use their native line ending (sauce for the goose is sauce for the gander, and all that...) * commit hooks be implemented to enforce this - but this should not be necessary if the above was implemented and socially enforced. Cheers, Mark
Cheers, Nick.
[1] http://mail.python.org/pipermail/python-dev/2006-March/062225.html
Sorry Dirkjan - I just noticed I didn't CC you on this mail originally. I'm wondering if you have any more thoughts on these EOL issues and if there is anything I can do to help? Cheers, Mark On 4/07/2009 2:03 PM, Mark Hammond wrote:
On 4/07/2009 12:30 PM, Nick Coghlan wrote: ...
I think there needs to be a solid answer in place for these use cases before the actual migration to Mercurial takes place. A hand-waving "use win32text" isn't enough - it needs to be "use win32text with these exact settings" (with server side hook support to enforce the rules).
And since Mercurial doesn't even allow us to say "this is a binary file" the way CVS used to I'm currently not seeing any way for that to happen except for win32text to be updated to correctly handle wild cards in combination with negative filters.
I agree with your conclusion. My ruminating on this over the last few months leaves me thinking this would involve:
* my older 'accepted but then lost' hg patch to allow an explicit 'none' rule for a single file to override wildcards.
* win32text be enhanced to use a normal versioned file in the root of the repo, much like hgingore, where a project can maintain project wide rules.
* win32text be enhanced such that all python developers, regardless of platform, are willing to use this extension, even if the majority of files happen to use their native line ending (sauce for the goose is sauce for the gander, and all that...)
* commit hooks be implemented to enforce this - but this should not be necessary if the above was implemented and socially enforced.
Cheers,
Mark
Cheers, Nick.
[1] http://mail.python.org/pipermail/python-dev/2006-March/062225.html
Nick Coghlan writes:
I'm not clear on the rationale for the explicit CRLF line ending on the email test message, but I would guess it is to ensure that the email module can handle CRLF line endings correctly regardless of platform.
IIRC, that's just the standard for text email. Lines must be less than 1000 characters long, should be no more than 78 characters long, and are terminated by CRLF. As they say, "honored more in the breach than the observance", but that's what the RFCs say.
So we must work without effective EOL support?
If that's the case (i.e. no effective EOL support, the way svn supported it), then I think the PEP should make that clear (e.g. in a discussion section). For the server-side hooks: it would be good to know exactly what they check, wrt. line endings. To find out what files should not be stored with LF line endings, do "svn pg -R svn:eol-style .|grep CRLF". For win32text, it would probably be good if the FAQ would provide the relevant configuration instructions; it would be really helpful if somebody familiar with Windows and hg could provide detailed instructions well in advance of August 1. If we don't have anybody familiar with Windows and hg, we have a really serious problem. Regards, Martin
2009/7/2 Dirkjan Ochtman <dirkjan@ochtman.nl>:
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though.
Thanks very much for working on this, Dirkjan. It may seem rather thankless now, but I'm sure once we wish to Mercurial, the praise will flow. :)
- Second of all, some planning. I've thought about it a bit, and I think we should aim for going live with hg on August 1. Given that I'm on vacation from 8-18 July (and I'm not sure whether I'll be able to actually work on it during that time, though I imagine I'll be able to spend some time on it at least), that's quite ambitious, so I'm going to say it's okay if it slips by a few days. Putting a deadline out there is a good thing, anyway.
Sounds good.
- Fifth, here's a list of things, off the top of my head, that still need doing:
* Get agreement on branch strategy and branch processing (list of branches + proposed handling at http://hg.python.org/pymigr/file/tip/all-branches.txt) <--- PLEASE REVIEW
The io-c branch doesn't need to stay.
* Get agreement on tag processing (first come up with a list) * Set up hg-ssh infra (should be easy) * Set up hooks (should be mostly straightforward) * Set up roundup integration (should be made easier by quick revision map hgweb extension) * Write docs
- Sixth (this is the good part), less obvious things that have been done or don't need doing:
I suppose this includes modifying sys.subversion as described in the PEP?
The SVN tags directory contains a lot of old stuff. Some of these are not, in fact, full tags, but contain only a smaller subset of the repository. I think we should keep all release tags, and consider other tags for inclusion based on requests from the developer community. I'd like to consider unifying the release tag naming scheme to make some things more consistent, if people feel that won't create too many problems. For example, Mercurial itself just uses '1.2.1' as a tag, where CPython would currently use r121.
+1 to unifying tag name style to the current cpython procedure.
Author map
In order to provide user names the way they are common in hg (in the 'First Last <user@example.org>' format), we need an author map to map cvs and svn user names to real names and their email addresses. I have a complete version of such a map in my migration tools repository [3]. The email addresses in it might be out of date; that's bound to happen, although it would be nice to try and have as many people as possible review it for addresses that are out of date. The current version also still seems to contain some encoding problems. Generating .hgignore
What effect will the encoding problems have? Does hg require ASCII characters?
Richard Tew has indicated that he'd like the Stackless repository to also be converted. What other projects in the svn.python.org repository should be converted? Do we want to convert the peps repository? distutils? others?
I think everything should be converted unless there's a reason not to. (such as the maintainer indicates she doesn't what to migrate)
A number of hooks is currently in use. The hg equivalents for these should be developed and deployed. The following hooks are being used:
* check whitespace: a hook to reject commits in case the whitespace doesn't match the rules for the Python codebase. Should be straightforward to re-implement from the current version. We can also offer a whitespace hook for use with client-side repositories that people can use; it could either warn about whitespace issues and/or truncate trailing whitespace from changed lines. Open issue: do we check only the tip after each push, or do we check every commit in a changegroup?
It might as well be on every commit because it will have to normalized on push anyway.
code.python.org was also proposed as the hostname. Personally, I think that using the VCS name in the hostname is good because it prevents confusion: it should be clear that you can't use svn or bzr for hg.python.org.
+1 for hg.python.org
Think first, commit later?
In recent history, old versions of Python have been maintained by a select group of people backporting patches from trunk to release branches. While this may not scale so well as the development pace grows, it also runs into some problems with the current crop of distributed versioning tools. These tools (I believe similar problems would exist for either git, bzr, or hg, though some may cope better than others) are based on the idea of a Directed Acyclic Graph (or DAG), meaning they keep track of relations of changesets.
The problem is that Python is much more complicated than the average project. We have many commits that are only applicable one maintenance branch, or just 2.x, or just 3.x; the trunk and py3k will never be subsets of each other. Regardless of where we make commits initially, we need to ability to manage special cases like this easily. -- Regards, Benjamin
Dirkjan Ochtman writes:
Mercurial has two basic ways of using branches: cloned branches, where each branch is kept in a separate repository, and named branches, where each revision keeps metadata to note on which branch it belongs. The former makes it easier to distinguish branches, at the expense of requiring more disk space on the client. The latter makes it a little easier to switch between branches, but often has somewhat unintuitive results for people (though this has been getting better in recent versions of Mercurial).
I'll have to try them again now that 1.3 is out, but I found Mercurial named branches fundamentally unsuited to release management.
I'm still a bit on the fence about whether Python should adopt cloned branches and named branches. Since it usually makes more sense to tag releases on the maintenance branch, for example, mainline history would not contain release tags if we used cloned branches.
Ditto named branches. The problem is that (unless the internal implementation has changed very recently) a Mercurial revision can be on exactly one named branch (or on the trunk).
A disadvantage might be that the used clones will be a good bit larger (since they essentially contain all other branches as well). This can me mitigated by keeping non-release (feature) branches in separate clones.
Which defeats the purpose of having named branches, really. (I mean the version control purpose; obviously it still can save disk space.)
Also note that it's still possible to clone a single named branch from a combined clone, by specifying the branch as in hg clone http://hg.python.org/main/#2.6-maint.
Unless you're really short on space, though, that's not a big deal. What would be more important to me (not that I matter for the purpose of Python, but in XEmacs -- also a Mercurial shop -- I do :-) would be the other way around: pulling an external branch into a named branch. I have a feeling that working with such a repository with others would be a little difficult.
too many problems. For example, Mercurial itself just uses '1.2.1' as a tag, where CPython would currently use r121.
Stick with the CPython notation. At XEmacs, continuity of tags has made our beta testers happy. (Well, the two of them who bothered to mention it, anyway. :-)
code.python.org was also proposed as the hostname. Personally, I think that using the VCS name in the hostname is good because it prevents confusion: it should be clear that you can't use svn or bzr for hg.python.org.
Agreed, although "can't" is a little too strong. It might work (there are a lot of places where http://ftp.example.com works just fine, for example), but we don't want people to expect it to, and "http://REPOHOST.python.org/" should take your browser or your client to the official repo (which will be the hg repo), not to some index of repos that happen to live on the same host.
Mercurial itself has a stable branch which is a ''strict'' subset of the unstable branch.
As others (MvL, I think) have commented, this isn't really relevant to Python which generally has four mainlines going at once. I don't see why the requirements are going to change with the shift to hg, and I see no reason why hg won't handle the existing workflow just fine. Note that PEP 374 was written on the assumption that the existing workflow will *not* change until the committers have gotten used to Mercurial, and then it will change in the natural way. Ie, some one will say, "you know, this bit of red tape isn't needed any more if we do X, so let's do X", and after a cascade of 100*"+1 " we do X. :-) Other than that, looks to be shaping up well. (Note: I don't have any comments on subversion-specific aspects, as XEmacs went directly from CVS to Mercurial).
On Fri, Jul 3, 2009 at 15:29, Stephen J. Turnbull<stephen@xemacs.org> wrote:
I'll have to try them again now that 1.3 is out, but I found Mercurial named branches fundamentally unsuited to release management.
Can you explain why, please? It's not clear from what you say below.
Ditto named branches. The problem is that (unless the internal implementation has changed very recently) a Mercurial revision can be on exactly one named branch (or on the trunk).
That's still true.
Which defeats the purpose of having named branches, really. (I mean the version control purpose; obviously it still can save disk space.)
Why does it defeat the purpose? What, in your opinion, is the purpose?
Unless you're really short on space, though, that's not a big deal. What would be more important to me (not that I matter for the purpose of Python, but in XEmacs -- also a Mercurial shop -- I do :-) would be the other way around: pulling an external branch into a named branch. I have a feeling that working with such a repository with others would be a little difficult.
Can you give an example?
Stick with the CPython notation. At XEmacs, continuity of tags has made our beta testers happy. (Well, the two of them who bothered to mention it, anyway. :-)
Right; Benjamin also mentioned that processing the tags just to be consistent with the recent tagging scheme would probably be the best solution.
As others (MvL, I think) have commented, this isn't really relevant to Python which generally has four mainlines going at once. I don't see why the requirements are going to change with the shift to hg, and I see no reason why hg won't handle the existing workflow just fine.
It will handle it, for sure, but I think it would all go easier if we could work with stricter subset branches (and leave the effective cherrypicking for the occasional problem). Cheers, Dirkjan
It will handle it, for sure, but I think it would all go easier if we could work with stricter subset branches (and leave the effective cherrypicking for the occasional problem).
So I think the PEP should propose a workflow (or: merge flow) if you think we would be better off with a different one. In proposing such a workflow, consider these requirements: - we current have four active "maintenance" branches (i.e. where the entire code basis evolves): trunk, 3k, 2.6, and 3.1 (3.0 also until this morning). - in addition, we have two security branches currently: 2.4 and 2.5, although 2.4 will be closed soon. - our committers consistently refuse to merge changes across branches themselves, and likely continue to do so unless there is some feature of hg that I missed (e.g. one were merging would happen without any user specifically asking for it) Regards, Martin
Martin v. Löwis schrieb:
It will handle it, for sure, but I think it would all go easier if we could work with stricter subset branches (and leave the effective cherrypicking for the occasional problem).
So I think the PEP should propose a workflow (or: merge flow) if you think we would be better off with a different one.
In proposing such a workflow, consider these requirements: - we current have four active "maintenance" branches (i.e. where the entire code basis evolves): trunk, 3k, 2.6, and 3.1 (3.0 also until this morning).
It seems that there is a consensus to separate the 2.x and 3.x repos, and that also makes sense to me. Using named branches for the maintenance branches should be possible, but I'm not opposed to using cloned branches either. What I really want to see is the common-subset approach for maintenance branches. Changesets still have to be transplanted from 2.x to 3.x or the other way round.
- in addition, we have two security branches currently: 2.4 and 2.5, although 2.4 will be closed soon. - our committers consistently refuse to merge changes across branches themselves, and likely continue to do so unless there is some feature of hg that I missed (e.g. one were merging would happen without any user specifically asking for it)
If the checkin is done in the proper (the maint) branch, at least merging of that change is automatic whenever someone does a hg merge. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
Georg Brandl writes:
What I really want to see is the common-subset approach for maintenance branches.
IMO, this unfortunately unlikely to work as you seem to expect, for a two technical reasons and for a social reason. The first technical reason is that a maintenance branch really is a branch, not a subset. The fix that is appropriate for a maintenance branch is often inappropriate in detail for the mainline, and vice versa. But Mercurial doesn't care about "in detail" vs. "in design"; both will result in a conflict the first time that branch is merged. Second, some fixes for the maintenance branch will simply not be appropriate for the development branch, as the problem has already been fixed "en passant" by some other change. This can probably be handled by doing what git calls an "ours" merge to make it look like the unnecessary patch is an ancestor of the tip, even though no code was actually applied to the mainline. However, this kind of operation is some what delicate, and even if it's mostly scripted, it's likely to be somewhat unreliable for people who don't use the script very often ... which leads to the social problem:
- our committers consistently refuse to merge changes across branches themselves, and likely continue to do so unless there is some feature of hg that I missed (e.g. one were merging would happen without any user specifically asking for it)
If the checkin is done in the proper (the maint) branch, at least merging of that change is automatic whenever someone does a hg merge.
Maybe. But I see two problematic sides to this from the social point of view, which is the same old problem really. First, to the extent that it doesn't run into the technical problems, it encourages people to *not* review patches for each branch they are committed to. "It will get automerged anyway." Anything that discourages review is a bad thing. It will cause the development branch to "age" faster because of accumulation of crufty patches that are good enough as minimally invasive fixes for bugs in a maintenance branch, but which should be more robust for the development branch. I think you will also find that some people are not particularly interested in fixing the maintenance branch for some of their patches for exactly the same reasons they currently don't, and they will continue to refuse to do the work to commit in the maintenance branch first. Especially after the first time they run into one of the technical problems described above. In the end, any policy to encourage a "subset" policy is likely to end up as a burden on the same people who currently do the cross branch merging.
In proposing such a workflow, consider these requirements:
It seems that there is a consensus to separate the 2.x and 3.x repos, and that also makes sense to me.
(I think) I wasn't primarily talking about the representation of branches in hg - to that, I fully trust recommendations from hg users and experts. What will need debate and discussion in the PEP is the workflow, ie. the order in which changes should be applied to the branches.
If the checkin is done in the proper (the maint) branch, at least merging of that change is automatic whenever someone does a hg merge.
I probably don't fully understand, but that seems to imply a workflow were all changes made to one branch should also automatically be integrated into a different branch. I'm curious as to how such a mechanism can apply to Python. Regards, Martin
Martin v. Löwis wrote:
What will need debate and discussion in the PEP is the workflow, ie. the order in which changes should be applied to the branches.
Particularly since there isn't even "one true workflow" *now*. I see three main variants go by on Python-checkins: - svnmerge based - commit to 2.x - backport to 2.6 with svnmerge - forward port to 3.x with svnmerge - backport to 3.0 (now 3.1) with svnmerge - manual port based - as above, but without using svnmerge - Py3k focused - commit to 3.x - manual backport to 2.x - possible svnmerge block of the backported 2.x commit - possible svnmerge based or manual backport to 2.6/3.1 While it would obviously be *nice* if every committer maintained 4 checkouts and either blocked or committed each change on the appropriate branches, I think actually *requiring* a specific workflow has the potential to cost us commits. Even if one workflow is designated the 'preferred' approach, the others still need to be supported to handle various possibilities for the "first" commit for a given change: - forward port only - commit to 2.6 - forward port to 2.x - forward port to 3.1 - forward port to 3.x - backport only - commit to 3.x - backport to 3.1 - backport to 2.x - backport to 2.6 - mixed, starting with 2.x (aka current svnmerge workflow) - commit to 2.x - backport to 2.6 - forward port to 3.x - backport to 3.1 - mixed, starting with 3.1 - commit to 3.1 - forward port to 3.x - backport to 2.6 - forward port or backport to 2.x Note that there are actual multiple variations even of the above workflows, based on the *source* of the various forward ports and backports. Also, each "forward port" or "backport" can be replaced by blocking the merge rather than applying it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Dirkjan Ochtman writes:
On Fri, Jul 3, 2009 at 15:29, Stephen J. Turnbull<stephen@xemacs.org> wrote:
I'll have to try them again now that 1.3 is out, but I found Mercurial named branches fundamentally unsuited to release management.
Can you explain why, please? It's not clear from what you say below.
Well, the main problem I had was one that you say has improved: the various issues of confusion due to the presence of multiple active heads in a single repository. IME, Mercurial strongly encourages a non-branching style. Although I can't fully explain in concrete terms what makes me feel that way, it's certainly consistent with your own inclination to advise "subset branches". Part of it comes from the fact that you can't have a single revision on two branches. I would really like the node of a release branch to be on both the branch and the mainline so that the tag appears in the history of both, but that's not possible. Another issue (which you say has improved) is the handling of multiple heads. With unnamed heads, it's just too painful to leave them hanging around, so you merge them immediately (in fact, all XEmacs committers currently active use either "pull -u" or the fetch extension). With named branches, the additional heads do tend to hang around. I found that named branches tended to get inadvertantly pushed, and worse, they'd end up being the tip, which Mercurial treats specially. In one case a completely private branch inadvertantly got pushed into my "pristine" clone of the public repo, and from there into the public repo, where (since it had been the tip in my private workspace) it ended up as the tip in public. Embarrassing, to say the least. Fortuntely, the branch was more or less ready to be pushed anyway, but one of my colleagues ended up working on that branch without realizing that he wasn't on (his) mainline any more, and wondered why some previously done work suddenly disappeared. A good time was had by nobody involved. I don't know if that has been fixed in hg; the experience was painful enough that my workflow adjusted immediately.
Ditto named branches. The problem is that (unless the internal implementation has changed very recently) a Mercurial revision can be on exactly one named branch (or on the trunk).
That's still true.
Which defeats the purpose of having named branches, really. (I mean the version control purpose; obviously it still can save disk space.)
Why does it defeat the purpose? What, in your opinion, is the purpose?
I use named branches to collect a sequence of revisions as a named object, for viewing and manipulation, as differentiated from some other sequence, for *various* values of "some other sequence". The problem is that suppose you have a branch A off the trunk, and you then (several revisions down the line of development) branch B off A. Now A meanders off and runs into "not ready for prime time" problems, while B just swims along. Problem is, you can't easily find the history of B relative to the trunk because much of its history (since forking from the *trunk*) is labeled A.
Unless you're really short on space, though, that's not a big deal. What would be more important to me (not that I matter for the purpose of Python, but in XEmacs -- also a Mercurial shop -- I do :-) would be the other way around: pulling an external branch into a named branch. I have a feeling that working with such a repository with others would be a little difficult.
Can you give an example?
No, I haven't tried it. What concerns me is that I suspect that the branch name will end up as part of the revision's internal identifier, and that means that if you and I separately create the same named branch we have to choose exactly the same name or the branches won't be recognized as the same, resulting in the mother of all spurious merge conflicts.
As others (MvL, I think) have commented, this isn't really relevant to Python which generally has four mainlines going at once. I don't see why the requirements are going to change with the shift to hg, and I see no reason why hg won't handle the existing workflow just fine.
It will handle it, for sure, but I think it would all go easier if we could work with stricter subset branches (and leave the effective cherrypicking for the occasional problem).
Sure, but what do you propose? That we nuke Python 3.1, 3.2, and 2.6? They're all pretty divergent from 2.7 by now, as well as from each other. Not to mention the "security branches" that are still around. And individual developers are for sure going to do interesting things in their private workspaces. I see that George Brandl and Martin van Loewis seem to be accomodating your viewpoint, but I don't get the impression that either you (as the hg migration proponent) nor they (as core committers) realize how far apart your assumptions are. You are talking about the Mercurial project, which has *one* line of development. There are many such projects; the ones I'm most familiar with are XEmacs and Scheme48, which have adopted a subset branch approach different the Mercurial project's (XEmacs is modeled on Scheme48). Works well enough (when I'm wearing my release manager hat; it's a little constraining as a developer used to the insane flexibility of git). But Python is *not* such a project. The problem is not coordinating concurrent development of a closeknit group of committers all working on the same mainline. In Python, there are *four* mainlines with rather different purposes, and a diverse group of developers, some who work on only one line of development, others who work on several, and a few who accept the role of coordinating across them.
At 12:20 PM 7/4/2009 +0900, Stephen J. Turnbull wrote:
IME, Mercurial strongly encourages a non-branching style. Although I can't fully explain in concrete terms what makes me feel that way, it's certainly consistent with your own inclination to advise "subset branches". Part of it comes from the fact that you can't have a single revision on two branches. I would really like the node of a release branch to be on both the branch and the mainline so that the tag appears in the history of both, but that's not possible.
I'm a total newbie to Mercurial - I've basically done little more than the tutorial, in fact. So somebody correct me if what I'm about to say is completely stupid. Wouldn't the simple thing to do in Mercurial, just be to use different repositories for long-lived branches? I mean, if you're not merging them that much anyway, what's the point? Isn't it just as easy to pull from another repository, as from another branch within that repository?
P.J. Eby writes:
Wouldn't the simple thing to do in Mercurial, just be to use different repositories for long-lived branches? I mean, if you're not merging them that much anyway, what's the point?
I basically agree with that, and so does Dirkjan, I think. I'm not sure why he brought up the idea of using named branches.
Isn't it just as easy to pull from another repository, as from another branch within that repository?
Not if the other repository is remote and offline, and pulling from a remote does take more time, as do things like diffs against that branch. The temptation is to leave the remote repos on the public host, rather than put in the effort to make sure your local clones are up to date. If you do, there are minor annoyances like that. So people who actually work on more than one line of development learn pretty soon to keep a local clone for each repo they're interested in. Problem solved, IMO, but some people worry about the extra space taken up.
I see that George Brandl and Martin van Loewis seem to be accomodating your viewpoint, but I don't get the impression that either you (as the hg migration proponent) nor they (as core committers) realize how far apart your assumptions are.
Actually, I (probably) don't agree to a merge flow different from the one that we currently have. All I asked that *if* Dirkjan is in favor of a change, he should propose it explicitly, and fleshed-out, in the PEP, so that people can tear it down. I know that several committers are unhappy with the current merge flow (Georg in particular), so this is important to discuss. Regards, Martin
Dirkjan Ochtman schrieb:
- Fifth, here's a list of things, off the top of my head, that still need doing:
* Get agreement on branch strategy and branch processing (list of branches + proposed handling at http://hg.python.org/pymigr/file/tip/all-branches.txt) <--- PLEASE REVIEW
Do you have a key to the second column in that file? E.g. the difference between "strip" and "discard" is not clear to me. "strip partial"? Why are there branch names starting with "../"? [PEP 385]
==============================================================
Motivation
After having decided to switch to the Mercurial DVCS, the actual migration still has to be performed. In the case of an important piece of infrastructure like the version control system for a large, distributed project like Python, this is a significant effort. This PEP is an attempt to describe the steps that must be taken for further discussion. It's somewhat similar to PEP 347 [1], which discussed the migration to SVN.
To make the most of hg, I (Dirkjan) would like to make a high-fidelity conversion, such that (a) as much of the svn metadata as possible is retained, and (b) all metadata is converted to formats that are common in Mercurial. This way, tools written for Mercurial can be optimally used. In order to do this, I want to use the hgsubversion [2] software to do an initial conversion. This hg extension is focused on providing high-quality conversion from Subversion to Mercurial for use in two-way correspondence, meaning it doesn't throw away as much available metadata as other solutions.
Such a conversion also seems like a good time to reconsider the contents of the repository and determine if some things are still valuable. In this spirit, the following sections also propose discarding some of the older metadata. Timeline
TBD; needs fully working hgsubversion and consensus on this document. Transition plan Branch strategy
Mercurial has two basic ways of using branches: cloned branches, where each branch is kept in a separate repository, and named branches, where each revision keeps metadata to note on which branch it belongs. The former makes it easier to distinguish branches, at the expense of requiring more disk space on the client. The latter makes it a little easier to switch between branches, but often has somewhat unintuitive results for people (though this has been getting better in recent versions of Mercurial).
I'm still a bit on the fence about whether Python should adopt cloned branches and named branches. Since it usually makes more sense to tag releases on the maintenance branch, for example, mainline history would not contain release tags if we used cloned branches. Also, Mercurial 1.2 and 1.3 have the necessary tools to make named branches less painful (because they can be properly closed and closed heads are no longer considered in relevant cases).
A disadvantage might be that the used clones will be a good bit larger (since they essentially contain all other branches as well). This can me mitigated by keeping non-release (feature) branches in separate clones. Also note that it's still possible to clone a single named branch from a combined clone, by specifying the branch as in hg clone http://hg.python.org/main/#2.6-maint. Keeping the py3k history in a separate clone problably also makes sense.
* Does it work with "hg pull" etc. too, afterwards? * Does it support more than one branch?
XXX To do: size comparison for selected separation scenarios. Converting branches
There are quite a lot of branches in SVN's branches directory. I propose to clean this up a bit, by employing the following the strategy:
* Keep all release (maintenance) branches * Discard branches that haven't been touched in 18 months, unless somone indicates there's still interest in such a branch * Keep branches that have been touched in the last 18 months, unless someone indicates the branch can be deprecated
I would just kill all feature branches unless someone indicates it is still used. There are very few active feature branches. (I guess in the case a branch gets killed erroneously it could still be re-created after the conversion?)
Converting tags
The SVN tags directory contains a lot of old stuff. Some of these are not, in fact, full tags, but contain only a smaller subset of the repository. I think we should keep all release tags, and consider other tags for inclusion based on requests from the developer community. I'd like to consider unifying the release tag naming scheme to make some things more consistent, if people feel that won't create too many problems. For example, Mercurial itself just uses '1.2.1' as a tag, where CPython would currently use r121.
+1 for readable tag names. +1 for throwing out old questionable tag names.
Generating .hgignore
The .hgignore file can be used in Mercurial repositories to help ignore files that are not eligible for version control. It does this by employing several possible forms of pattern matching. The current Python repository already includes a rudimentary .hgignore file to help with using the hg mirrors.
It might be useful to have the .hgignore be generated automatically from svn:ignore properties. This would make sure all historic revisions also have useful ignore information (though one could argue ignoring isn't really relevant to just checking out an old revision).
I guess that's not necessary. People can just add stuff to .hgignore when they see something that should be there.
hg-ssh
Developers should access the repositories through ssh, similar to the current setup. Public keys can be used to grant people access to a shared hg@ account. A hgwebdir instance should also be set up for easy browsing and read-only access. If we're using ssh, developers should trivially be able to start new clones (for longer-term features that profit from a separate branch).
+1.
Hooks
A number of hooks is currently in use. The hg equivalents for these should be developed and deployed. The following hooks are being used:
* check whitespace: a hook to reject commits in case the whitespace doesn't match the rules for the Python codebase. Should be straightforward to re-implement from the current version. We can also offer a whitespace hook for use with client-side repositories that people can use; it could either warn about whitespace issues and/or truncate trailing whitespace from changed lines. Open issue: do we check only the tip after each push, or do we check every commit in a changegroup?
Only checking the tip would make it possible for people to revert their whitespace commits, but then -- if they have the local hook -- they shouldn't do that anyway.
* commit mails: we can leverage the notify extension for this
As long as it can send diffs...
* check contributors: in the current setup, all changesets bear the username of committers, who must have signed the contributor agreement. In a DVCS, the committers are not necessarily the same people who push, and so we can't check if the committer is a contributor. We could use a hook to check if the committer is a contributor if we keep a list of registered contributors.
That gets very ugly as soon as you start pulling from repos that just fix a small typo or so.
code.python.org was also proposed as the hostname. Personally, I think that using the VCS name in the hostname is good because it prevents confusion: it should be clear that you can't use svn or bzr for hg.python.org.
Yes, and it mirrors svn.python.org.
Mercurial itself has a stable branch which is a ''strict'' subset of the unstable branch. This means that generally all fixes for the stable branch get committed against the tip of the stable branch, then they get merged into the unstable branch (which already contains the parent of the new cset). This provides a largely frictionless environment for moving changes from stable to unstable branches. Mistakes, where a change that should go on stable goes on unstable first, do happen, but they're usually easy to fix. That can be done by copying the change over to the stable branch, then trivial-merging with unstable -- meaning the merge in fact ignores the parent from the stable branch).
This strategy means a little more work for regular committers, because they have to think about whether their change should go on stable or unstable; they may even have to ask someone else (the RM) before committing. But it also relieves a dedicated group of committers of regular backporting duty, in addition to making it easier to work with the tool.
Strong +1 for that.
I propose that the revision identifier will be the short version of hg's revision hash, for example 'dd3ebf81af43', augmented with '+' (instead of 'M') if the working directory from which it was built was modified. This mirrors the output of the hg id command, which is intended for this kind of usage.
For the tag/branch identifier, I propose that hg will check for tags on the currently checked out revision, use the tag if there is one ('tip' doesn't count), and uses the branch name otherwise. sys.subversion becomes
* ('CPython', '2.6.2', 'dd3ebf81af43') * ('CPython', 'default', 'af694c6a888c+')
and the build info string becomes
* '2.6.2:dd3ebf81af43, Jun 2 2009, 09:58:33' * 'default:af694c6a888c+, Jun 2 2009, 01:24:14'
This reflects that the default branch in hg is called 'default' instead of Subversion's 'trunk', and reflects the proposed new tag format.
Looks good to me. cheers, Georg
On Fri, Jul 3, 2009 at 17:17, Georg Brandl<g.brandl@gmx.net> wrote:
Do you have a key to the second column in that file? E.g. the difference between "strip" and "discard" is not clear to me. "strip partial"?
strip == discard. strip = remove, merged should be obvious, keep-clone means we'll keep the branch, in a separate clone, and keep-named means we'll keep the branch as a named branch in one of the other clones.
Why are there branch names starting with "../"?
Because some people in the history put their branch at /python/branchname instead of /python/branches/branchname. That part will get cleaned up. [Regarding clone#branch syntax:]
* Does it work with "hg pull" etc. too, afterwards?
Yes.
* Does it support more than one branch?
You can only reference one branch at a time using the fragment specifier.
I would just kill all feature branches unless someone indicates it is still used. There are very few active feature branches.
(I guess in the case a branch gets killed erroneously it could still be re-created after the conversion?)
Re-creating it after the conversion could be done, I guess, but I'd prefer to get it right, the first time. As the branch map says, I'm mostly removing all the feature branches anyway.
As long as it can send diffs...
It can.
That gets very ugly as soon as you start pulling from repos that just fix a small typo or so.
Okay, so we do no automatic checking. We can always write a script that walks the repository and finds all significant contributors. Cheers, Dirkjan
On Thu, Jul 2, 2009 at 13:42, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though.
- First of all, I've got the basic conversion down, I've done it a few times now, with progressively better results. You can view some results at http://hg.python.org/, which has a preliminary cpython repository. *** The changeset hashes for that repo will change, so you won't be able to commit or pull from it in the future.***
- Second of all, some planning. I've thought about it a bit, and I think we should aim for going live with hg on August 1. Given that I'm on vacation from 8-18 July (and I'm not sure whether I'll be able to actually work on it during that time, though I imagine I'll be able to spend some time on it at least), that's quite ambitious, so I'm going to say it's okay if it slips by a few days. Putting a deadline out there is a good thing, anyway.
Fine by me as long as people realize that if anything is questionable then the switch will not happen. Getting this right takes precedence over any deadline. And obviously we will need to do at least one live conversion on python.org hardware to make sure everything will work smoothly.
- Third of all, to make this possible, it would be helpful if I got more feedback on the PEP. Last time I raised it, there was virtually nothing. This time, I'll include it inline so there's hopefully less of a barrier to reviewing it.
- Fourth, Mercurial 1.3 was just released! Bet you didn't see that coming. It's looking like a pretty good release, with an experimental version of the much-coveted subrepository support (like svn:externals). This also means that the latest version of hgsubversion, the tool I used for the conversion, will be more accessible for converting other projects. You know you want to!
And will make the idea of splitting out the standard library and tests a reasonable thing to do.
- Fifth, here's a list of things, off the top of my head, that still need doing:
* Get agreement on branch strategy and branch processing (list of branches + proposed handling at http://hg.python.org/pymigr/file/tip/all-branches.txt) <--- PLEASE REVIEW * Get agreement on tag processing (first come up with a list) * Set up hg-ssh infra (should be easy) * Set up hooks (should be mostly straightforward) * Set up roundup integration (should be made easier by quick revision map hgweb extension) * Write docs
- Sixth (this is the good part), less obvious things that have been done or don't need doing:
* .hgignore generation (I've been convinced it's too hard, the current version will do)
Yeah, we can do this manually.
* revlog reordering (it's painless and a big win)
I'll get through all of these myself, but obviously any help would be welcome. For any hg users, writing docs should be an easy start. For others, please review the PEP (below), the branch map in http://hg.python.org/pymigr/file/tip/all-branches.txt and the author map at http://hg.python.org/pymigr/file/tip/author-map (not that much has changed since the start, so if you've looked at it already, feel free to skip this part). Right now I'm a little stuck on branch processing, because it's a long running script that needs a bunch of debugging, but I'll get going on that again.
I think that's all I can think of for now, I'll update the PEP with new bits soon. Here it is, ready for your review:
==============================================================
Motivation
After having decided to switch to the Mercurial DVCS, the actual migration still has to be performed. In the case of an important piece of infrastructure like the version control system for a large, distributed project like Python, this is a significant effort. This PEP is an attempt to describe the steps that must be taken for further discussion. It's somewhat similar to PEP 347 [1], which discussed the migration to SVN.
To make the most of hg, I (Dirkjan) would like to make a high-fidelity conversion, such that (a) as much of the svn metadata as possible is retained, and (b) all metadata is converted to formats that are common in Mercurial. This way, tools written for Mercurial can be optimally used. In order to do this, I want to use the hgsubversion [2] software to do an initial conversion. This hg extension is focused on providing high-quality conversion from Subversion to Mercurial for use in two-way correspondence, meaning it doesn't throw away as much available metadata as other solutions.
Such a conversion also seems like a good time to reconsider the contents of the repository and determine if some things are still valuable. In this spirit, the following sections also propose discarding some of the older metadata. Timeline
TBD; needs fully working hgsubversion and consensus on this document. Transition plan Branch strategy
Mercurial has two basic ways of using branches: cloned branches, where each branch is kept in a separate repository, and named branches, where each revision keeps metadata to note on which branch it belongs. The former makes it easier to distinguish branches, at the expense of requiring more disk space on the client. The latter makes it a little easier to switch between branches, but often has somewhat unintuitive results for people (though this has been getting better in recent versions of Mercurial).
I'm still a bit on the fence about whether Python should adopt cloned branches and named branches. Since it usually makes more sense to tag releases on the maintenance branch, for example, mainline history would not contain release tags if we used cloned branches. Also, Mercurial 1.2 and 1.3 have the necessary tools to make named branches less painful (because they can be properly closed and closed heads are no longer considered in relevant cases).
A disadvantage might be that the used clones will be a good bit larger (since they essentially contain all other branches as well). This can me mitigated by keeping non-release (feature) branches in separate clones. Also note that it's still possible to clone a single named branch from a combined clone, by specifying the branch as in hg clone http://hg.python.org/main/#2.6-maint. Keeping the py3k history in a separate clone problably also makes sense.
While I really like the idea of using named branches for each release so that there is a single py3k branch that contains all relevant history for every release, I think we should start simple and go with cloned branches. That way the workflow does not radically shift from what we do now for svn to start. Once the conversion is done and people are comfortable with hg we can then discuss moving towards a named branch approach.
XXX To do: size comparison for selected separation scenarios. Converting branches
There are quite a lot of branches in SVN's branches directory. I propose to clean this up a bit, by employing the following the strategy:
* Keep all release (maintenance) branches * Discard branches that haven't been touched in 18 months, unless somone indicates there's still interest in such a branch * Keep branches that have been touched in the last 18 months, unless someone indicates the branch can be deprecated
Sounds reasonable to me. We can just make a list and send it to python-committers to make final decisions of what should stick around.
Converting tags
The SVN tags directory contains a lot of old stuff. Some of these are not, in fact, full tags, but contain only a smaller subset of the repository. I think we should keep all release tags, and consider other tags for inclusion based on requests from the developer community. I'd like to consider unifying the release tag naming scheme to make some things more consistent, if people feel that won't create too many problems. For example, Mercurial itself just uses '1.2.1' as a tag, where CPython would currently use r121.
I don't use tags so I don't really care, but in the name of easy transition I say we don't change the naming scheme (although I have no issue dropping obscure tags).
Author map
In order to provide user names the way they are common in hg (in the 'First Last <user@example.org>' format), we need an author map to map cvs and svn user names to real names and their email addresses. I have a complete version of such a map in my migration tools repository [3]. The email addresses in it might be out of date; that's bound to happen, although it would be nice to try and have as many people as possible review it for addresses that are out of date. The current version also still seems to contain some encoding problems.
Something else that can go out to python-committers before the switch.
Generating .hgignore
The .hgignore file can be used in Mercurial repositories to help ignore files that are not eligible for version control. It does this by employing several possible forms of pattern matching. The current Python repository already includes a rudimentary .hgignore file to help with using the hg mirrors.
It might be useful to have the .hgignore be generated automatically from svn:ignore properties. This would make sure all historic revisions also have useful ignore information (though one could argue ignoring isn't really relevant to just checking out an old revision).
Don't bother with anything automatic. We can change the .hgignore file by hand. We all know glob and regex syntax. =)
Revlog reordering
As an optional optimization technique, we should consider trying a reordering pass on the revlogs (internal Mercurial files) resulting from the conversion. In some cases this results in dramatic decreases in on-disk repository size.
Fine by me.
Other repositories
Richard Tew has indicated that he'd like the Stackless repository to also be converted. What other projects in the svn.python.org repository should be converted? Do we want to convert the peps repository? distutils? others?
I don't think there is a single project we host -- all two of them -- that have not said they want to convert. So I say convert everything and let's turn off the svn server by the end of the year.
Infrastructure hg-ssh
Developers should access the repositories through ssh, similar to the current setup. Public keys can be used to grant people access to a shared hg@ account. A hgwebdir instance should also be set up for easy browsing and read-only access. If we're using ssh, developers should trivially be able to start new clones (for longer-term features that profit from a separate branch). Hooks
A number of hooks is currently in use. The hg equivalents for these should be developed and deployed. The following hooks are being used:
* check whitespace: a hook to reject commits in case the whitespace doesn't match the rules for the Python codebase. Should be straightforward to re-implement from the current version. We can also offer a whitespace hook for use with client-side repositories that people can use; it could either warn about whitespace issues and/or truncate trailing whitespace from changed lines. Open issue: do we check only the tip after each push, or do we check every commit in a changegroup? * commit mails: we can leverage the notify extension for this * buildbots: both the regular and the community build masters must be notified. Fortunately buildbot includes support for hg. I've also implemented this for Mercurial itself, so I don't expect problems here. * check contributors: in the current setup, all changesets bear the username of committers, who must have signed the contributor agreement. In a DVCS, the committers are not necessarily the same people who push, and so we can't check if the committer is a contributor. We could use a hook to check if the committer is a contributor if we keep a list of registered contributors.
Can we check these scripts into the repository itself? That way there is a chance of reuse as hg commands, e.g. ``hg pydev-ci`` as a replacement for ``make patchcheck``.
hgwebdir
A more or less stock hgwebdir installation should be set up. We might want to come up with a style to match the Python website. It may also be useful to build a quick extension to augment the URL rev parser so that it can also take r[0-9]+ args and come up with the matching hg revision. After migration Where to get code
It needs to be decided where the hg repositories will live. I'd like to propose to keep the hgwebdir instance at hg.python.org. This is an accepted standard for many organizations, and an easy parallel to svn.python.org. The 2.7 (trunk) repo might live at http://hg.python.org/main/, for example, with py3k at http://hg.python.org/py3k/. For write access, developers will have to use ssh, which could be ssh://hg@hg.python.org/main/. A demo installation will be set up with a preliminary conversion so people can experiment and review; it can live at http://hg.python.org/example/.
code.python.org was also proposed as the hostname. Personally, I think that using the VCS name in the hostname is good because it prevents confusion: it should be clear that you can't use svn or bzr for hg.python.org.
How about hg.python.org for the official branches and we keep code.python.org for personal branches of the developers like we have done with the bzr experiments?
hgwebdir can already provide tarballs for every changeset. I think this obviates the need for daily snapshots; we can just point users to tip.tar.gz instead, meaning they will get the latest. If desired, we could even use buildbot results to point to the last good changeset.
I like the stable buildbot tarball idea.
Python-specific documentation
hg comes with good built-in documentation (available through hg help) and a wiki [4] that's full of useful information and recipes. In addition to that, the parts of the developer FAQ [5] concerning version control will gain a section on using hg for Python development. Some of the text will be dependent on the outcome of debate about this PEP (for example, the branching strategy).
Think first, commit later?
In recent history, old versions of Python have been maintained by a select group of people backporting patches from trunk to release branches. While this may not scale so well as the development pace grows, it also runs into some problems with the current crop of distributed versioning tools. These tools (I believe similar problems would exist for either git, bzr, or hg, though some may cope better than others) are based on the idea of a Directed Acyclic Graph (or DAG), meaning they keep track of relations of changesets.
Mercurial itself has a stable branch which is a ''strict'' subset of the unstable branch. This means that generally all fixes for the stable branch get committed against the tip of the stable branch, then they get merged into the unstable branch (which already contains the parent of the new cset). This provides a largely frictionless environment for moving changes from stable to unstable branches. Mistakes, where a change that should go on stable goes on unstable first, do happen, but they're usually easy to fix. That can be done by copying the change over to the stable branch, then trivial-merging with unstable -- meaning the merge in fact ignores the parent from the stable branch).
This strategy means a little more work for regular committers, because they have to think about whether their change should go on stable or unstable; they may even have to ask someone else (the RM) before committing. But it also relieves a dedicated group of committers of regular backporting duty, in addition to making it easier to work with the tool.
Now would be a good time to consider changing strategies in this regard, although it would be relatively easy to switch to such a model later on.
As I have said, we should change our workflow habits after the switch and people are comfortable with hg.
The future of Subversion
What happens to the Subversion repositories after the migration? Since the svn server contains a bunch of repositories, not just the CPython one, it will probably live on for a bit as not every project may want to migrate or it takes longer for other projects to migrate. To prevent people from staying behind, we may want to remove migrated projects from the repository. Build identification
Python currently provides the sys.subversion tuple to allow Python code to find out exactly what version of Python it's running against. The current version looks something like this:
* ('CPython', 'tags/r262', '71600') * ('CPython', 'trunk', '73128M')
Another value is returned from Py_GetBuildInfo() in the C API, and available to Python code as part of sys.version:
* 'r262:71600, Jun 2 2009, 09:58:33' * 'trunk:73128M, Jun 2 2009, 01:24:14'
I propose that the revision identifier will be the short version of hg's revision hash, for example 'dd3ebf81af43', augmented with '+' (instead of 'M') if the working directory from which it was built was modified. This mirrors the output of the hg id command, which is intended for this kind of usage.
For the tag/branch identifier, I propose that hg will check for tags on the currently checked out revision, use the tag if there is one ('tip' doesn't count), and uses the branch name otherwise. sys.subversion becomes
* ('CPython', '2.6.2', 'dd3ebf81af43') * ('CPython', 'default', 'af694c6a888c+')
and the build info string becomes
* '2.6.2:dd3ebf81af43, Jun 2 2009, 09:58:33' * 'default:af694c6a888c+, Jun 2 2009, 01:24:14'
This reflects that the default branch in hg is called 'default' instead of Subversion's 'trunk', and reflects the proposed new tag format.
Should we consider adding a sys.revision attribute and begin the deprecation of sys.subversion?
References [1] http://www.python.org/dev/peps/pep-0347/ [2] http://bitbucket.org/durin42/hgsubversion/ [3] http://hg.xavamedia.nl/cpython/pymigr/ [4] http://www.selenic.com/mercurial/wiki/ [5] http://www.python.org/dev/faq/#version-control
=====================================================
Cheers,
Dirkjan _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
On Fri, Jul 03, 2009, Brett Cannon wrote:
Should we consider adding a sys.revision attribute and begin the deprecation of sys.subversion?
+1 -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha
Should we consider adding a sys.revision attribute and begin the deprecation of sys.subversion?
I wouldn't mind killing sys.subversion "right away" (i.e. in trunk and 3k - obviously it has to stay in 2.6 and 3.1, and all the older branches). I'm -1 on calling it "sys.revision", as this makes it difficult to tell what the actual versioning system was, and hence how the data should be interpreted. It will already be a problem for 2.6, when 2.6.3 will currently have a sys.subversion[2] of 'dd3ebf81af43', which will surely crash existing applications. I'm not sure what the motivation for a sys.revision is; it's probably similar to the desire of calling the machine code.python.org (instead of hg.python.org). It gives the illusion of being agnostic of the actual RCS being used. However, this is a complete illusion: anybody using it (either code.python.org, or sys.revision), *cannot* be agnostic of the specific technology. Regards, Martin
On Fri, Jul 3, 2009 at 14:00, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Should we consider adding a sys.revision attribute and begin the deprecation of sys.subversion?
I wouldn't mind killing sys.subversion "right away" (i.e. in trunk and 3k - obviously it has to stay in 2.6 and 3.1, and all the older branches).
I'm -1 on calling it "sys.revision", as this makes it difficult to tell what the actual versioning system was, and hence how the data should be interpreted. It will already be a problem for 2.6, when 2.6.3 will currently have a sys.subversion[2] of 'dd3ebf81af43', which will surely crash existing applications.
I'm not sure what the motivation for a sys.revision is; it's probably similar to the desire of calling the machine code.python.org (instead of hg.python.org). It gives the illusion of being agnostic of the actual RCS being used. However, this is a complete illusion: anybody using it (either code.python.org, or sys.revision), *cannot* be agnostic of the specific technology.
We could add another value in the tuple that specifies the VCS: ('CPython', 'branches/release25-maint', '61464', 'svn'). I agree that VCSs are not universally the same, but the concept of a revision is universal. -Brett
We could add another value in the tuple that specifies the VCS: ('CPython', 'branches/release25-maint', '61464', 'svn'). I agree that VCSs are not universally the same, but the concept of a revision is universal.
Actually, I think that's not the case. For bzr, the usual way of identifying a revision is by revision number, which, however, is not unique within a project, as each branch will use contiguous integers for numbers. There are also unique identifications - so a bzr revision has actually two numbers. More general, in a DVCS, it is not possible to access the revision being referred to by such a tuple. For sys.subversion, if [0]=='CPython', then you could go to svn.python.org. For a DVCS, the revision being identified may not be publically available, or may not live on a host that you can infer from your proposed sys.revision. For cloned branches, I wonder how sys.revision[1] would be computed. Regards, Martin
On Fri, Jul 3, 2009 at 14:52, "Martin v. Löwis" <martin@v.loewis.de> wrote:
We could add another value in the tuple that specifies the VCS: ('CPython', 'branches/release25-maint', '61464', 'svn'). I agree that VCSs are not universally the same, but the concept of a revision is universal.
Actually, I think that's not the case. For bzr, the usual way of identifying a revision is by revision number, which, however, is not unique within a project, as each branch will use contiguous integers for numbers. There are also unique identifications - so a bzr revision has actually two numbers.
More general, in a DVCS, it is not possible to access the revision being referred to by such a tuple. For sys.subversion, if [0]=='CPython', then you could go to svn.python.org. For a DVCS, the revision being identified may not be publically available, or may not live on a host that you can infer from your proposed sys.revision.
For cloned branches, I wonder how sys.revision[1] would be computed.
So are you saying we should drop the idea of a revision value altogether, or just embrace the differences and add a sys.mercurial attribute? -Brett
So are you saying we should drop the idea of a revision value altogether, or just embrace the differences and add a sys.mercurial attribute?
That's what I would propose. It should be a best-effort(*) approach at providing all information that is needed to really find the source used for the specific version. Regards, Martin (*) even for svn it was best-effort only in case there were local modifications.
Martin v. Löwis schrieb:
We could add another value in the tuple that specifies the VCS: ('CPython', 'branches/release25-maint', '61464', 'svn'). I agree that VCSs are not universally the same, but the concept of a revision is universal.
Actually, I think that's not the case. For bzr, the usual way of identifying a revision is by revision number, which, however, is not unique within a project, as each branch will use contiguous integers for numbers. There are also unique identifications - so a bzr revision has actually two numbers.
More general, in a DVCS, it is not possible to access the revision being referred to by such a tuple. For sys.subversion, if [0]=='CPython', then you could go to svn.python.org. For a DVCS, the revision being identified may not be publically available, or may not live on a host that you can infer from your proposed sys.revision.
At least you can tell that if the given hash is not present in the mainline repo, the build contains something that doesn't come from python.org. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On Jul 3, 2009, at 5:00 PM, Martin v. Löwis wrote:
I'm -1 on calling it "sys.revision", as this makes it difficult to tell what the actual versioning system was, and hence how the data should be interpreted. It will already be a problem for 2.6, when 2.6.3 will currently have a sys.subversion[2] of 'dd3ebf81af43', which will surely crash existing applications.
I can release a 2.6.3 right before the cut-over (well, just about any time between now and August 1st). Should we just plan now for a 2.6.3 on say July 24th, with a release candidate on July 20th?
I'm not sure what the motivation for a sys.revision is; it's probably similar to the desire of calling the machine code.python.org (instead of hg.python.org). It gives the illusion of being agnostic of the actual RCS being used. However, this is a complete illusion: anybody using it (either code.python.org, or sys.revision), *cannot* be agnostic of the specific technology.
Agreed. I originally chose code.python.org because I didn't want to be biased (maybe I should have been :). +1 for hg.python.org. I'd prefer to spell out sys.mercurial_revision. -Barry
Barry Warsaw wrote:
On Jul 3, 2009, at 5:00 PM, Martin v. Löwis wrote:
I'm -1 on calling it "sys.revision", as this makes it difficult to tell what the actual versioning system was, and hence how the data should be interpreted. It will already be a problem for 2.6, when 2.6.3 will currently have a sys.subversion[2] of 'dd3ebf81af43', which will surely crash existing applications.
I can release a 2.6.3 right before the cut-over (well, just about any time between now and August 1st). Should we just plan now for a 2.6.3 on say July 24th, with a release candidate on July 20th?
I'm fine with that plan - but the original problem remains. We will surely release 2.6.4 at some point, and it will have a different version identification (based on hg rev ids). So those existing applications (which are probably few) will then crash for 2.6.4, unless we continue maintaining 2.6 in subversion, or just arrange to fake sys.subversion somehow (e.g. freezing it on the last subversion revision - which might still break applications that insist on accessing the revision mentioned - not sure whether such applications actually exist). Regards, Martin
On Jul 3, 2009, at 6:15 PM, Martin v. Löwis wrote:
I'm fine with that plan - but the original problem remains. We will surely release 2.6.4 at some point, and it will have a different version identification (based on hg rev ids).
So those existing applications (which are probably few) will then crash for 2.6.4, unless we continue maintaining 2.6 in subversion, or just arrange to fake sys.subversion somehow (e.g. freezing it on the last subversion revision - which might still break applications that insist on accessing the revision mentioned - not sure whether such applications actually exist).
Doesn't Mercurial support an Subversion bridge? Would it be possible to provide a /read-only/ copy of the hg branches for 2.4 (maybe), 2.5, 2.6, and 3.1? If so, then the release managers would simply have to cut their releases from the svn copy instead of the hg master. /All/ other work would be done from the hg master. This shouldn't be too much of a burden since it's done so rarely and would end with the EOL of each of those branches. It would mean maintaining the bridge until all currently released versions are EOL. If that's not possible or feasible, then given the documented sys.subversion semantics, I think we should just freeze the tuple at e.g. ('CPython', 'branches/release26-maint', None). -Barry
Doesn't Mercurial support an Subversion bridge? Would it be possible to provide a /read-only/ copy of the hg branches for 2.4 (maybe), 2.5, 2.6, and 3.1? If so, then the release managers would simply have to cut their releases from the svn copy instead of the hg master. /All/ other work would be done from the hg master. This shouldn't be too much of a burden since it's done so rarely and would end with the EOL of each of those branches.
It would mean maintaining the bridge until all currently released versions are EOL.
That sounds very complicated (but Dirkjan should correct me if I'm wrong). I have already mentally prepared to port welease to hg, to create the missing 2.4 and 2.5 releases.
If that's not possible or feasible, then given the documented sys.subversion semantics, I think we should just freeze the tuple at e.g. ('CPython', 'branches/release26-maint', None).
That would work for me (I had to re-read the documentation to see that None is a valid, documented value for version, and allowed if "the tree was exported"). Interesting to notice that sys.subversion didn't even last the life time of one feature release :-) Regards, Martin
On Sat, Jul 4, 2009 at 00:37, Barry Warsaw<barry@python.org> wrote:
Doesn't Mercurial support an Subversion bridge? Would it be possible to provide a /read-only/ copy of the hg branches for 2.4 (maybe), 2.5, 2.6, and 3.1? If so, then the release managers would simply have to cut their releases from the svn copy instead of the hg master. /All/ other work would be done from the hg master. This shouldn't be too much of a burden since it's done so rarely and would end with the EOL of each of those branches.
There is some push support in hgsubversion, but it doesn't do tags at this time, for example. I think this setup would be needlessly complicated (and tools will need to learn the new revision specifier anyway, so why not have it learn them sooner rather than later)? Cheers, Dirkjan
On Fri, Jul 3, 2009 at 20:04, Brett Cannon<brett@python.org> wrote:
Fine by me as long as people realize that if anything is questionable then the switch will not happen. Getting this right takes precedence over any deadline. And obviously we will need to do at least one live conversion on python.org hardware to make sure everything will work smoothly.
I'm not sure I see the need to do a (live? what does that mean in this context) on python.org hardware. Why exactly is that better than me doing it on one of my boxes, as long as all the necessary tools and an idea of how to do it are publically available (from the pymigr repo, for example)?
And will make the idea of splitting out the standard library and tests a reasonable thing to do.
In due time, yes.
While I really like the idea of using named branches for each release so that there is a single py3k branch that contains all relevant history for every release, I think we should start simple and go with cloned branches. That way the workflow does not radically shift from what we do now for svn to start. Once the conversion is done and people are comfortable with hg we can then discuss moving towards a named branch approach.
I don't think the cloned branches is much simpler than the named branches approach, in several ways. For example, populating the branch part of a sys.whatever value is significantly harder. Also, if you follow a useful tagging approach, doing cloned branches means that release tags aren't available on trunk/main/default. That seems like a step backwards.
Sounds reasonable to me. We can just make a list and send it to python-committers to make final decisions of what should stick around.
This list exists and has been referenced in my email a few times.
I don't use tags so I don't really care, but in the name of easy transition I say we don't change the naming scheme (although I have no issue dropping obscure tags).
The current proposal is to clean up old tags to agree with the current naming scheme (and dropping obscure and partial tags).
Something else that can go out to python-committers before the switch.
This should just be done ASAP, it helps with a smooth conversion process.
I don't think there is a single project we host -- all two of them -- that have not said they want to convert. So I say convert everything and let's turn off the svn server by the end of the year.
I say we tackle each one as we go. I say doing a good conversion job is valuable, and we should take as much time as we need (though not more). You advocate something similar below for the Python conversion.
Can we check these scripts into the repository itself? That way there is a chance of reuse as hg commands, e.g. ``hg pydev-ci`` as a replacement for ``make patchcheck``.
I'm not sure there's an easy way to make them into commands (although I guess you could make an extension to that effect), but hooks would be very easy.
How about hg.python.org for the official branches and we keep code.python.org for personal branches of the developers like we have done with the bzr experiments?
I think that's just confusing. Most people seem to like hg.python.org, and it's obvious that hg goes to hg.p.o. Dividing up the namespace only makes it harder to find things.
As I have said, we should change our workflow habits after the switch and people are comfortable with hg.
Well, not everyone agrees, and although I think it doesn't matter much for the conversion itself, it may affect the branching strategy discussion. (sys.revision discussion elided because some others have already been bikeshedding on it.) Cheers, Dirkjan
On Sat, Jul 4, 2009 at 01:03, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
Fine by me as long as people realize that if anything is questionable
On Fri, Jul 3, 2009 at 20:04, Brett Cannon<brett@python.org> wrote: then
the switch will not happen. Getting this right takes precedence over any deadline. And obviously we will need to do at least one live conversion on python.org hardware to make sure everything will work smoothly.
I'm not sure I see the need to do a (live? what does that mean in this context) on python.org hardware.
"Live" as in run through all the steps on python.org hardware w/o hitting any final switch that turns off svn.
Why exactly is that better than me doing it on one of my boxes, as long as all the necessary tools and an idea of how to do it are publically available (from the pymigr repo, for example)?
Because there are different OSs, installed software, etc. Basically because you just never know. =) Plus it will make Martin sleep better.
And will make the idea of splitting out the standard library and tests a reasonable thing to do.
In due time, yes.
While I really like the idea of using named branches for each release so that there is a single py3k branch that contains all relevant history for every release, I think we should start simple and go with cloned branches. That way the workflow does not radically shift from what we do now for svn to start. Once the conversion is done and people are comfortable with hg we can then discuss moving towards a named branch approach.
I don't think the cloned branches is much simpler than the named branches approach, in several ways. For example, populating the branch part of a sys.whatever value is significantly harder. Also, if you follow a useful tagging approach, doing cloned branches means that release tags aren't available on trunk/main/default. That seems like a step backwards.
I personally prefer named branches, but that's just me and I am not about to force my preferences on everyone. Guess we just have to see if others have opinions against named branches.
Sounds reasonable to me. We can just make a list and send it to python-committers to make final decisions of what should stick around.
This list exists and has been referenced in my email a few times.
Sure, but as inlining the PEP in this email thread has shown, not making people click a link helps. =) Plus a separate email makes it very obvious that people need to check their email instead of making it a bullet point in a much larger email.
I don't use tags so I don't really care, but in the name of easy transition I say we don't change the naming scheme (although I have no issue dropping obscure tags).
The current proposal is to clean up old tags to agree with the current naming scheme (and dropping obscure and partial tags).
Fine by me.
Something else that can go out to python-committers before the switch.
This should just be done ASAP, it helps with a smooth conversion process.
I don't think there is a single project we host -- all two of them -- that have not said they want to convert. So I say convert everything and let's turn off the svn server by the end of the year.
I say we tackle each one as we go. I say doing a good conversion job is valuable, and we should take as much time as we need (though not more). You advocate something similar below for the Python conversion.
I am not suggesting we do all conversions on the same day, just that everything should eventually be converted.
Can we check these scripts into the repository itself? That way there is a chance of reuse as hg commands, e.g. ``hg pydev-ci`` as a replacement for ``make patchcheck``.
I'm not sure there's an easy way to make them into commands (although I guess you could make an extension to that effect), but hooks would be very easy.
OK. I was just hoping we could factor the code in such a way as to share the basic steps the hooks were checking so as to reuse them in a command.
How about hg.python.org for the official branches and we keep code.python.org for personal branches of the developers like we have done with the bzr experiments?
I think that's just confusing. Most people seem to like hg.python.org, and it's obvious that hg goes to hg.p.o. Dividing up the namespace only makes it harder to find things.
Yeah, I realize I have lost this battle.
As I have said, we should change our workflow habits after the switch and people are comfortable with hg.
Well, not everyone agrees, and although I think it doesn't matter much for the conversion itself, it may affect the branching strategy discussion.
Sure, to an extent.
(sys.revision discussion elided because some others have already been bikeshedding on it.)
I think it has been answered; sys.subversion goes away and sys.mercurial or sys.mercurial_revision comes into existence. -Brett
On Sat, 4 Jul 2009 at 12:28, Brett Cannon wrote:
On Sat, Jul 4, 2009 at 01:03, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
Fine by me as long as people realize that if anything is questionable
On Fri, Jul 3, 2009 at 20:04, Brett Cannon<brett@python.org> wrote: then
the switch will not happen. Getting this right takes precedence over any deadline. And obviously we will need to do at least one live conversion on python.org hardware to make sure everything will work smoothly.
I'm not sure I see the need to do a (live? what does that mean in this context) on python.org hardware.
"Live" as in run through all the steps on python.org hardware w/o hitting any final switch that turns off svn.
"Live" should also (and I think this is critical) include a period of at least a week where we tell python-dev/python-committers "hg.python.org looks like it is going to look at cutover, please try out the workflow". The idea would be to have committers actually exercise the workflow on their platform for at least one patch, including whatever replaces the svnmerge step. I can almost guarantee we will find some issues that need to be resolved, and that people won't try it out thoroughly until it is "live". --David
2009/7/4 Brett Cannon <brett@python.org>:
While I really like the idea of using named branches for each release so that there is a single py3k branch that contains all relevant history for every release, I think we should start simple and go with cloned branches. That way the workflow does not radically shift from what we do now for svn to start. Once the conversion is done and people are comfortable with hg we can then discuss moving towards a named branch approach.
I don't think the cloned branches is much simpler than the named branches approach, in several ways. For example, populating the branch part of a sys.whatever value is significantly harder. Also, if you follow a useful tagging approach, doing cloned branches means that release tags aren't available on trunk/main/default. That seems like a step backwards.
I personally prefer named branches, but that's just me and I am not about to force my preferences on everyone. Guess we just have to see if others have opinions against named branches.
Personally, I prefer clones, as it seems to me that Mercurial named branches are not quite what people generally think of when they think of "branches", in some subtle respects. I could be wrong, as I don't personally use named branches. However, there's been a small thread on distutils-sig recently with a new Mercurial user complaining that he's got confused and messed up a repository. Without digging particularly deeply, it appears that the problems were caused by confusion over named branches. FWIW, the Mercurial book (http://hgbook.red-bean.com/read/managing-releases-and-branchy-development.ht...) describes named branches with the comment "If you're more in the “power user” category (*and* your collaborators are too)" (author's emphasis). I'm not sure we want to require contributors to be "power users" of Mercurial... Paul. PS Sorry for responding to an old thread - I couldn't locate more recent discussion, although I thought there had been some.
Dirkjan Ochtman wrote:
It needs to be decided where the hg repositories will live. I'd like to propose to keep the hgwebdir instance at hg.python.org. This is an accepted standard for many organizations, and an easy parallel to svn.python.org. The 2.7 (trunk) repo might live at http://hg.python.org/main/, for example, with py3k at http://hg.python.org/py3k/.
I would very much like the 'k' dropped from the py3 name. It was a funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers. I think the two repos should be either symmetrically named hg.python.org/py2 hg.python.org/py3 If one must be designated 'main', it should be py3. Continuing to call py2 'main' will continue to discourage use of py3. Terry Jan Reedy
Terry Reedy wrote:
Dirkjan Ochtman wrote:
It needs to be decided where the hg repositories will live. I'd like to propose to keep the hgwebdir instance at hg.python.org. This is an accepted standard for many organizations, and an easy parallel to svn.python.org. The 2.7 (trunk) repo might live at http://hg.python.org/main/, for example, with py3k at http://hg.python.org/py3k/.
I would very much like the 'k' dropped from the py3 name. It was a funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers.
I think the two repos should be either symmetrically named
hg.python.org/py2 hg.python.org/py3
If one must be designated 'main', it should be py3.
Continuing to call py2 'main' will continue to discourage use of py3.
We could regard py3k as the phase from the original concept of Python 3 to its 'prototype', Python 3.0. Python 3.1 would be the first 'real/usable' version.
MRAB wrote:
Terry Reedy wrote:
Dirkjan Ochtman wrote:
It needs to be decided where the hg repositories will live. I'd like to propose to keep the hgwebdir instance at hg.python.org. This is an accepted standard for many organizations, and an easy parallel to svn.python.org. The 2.7 (trunk) repo might live at http://hg.python.org/main/, for example, with py3k at http://hg.python.org/py3k/.
I would very much like the 'k' dropped from the py3 name. It was a funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers.
I think the two repos should be either symmetrically named
hg.python.org/py2 hg.python.org/py3
If one must be designated 'main', it should be py3.
Continuing to call py2 'main' will continue to discourage use of py3.
We could regard py3k as the phase from the original concept of Python 3 to its 'prototype', Python 3.0.
Right. And that phase is over, especially with Barry posting today on python-list that there will be no more 3.0.x releases ever.
Python 3.1 would be the first 'real/usable' version.
Right. 'is'. as Barry also posted. tjr
On Fri, Jul 3, 2009 at 11:22, Terry Reedy <tjreedy@udel.edu> wrote:
Dirkjan Ochtman wrote:
It needs to be decided where the hg repositories will live. I'd like
to propose to keep the hgwebdir instance at hg.python.org. This is an accepted standard for many organizations, and an easy parallel to svn.python.org. The 2.7 (trunk) repo might live at http://hg.python.org/main/, for example, with py3k at http://hg.python.org/py3k/.
I would very much like the 'k' dropped from the py3 name. It was a funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers.
Is it really that confusing? I have never heard of anyone asking "what is py3k?" Plus I like keeping that bit of Python history around. I know I still use py3k as shorthand for Python 3.x. And we are not that serious of a bunch. =)
I think the two repos should be either symmetrically named
hg.python.org/py2 hg.python.org/py3
If we make it universal I say it should be '2.x' and '3.x'. The whole 'py' prefix is redundant.
If one must be designated 'main', it should be py3.
Continuing to call py2 'main' will continue to discourage use of py3.
Yeah, 2.x shouldn't be special anymore.
On Fri, Jul 3, 2009 at 23:41, Brett Cannon<brett@python.org> wrote:
If we make it universal I say it should be '2.x' and '3.x'. The whole 'py' prefix is redundant.
Right, I was aiming for /python/2.x and /python/3.x as well. Actually, I currently have /cpython to also make CPython less special among it's peers, but that idea was met with some resistance on #python-dev. Cheers, Dirkjan
On Fri, Jul 3, 2009 at 15:00, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
On Fri, Jul 3, 2009 at 23:41, Brett Cannon<brett@python.org> wrote:
If we make it universal I say it should be '2.x' and '3.x'. The whole 'py' prefix is redundant.
Right, I was aiming for /python/2.x and /python/3.x as well.
Actually, I currently have /cpython to also make CPython less special among it's peers, but that idea was met with some resistance on #python-dev.
Don't worry about doing that right now. When the stdlib gets separated out we can revisit this. But for now just leave it as python. -Brett
Brett Cannon wrote:
On Fri, Jul 3, 2009 at 15:00, Dirkjan Ochtman <dirkjan@ochtman.nl <mailto:dirkjan@ochtman.nl>> wrote: Actually, I currently have /cpython to also make CPython less special among it's peers, but that idea was met with some resistance on #python-dev.
Don't worry about doing that right now. When the stdlib gets separated out we can revisit this. But for now just leave it as python.
Yes, if we ever do that split we can leave the common code (stdlib, test suite) under /python and move the CPython specific stuff to /cpython. +1 on 2.x and 3.x for the main development branches (although that does lead me to wonder if we should just make the maintenance branches 2.6, 3.1, etc). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Brett Cannon wrote:
I would very much like the 'k' dropped from the py3 name. It was a funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers.
Is it really that confusing? I have never heard of anyone asking "what is py3k?"
Do you read python-list? It has been asked. Also, some people seem to think that py3k is different from python 3. tjr
On Sat, Jul 4, 2009 at 01:55, Terry Reedy <tjreedy@udel.edu> wrote:
Brett Cannon wrote:
I would very much like the 'k' dropped from the py3 name. It was a
funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers.
Is it really that confusing? I have never heard of anyone asking "what is py3k?"
Do you read python-list?
No as it would take up so much of my Python time I wouldn't be able to code anymore.
It has been asked. Also, some people seem to think that py3k is different from python 3.
Well, I still would not like to lose the py3k label. But if we do I still say 2.x/3.x instead of py2/py3. -Brett
tjr
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
On Sat, 4 Jul 2009 04:22:57 am Terry Reedy wrote:
I would very much like the 'k' dropped from the py3 name. It was a funny joke when py3 was vaporware, now it is excess baggage which only puzzles non-insiders and newcomers.
+1 Some day we'll be using Python3.7 and wondering what the "k" means. Alpha, beta, .... kappa version? -- Steven D'Aprano
- First of all, I've got the basic conversion down, I've done it a few times now, with progressively better results. You can view some results at http://hg.python.org/, which has a preliminary cpython repository. *** The changeset hashes for that repo will change, so you won't be able to commit or pull from it in the future.***
I'd like to see as much infrastructure going "life" well in advance of the switch over. Ideally, when we do switch, there should be no surprises (i.e. all problems should be known, and, ideally, be resolved).
* Set up hg-ssh infra (should be easy) * Set up hooks (should be mostly straightforward) * Set up roundup integration (should be made easier by quick revision map hgweb extension) * Write docs
These are the ones that I was thinking of (and mentioned before, so they got onto your list :-) I would drop the roundup integration from the things that need to be done pre-migration - there currently is no svn integration, so not having it for hg is not a step backwards.
* buildbots: both the regular and the community build masters must be notified. Fortunately buildbot includes support for hg. I've also implemented this for Mercurial itself, so I don't expect problems here.
This should also be setup before the switchover, with a few test buildslaves (say, a Unix one and a Windows one).
In a DVCS, the committers are not necessarily the same people who push, and so we can't check if the committer is a contributor. We could use a hook to check if the committer is a contributor if we keep a list of registered contributors.
In the first sentence, you say that it can't actually work - so I think we should drop the test.
I propose that the revision identifier will be the short version of hg's revision hash, for example 'dd3ebf81af43', augmented with '+' (instead of 'M') if the working directory from which it was built was modified. This mirrors the output of the hg id command, which is intended for this kind of usage.
I would like to see this well before the switch also. It could be a patch (unified diff) stored in the tracker, or it could be an actual branch that then needs to get merged with all active branches (IIUC). Regards, Martin
Martin v. Löwis wrote:
I would drop the roundup integration from the things that need to be done pre-migration - there currently is no svn integration, so not having it for hg is not a step backwards.
That's not quite true - the tracker turns text like "r64537" into a link to the appropriate changeset on svn.python.org. It would be unfortunate to lose that easy ability to indicate when a bug was fixed (or introduced). This is more something on the roundup side though: recognising the presence of a hg id in a comment and turning it into a link to hg.python.org. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan wrote:
Martin v. Löwis wrote:
I would drop the roundup integration from the things that need to be done pre-migration - there currently is no svn integration, so not having it for hg is not a step backwards.
That's not quite true - the tracker turns text like "r64537" into a link to the appropriate changeset on svn.python.org. It would be unfortunate to lose that easy ability to indicate when a bug was fixed (or introduced).
This is more something on the roundup side though: recognising the presence of a hg id in a comment and turning it into a link to hg.python.org.
Ah, right. That must be done, of course (although I suppose there is little hope to have the existing references continue to work). Regards, Martin
Martin v. Löwis wrote:
Ah, right. That must be done, of course (although I suppose there is little hope to have the existing references continue to work).
I'd guess that the only way to keep those functional is to keep svn.python.org around in read-only mode. Although I'm not entirely sure about that - would it be possible to build up a mapping from svn revision numbers to hg repository IDs as the history information is being migrated? If that wasn't too difficult, it might be worth doing. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, Jul 4, 2009 at 07:13, Nick Coghlan<ncoghlan@gmail.com> wrote:
I'd guess that the only way to keep those functional is to keep svn.python.org around in read-only mode.
No, actually: the idea (I think I floated it in the PEP, as well), is that I can write a simple extension for hgweb that knows the mapping of SVN rev to hg rev and so can make hg.python.org/python/2.x/rev/r32542 come out to the changeset that resulted from converting that revision. Cheers, Dirkjan
Dirkjan Ochtman wrote:
On Sat, Jul 4, 2009 at 07:13, Nick Coghlan<ncoghlan@gmail.com> wrote:
I'd guess that the only way to keep those functional is to keep svn.python.org around in read-only mode.
No, actually: the idea (I think I floated it in the PEP, as well), is that I can write a simple extension for hgweb that knows the mapping of SVN rev to hg rev and so can make hg.python.org/python/2.x/rev/r32542 come out to the changeset that resulted from converting that revision.
Ah, that's an excellent solution - good to hear it :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sat, Jul 4, 2009 at 00:09, "Martin v. Löwis"<martin@v.loewis.de> wrote:
I would drop the roundup integration from the things that need to be done pre-migration - there currently is no svn integration, so not having it for hg is not a step backwards.
Yeah, I mean just the linking here.
In the first sentence, you say that it can't actually work - so I think we should drop the test.
Okay.
I would like to see this well before the switch also. It could be a patch (unified diff) stored in the tracker, or it could be an actual branch that then needs to get merged with all active branches (IIUC).
Yeah. Some help here would be welcome, too, as C is not a language I use often. Cheers, Dirkjan
Dirkjan Ochtman wrote:
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though.
Two things: * We need some form of documentation of how committers are expected to work with the hg repo. (This is also missing for the Subversion repo, which due to the 3.x branch has gotten somewhat unclear - at least for me) It is currently not clear where to check in patches, whether and where to backport or forward-patch, which branches to consider closed, etc. E.g. if I check in something in trunk/ (Python 2.7), do I have to forward patch this change to the 3.0 branch (guess not), the py3k/ branch (Python 3.1), or will someone else take care of this, so that it's better not to interfere by doing it myself ? * Our tracker, checkin messages, online documentation and lots of Python resources on the web are full of references to the Subversion rXYZ notation of changesets. The PEP has to provide some way to gracefully provide a redirect from those URLs to the new ones (both for the source code browser and the bug search tool on python.org). The PEP will also have to address the same issue for checkin messages. Perhaps by converting the rXYZ notation to the hash based values during migration or by adding the has value after the rXYZ value in parens ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 06 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Mon, Jul 6, 2009 at 07:51, M.-A. Lemburg <mal@egenix.com> wrote:
Dirkjan Ochtman wrote:
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though.
Two things:
* We need some form of documentation of how committers are expected to work with the hg repo. (This is also missing for the Subversion repo, which due to the 3.x branch has gotten somewhat unclear - at least for me)
I am planning to get a version of the dev FAQ written up that covers most of what it already does now for svn.
It is currently not clear where to check in patches, whether and where to backport or forward-patch, which branches to consider closed, etc.
E.g. if I check in something in trunk/ (Python 2.7), do I have to forward patch this change to the 3.0 branch (guess not), the py3k/ branch (Python 3.1), or will someone else take care of this, so that it's better not to interfere by doing it myself ?
This question is partially answered by http://www.python.org/dev/faq/#how-do-i-merge-between-branches, but I agree that we should have either this spelled out in the FAQ or a committer-specific doc at www.python.org/dev/ that makes this all very obvious. -Brett
Brett Cannon wrote:
On Mon, Jul 6, 2009 at 07:51, M.-A. Lemburg <mal@egenix.com> wrote:
Dirkjan Ochtman wrote:
In response to some rumblings on python-committers and just to request more feedback, a progress report. I know it's long, I've tried to put to keep it concise and chunked, though. Two things:
* We need some form of documentation of how committers are expected to work with the hg repo. (This is also missing for the Subversion repo, which due to the 3.x branch has gotten somewhat unclear - at least for me)
I am planning to get a version of the dev FAQ written up that covers most of what it already does now for svn.
It is currently not clear where to check in patches, whether and where to backport or forward-patch, which branches to consider closed, etc.
E.g. if I check in something in trunk/ (Python 2.7), do I have to forward patch this change to the 3.0 branch (guess not), the py3k/ branch (Python 3.1), or will someone else take care of this, so that it's better not to interfere by doing it myself ?
This question is partially answered by http://www.python.org/dev/faq/#how-do-i-merge-between-branches, but I agree that we should have either this spelled out in the FAQ or a committer-specific doc at www.python.org/dev/ that makes this all very obvious.
The merge process itself is more or less clear. What I'm missing is the agreed upon strategy for applying the patches to the various branches. I've seen a few discussions about this, but no final statement of what strategy to follow and whether hg makes this easier (AFAIR, that was the main argument for switching to hg). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 07 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Tue, 7 Jul 2009 at 15:26, M.-A. Lemburg wrote:
The merge process itself is more or less clear. What I'm missing is the agreed upon strategy for applying the patches to the various branches.
I've seen a few discussions about this, but no final statement of what strategy to follow and whether hg makes this easier (AFAIR, that was the main argument for switching to hg).
I think the main reason for switching was that it would make it easier for non-core-committers to maintain branches and submit patches (as changesets core committers can pull). I don't think it was ever clear that the merge workflow would in fact get easier, except insofar as hg's merge support is better than SVN's (at least, I believe people have said that last is true). There is _hope_ that it will be easier, but I think it remains to be proven/worked out. And I believe there is no tool like svnmerge for tracking changesets to be merged, which could be an issue that needs a resolution. IIUC, the discussion about named versus cloned branches is part of figuring out the workflow.... --David
I've seen a few discussions about this, but no final statement of what strategy to follow and whether hg makes this easier (AFAIR, that was the main argument for switching to hg).
I think the main reason for switching was that it would make it easier for non-core-committers to maintain branches and submit patches (as changesets core committers can pull).
Indeed, it's all in PEP 374: - motivation 1: give direct tool support to non-committers - motivation 2: allow disconnected (off-line) access to the repository - motivation 3: allow revisions of a patch - motivation 4: allow tracking the mainline while working on a patch - motivation 5: avoid using arcane third-party tools for merge tracking So merge tracking was not the main argument, but the fifth. Regards, Martin
R. David Murray writes:
On Tue, 7 Jul 2009 at 15:26, M.-A. Lemburg wrote:
I've seen a few discussions about this, but no final statement of what strategy to follow and whether hg makes this easier (AFAIR, that was the main argument for switching to hg).
Yes, yes, yes, and no. In reverse order, no: the main argument for switching to hg is that it makes private branching easier. Yes: hg will make public branching easier, too, but that can't be proved until the workflow adjustments get worked out. For that reason, it is essential that the current workflow be supportable essentially without change, and it is. With respect to "how", I'm not a Mercurial geek, but I have been working with Mercurial queues a bit recently in another area, and I think they have some promise for helping organize the workflow. (Although by themselves they're clearly not sufficient, since they're oriented to a single branch.) Yes: there has been no final statement of what strategy to follow because opinions are extremely varied, even as to what is feasible with Mercurial. For example, Dirkjan and Georg want a workflow that makes moving patches among the public branches worry-free Mercurial merges. I believe that means (to the extent it is implemented) essentially gutting the current strategy of cutting maintenance branches and simply lagging the maintenance branches relative to the dev branches, and that it's infeasible for py3k vs. py2. I can't substantiate that; maybe the patch flow would support what they want, I'm not that familiar with how much patches currently morph across branches. And yes: there are a few inconclusive discussions. That's why PEP 374 was written consciously with the intent of postponing the hard issues of workflow across the public branches in favor of picking off the low hanging fruit of private branching and disconnected version control.
I think the main reason for switching was that it would make it easier for non-core-committers to maintain branches and submit patches (as changesets core committers can pull).
Patches or "bundles" aka merge directives. "Pulling" submissions is probably not going to happen; that's a practice that is common with highly distributed workflows, but Python has a fairly centralized workflow.
but I think it remains to be proven/worked out. And I believe there is no tool like svnmerge for tracking changesets to be merged, which could be an issue that needs a resolution.
I think that Mercurial queues or some related extension can be adapted to this, but I can't say for sure (after all, I was the git person on the PEP 374 team :-).
IIUC, the discussion about named versus cloned branches is part of figuring out the workflow....
Peripherally. But actually it's not really relevant to workflow. Anything that can be done with named branches can be done with cloned branches, possibly requiring substantially more space. That discussion really is about whether anything is *lost* by using named branches. I worry that something is lost, but the discussion so far has been inconclusive.
M.-A. Lemburg wrote:
Dirkjan Ochtman wrote: * Our tracker, checkin messages, online documentation and lots of Python resources on the web are full of references to the Subversion rXYZ notation of changesets.
The PEP has to provide some way to gracefully provide a redirect from those URLs to the new ones (both for the source code browser and the bug search tool on python.org).
The PEP will also have to address the same issue for checkin messages. Perhaps by converting the rXYZ notation to the hash based values during migration or by adding the has value after the rXYZ value in parens ?!
I've asked this question before: Dirkjan indicated that he will be writing a web redirection filter that translates from the SVN rXYZ revision numbers to the Hg hash value that corresponds to that revision in the history conversion. I suggest that we also run a version of that redirection filter to remap the old svn.python.org links in addition to making them accessible through hg.python.org as Dirkjan proposed. Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2009-July/090397.html -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
I suggest that we also run a version of that redirection filter to remap the old svn.python.org links in addition to making them accessible through hg.python.org as Dirkjan proposed.
Not sure what links you are talking about, but we also need to keep the current svn.python.org up as-is, for all the stuff that won't be migrated. Regards, Martin
Martin v. Löwis wrote:
I suggest that we also run a version of that redirection filter to remap the old svn.python.org links in addition to making them accessible through hg.python.org as Dirkjan proposed.
Not sure what links you are talking about, but we also need to keep the current svn.python.org up as-is, for all the stuff that won't be migrated.
In that case, no need to redirect anything (although it will still be possible to look up the old SVN revisions in the Hg web view). The previous discussion was based on the assumption that it was going to be possible to switch the SVN server off at some point rather than just disallowing commits to the projects that had migrated to Mercurial. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan wrote:
Martin v. Löwis wrote:
I suggest that we also run a version of that redirection filter to remap the old svn.python.org links in addition to making them accessible through hg.python.org as Dirkjan proposed. Not sure what links you are talking about, but we also need to keep the current svn.python.org up as-is, for all the stuff that won't be migrated.
Good point. So the old svn revision links will continue to work.
In that case, no need to redirect anything (although it will still be possible to look up the old SVN revisions in the Hg web view).
The previous discussion was based on the assumption that it was going to be possible to switch the SVN server off at some point rather than just disallowing commits to the projects that had migrated to Mercurial.
Is there a standard notation for hg revisions that roundup could detect and turn into links (much like it does for svn) ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 07 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Tue, Jul 7, 2009 at 10:16, M.-A. Lemburg<mal@egenix.com> wrote:
Is there a standard notation for hg revisions that roundup could detect and turn into links (much like it does for svn) ?
[a-f0-9]{12} should mostly do. (Sorry for my absence from the discussion for some time. I'll try to update the PEP and summarize ongoing discussions here soon.) Cheers, Dirkjan
Dirkjan Ochtman wrote:
On Tue, Jul 7, 2009 at 10:16, M.-A. Lemburg<mal@egenix.com> wrote:
Is there a standard notation for hg revisions that roundup could detect and turn into links (much like it does for svn) ?
[a-f0-9]{12} should mostly do.
Hmm, no prefix or suffix ? So we'll always have to write "see deadbeefdeadbeefff for details" or "Reverting f00fl33df00fl33d00 after problems on Pentium CPUs" ?! Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 07 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Tue, Jul 7, 2009 at 15:32, M.-A. Lemburg<mal@egenix.com> wrote:
Hmm, no prefix or suffix ?
No, not really. hg often shows revision integers as well, but as they aren't globally consistent, they're of little value in communicating changeset pointers.
So we'll always have to write "see deadbeefdeadbeefff for details" or "Reverting f00fl33df00fl33d00 after problems on Pentium CPUs" ?!
Yes. And it's really not that bad. Cheers, Dirkjan
Hmm, no prefix or suffix ?
No, not really. hg often shows revision integers as well, but as they aren't globally consistent, they're of little value in communicating changeset pointers.
I think MAL wasn't asking for a "1354:" prefix, but for a, say, "r" or "R" prefix, or perhaps "V" or "merc:". I think you are proposing that there is no prefix because the chance for a misidentification of some word as a hg revision number is small, as it has to be exactly 12 hex digits. Regards, Martin
Martin v. Löwis wrote:
I think you are proposing that there is no prefix because the chance for a misidentification of some word as a hg revision number is small, as it has to be exactly 12 hex digits.
So triggering it accidentally would require a 12 letter word or object name that used only the letter a-f? Agreed that seems very unlikely. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
M.-A. Lemburg wrote:
Dirkjan Ochtman wrote:
On Tue, Jul 7, 2009 at 10:16, M.-A. Lemburg<mal@egenix.com> wrote:
Is there a standard notation for hg revisions that roundup could detect and turn into links (much like it does for svn) ? [a-f0-9]{12} should mostly do.
Hmm, no prefix or suffix ?
So we'll always have to write "see deadbeefdeadbeefff for details" or "Reverting f00fl33df00fl33d00 after problems on Pentium CPUs" ?!
No, rather "see deadbeefdead for details" and "Reverting f00fl33df00fl33d00 after problems on Pentium CPUs". In practice, rather 30ba63d28b1b or 12fb3b32d75d (from html5lib, fwiw). Dirkjan meant "[a-f0-9]{12}" literally. Regards, Martin
participants (18)
-
"Martin v. Löwis" -
Aahz -
Barry Warsaw -
Benjamin Peterson -
Brett Cannon -
Dirkjan Ochtman -
Georg Brandl -
M.-A. Lemburg -
Mark Hammond -
Mark Hammond -
MRAB -
Nick Coghlan -
P.J. Eby -
Paul Moore -
R. David Murray -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy