[Twisted-Python] Tracking twisted repository post 2.4.0?
We've got some locally maintained modifications to Twisted that prior to 2.4.0 we were tracking by syncing our repository with the Twisted trunk (as a vendor) at the overall official tags. But with the tagging changes with 2.4.0 (and without a tag for the overall trunk), I was wondering if there were suggestions on how to adjust our setup. I can understand why there is no single tag for the trunk, since an overall "Sumo" release of twisted (and even the overall 2.4.0 concept) is really a combination of different releases of the various subpackages, but I can't figure out how to use the existing tags to track those pieces we are interested in. For example, we really use core and web, so I had thought I could just export, for example, the twisted-core-2.4.0 and twisted-web-0.6.0 tags from tags/releases, merge them together and call that our "vendor" dump for 2.4.0. But it would appear that the individual project tags are still based on tags for the complete trunk, and thus contain all trunk files as of when the subproject was tagged. Thus, if I try to use a checkout/export from twisted-web-0.6.0, then depending on when it was tagged, it may overlay files that came from twisted-core-2.4.0 with different versions that aren't really part of core 2.4.0. So how (at the subversion tag level) if at all can I isolate those files from beneath the twisted-web-0.6.0 tag that only apply to twisted-web, so that I can add them to the files from twisted-core-2.4.0 to yield my simulated vendor dump? Is anyone else tracking the repository (but not instant to instant at the trunk, but rather at release points) and if so, how are you doing it post 2.4.0? Thanks. -- David
On 15 Jun 2006 20:07:58 -0400, David Bolen <db3l@fitlinxx.com> wrote:
We've got some locally maintained modifications to Twisted that prior to 2.4.0 we were tracking by syncing our repository with the Twisted trunk (as a vendor) at the overall official tags.
Have you submitted those as patches in the bugtracker? If so, are we being really sluggish about applying them? :) As you have unfortunately noticed with this change... we don't currently provide any support for configurations like this. I'd personally like to discourage it, because long-term local modifications to Twisted indicate a failure of process, either you submitting them or us accepting them.
But with the tagging changes with 2.4.0 (and without a tag for the overall trunk), I was wondering if there were suggestions on how to adjust our setup.
I can understand why there is no single tag for the trunk, since an overall "Sumo" release of twisted
To reduce confusion we're trying to remove the word "sumo" everywhere. If you spot that word still being used somewhere on the site it's a bug, please let us know. Right now it's just "a release of Twisted".
So how (at the subversion tag level) if at all can I isolate those files from beneath the twisted-web-0.6.0 tag that only apply to twisted-web, so that I can add them to the files from twisted-core-2.4.0 to yield my simulated vendor dump?
You can't, really. Hopefully the internal repository structure of Twisted will one day look more like this: http://divmod.org/trac/browser/trunk and the tags will refer to project directories, like this: http://divmod.org/trac/browser/tags/releases but until that happens, the setup.py files for separate projects determine a fairly arbitrary collection of files. Specifically, ownership of twisted/plugins/ is spread out between all subprojects. Also, some files are moved around; the "topfiles" directory is pulled out to the root of each subproject as part of the release process.
Is anyone else tracking the repository (but not instant to instant at the trunk, but rather at release points) and if so, how are you doing it post 2.4.0?
Divmod doesn't track Twisted releases through SVN; any changes we are in the process of contributing but which haven't been released are kept in our utility package "Epsilon". Epsilon has a "hotfixes" module which allows us to modify Twisted's behavior in cases where we really need to modify the library rather than just package additional functionality outside it -- you might look at that implementation as an example of a way to formalize monkeypatching and reduce its fragility.
glyph@divmod.com writes:
On 15 Jun 2006 20:07:58 -0400, David Bolen <db3l@fitlinxx.com> wrote:
We've got some locally maintained modifications to Twisted that prior to 2.4.0 we were tracking by syncing our repository with the Twisted trunk (as a vendor) at the overall official tags.
Have you submitted those as patches in the bugtracker? If so, are we being really sluggish about applying them? :)
It depends. There really aren't many of them but a few are changes to PB to pass some additional types (such as mx.DateTime) that I don't really see as valid for general incorporation into Twisted, but that were the simplest way to resolve some issues for ourselves in the past. There was also an ongoing email discussion about better passing of exception objects through PB that we implemented during the discussion but I am not certain if all the same support got committed into the core (it wasn't clear at the time what was willing to be accepted). Others did get submitted (see ticket 669), but show an example of how even when a fix does eventually make it in (in this case via a reworked implementation) there can be a long period of time - across multiple Twisted "releases" - where we need to maintain it as a local patch for our own operations before we can retire it.
As you have unfortunately noticed with this change... we don't currently provide any support for configurations like this. I'd personally like to discourage it, because long-term local modifications to Twisted indicate a failure of process, either you submitting them or us accepting them.
But I think it's somewhat unrealistic to expect there not to be short term local modifications - certainly I find that to be fairly common with a large percentage of open source packages I track. It's not unexpected - when I fix something locally it's normally to scratch an immediate itch I have, but there's bound to be some latency before either it can be cleaned up for a good submission to the package maintainer and/or it gets accepted/applied. It's unfortunate that the recent changes in the Twisted source tree aren't that friendly to doing this (particularly with generally easy ways to handle such issues with CVS and/or Subversion). Long term I'd agree - although I do still see times where some modifications just won't likely get accepted back and will be maintained for a while.
I can understand why there is no single tag for the trunk, since an overall "Sumo" release of twisted
To reduce confusion we're trying to remove the word "sumo" everywhere. If you spot that word still being used somewhere on the site it's a bug, please let us know.
Right now it's just "a release of Twisted".
No problem. Off the top of my head I only remember noticing it on the web site on the "What is Twisted" page, but I think it was in an older News entry.
but until that happens, the setup.py files for separate projects determine a fairly arbitrary collection of files. Specifically, ownership of twisted/plugins/ is spread out between all subprojects. Also, some files are moved around; the "topfiles" directory is pulled out to the root of each subproject as part of the release process.
I did notice that but figured I'd ignore the root files when combining for a vendor dump (since they also existed independently under topfiles). I suppose I could try just executing the setup for each sub-project I am interested in, and using the result of the build directory as the appropriate collection of files. At least that way I'm properly in sync with what is considered part of each sub-project.
Divmod doesn't track Twisted releases through SVN; any changes we are in the process of contributing but which haven't been released are kept in our utility package "Epsilon". Epsilon has a "hotfixes" module which allows us to modify Twisted's behavior in cases where we really need to modify the library rather than just package additional functionality outside it -- you might look at that implementation as an example of a way to formalize monkeypatching and reduce its fragility.
Perhaps, although to be honest, I'd be contrasting that effort versus just using the source code system's capabilities, since managing such local modifications is one of the things it does reasonably well, and I already do for a number of other projects we use. But thanks, at least I know I'm not missing something obvious from the transition to the 2.4.0 tagging approach. -- David
On Jun 16, 2006, at 4:24 PM, David Bolen wrote:
As you have unfortunately noticed with this change... we don't currently provide any support for configurations like this. I'd personally like to discourage it, because long-term local modifications to Twisted indicate a failure of process, either you submitting them or us accepting them.
But I think it's somewhat unrealistic to expect there not to be short term local modifications - certainly I find that to be fairly common with a large percentage of open source packages I track. It's not unexpected - when I fix something locally it's normally to scratch an immediate itch I have, but there's bound to be some latency before either it can be cleaned up for a good submission to the package maintainer and/or it gets accepted/applied. It's unfortunate that the recent changes in the Twisted source tree aren't that friendly to doing this (particularly with generally easy ways to handle such issues with CVS and/or Subversion).
Long term I'd agree - although I do still see times where some modifications just won't likely get accepted back and will be maintained for a while.
I'm very sympathetic to this, and I'd really like for the SVN source tree to be arranged better than it is. The main constraint is keeping the ability to run from a svn checkout directly, without an install step. I made a branch that did reorganize things by subproject. And that pretty much worked fine except that trial gets totally confused when running regressions on it. I then also patched trial to make it not blow up. However, that particular way of fixing trial was rejected, and no other has been implemented yet. Reorganization is essentially blocked until this issue is solved. James
participants (3)
-
David Bolen -
glyph＠divmod.com -
James Y Knight