[Twisted-Python] On splitting Twisted into subprojects
Note: Cross-posted to twisted-web and twisted-python. Decide between them for your follow-ups please :) This is intended to spark a little more discussion on splitting Twisted into subprojects, as discussed on Aussie-time on IRC (Jerub, jml, spiv and myself). It seems like the twisted-web project are planning to split off fairly soon: see http://twistedmatrix.com/pipermail/twisted-web/2004-April/000312.html It's pretty likely that this split will go ahead without any kind of imposed-from-above decisions about where the website will go, what the release cycle will look like, where the docs will move to, what the status of shared docs will be (I tend to focus on docs...) The nightmare scenario for splitting twisted into subprojects is that every subproject develops its own release code, release procedures, testing framework, website, documentation style, community, policies... etc etc. And shortly after that we all spiral into the sun or something. There appeared to us to be three possible ways of avoiding this: 1. Luck 2. Release code, release procedures, testing framework, website, documentation style, community, policies... imposed from above 3. Twisted Web (or whatever other project splits first) acting as a "model split": evolving release mechanisms, policy etc etc that are fundamentally sane and wonderful so that noone would think of doing things any other way. Since 3 looks like being the most achievable, this is basically an appeal to the twisted-web folk to keep in mind that they should, where possible, try and make twisted-web the Model of a Modern Twisted Subproject. And make it as easy as possible for other subprojects to copy whatever it is that they're going to do. I personally would like to offer my help in documenting subproject procedures as they evolve, and storing that documentation in some kind of prominent place. -Mary
Mary Gardiner wrote: Hi Mary; Sorry I haven't kept up with the discussion about the Twisted split lately.
This is intended to spark a little more discussion on splitting Twisted into subprojects, as discussed on Aussie-time on IRC (Jerub, jml, spiv and myself).
It seems like the twisted-web project are planning to split off fairly soon: see http://twistedmatrix.com/pipermail/twisted-web/2004-April/000312.html
Actually, I'm going to beat them to it: Jp and I are going to be splitting off twisted.news this weekend. (*looks around for Jp*)
It's pretty likely that this split will go ahead without any kind of imposed-from-above decisions about where the website will go, what the release cycle will look like, where the docs will move to, what the status of shared docs will be (I tend to focus on docs...)
Web site will be somewhere on twistedmatrix.com. How about: http://projects.twistedmatrix.com/<projectname> ?
The nightmare scenario for splitting twisted into subprojects is that every subproject develops its own release code, release procedures, testing framework, website, documentation style, community, policies... etc etc. And shortly after that we all spiral into the sun or something.
I've been working on the release automation the last few days. At *least* the SVN release process workflow will probably remain the same between all projects; I'll be splitting release-twisted off into a module in twisted.python soon to facilitate other projects using it. We *want* to keep the community, documentation styles, web site, policies, all integrated fairly well. A little more on this later..
2. Release code, release procedures, testing framework, website, documentation style, community, policies... imposed from above
Ok. :-)
3. Twisted Web (or whatever other project splits first) acting as a "model split": evolving release mechanisms, policy etc etc that are fundamentally sane and wonderful so that noone would think of doing things any other way.
Well, this is why I'm starting with twisted.news. I'll also probably get to a few more projects before the web split occurs, all of them less popular. news, flow, (im, words), lore, conch, probably in something like that order. Web is going to be more complex, because they're doing a rewrite now, and they don't know how they want to integrate nevow, etc...
I personally would like to offer my help in documenting subproject procedures as they evolve, and storing that documentation in some kind of prominent place.
You posted about the documentation a short while ago, and I'll respond about it here. Mary wants the documentation to be kept in a single project. I don't think this is how it should be done; we'll allow each project to keep its own doc/ directory. Each project should be responsible for its own documentation, but I can personally ensure that you have access to the docs for all the projects, though, Mary. :-) HOWEVER, don't worry that the documentation will become disparate in any user-visible sense. All projects will have their documentation visible somewhere within twistedmatrix.com/, and we could even have a monolithic "docs" section that links to all the others if we find it becomes necessary (I think the current 'howto' page is an absolute mess, though). The "Twisted Book" is up for debate; personally, I don't care about it, and I'd be inclined to just have a per-project "Book" (collection of HOWTOs in ps/pdf form). If people really think it's a good idea, though, there's no reason that we can't have the same monolithic book. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://radix.twistedmatrix.com/
On Apr 17, 2004, at 1:02 PM, Christopher Armstrong wrote:
The "Twisted Book" is up for debate; personally, I don't care about it, and I'd be inclined to just have a per-project "Book" (collection of HOWTOs in ps/pdf form). If people really think it's a good idea, though, there's no reason that we can't have the same monolithic book.
The "twisted book" should not be a random conglomeration of HOWTOs (which are badly named anyway, they're *not* how-to documents, they're articles about concepts), it should be a book with some organization and structure. The articles should stand on their own where possible, but there are a lot of interrelated concepts that you have to learn and they should reference each other.
On Sat, Apr 17, 2004, Christopher Armstrong wrote:
Mary wants the documentation to be kept in a single project. I don't think this is how it should be done; we'll allow each project to keep its own doc/ directory. Each project should be responsible for its own documentation, but I can personally ensure that you have access to the docs for all the projects, though, Mary. :-)
HOWEVER, don't worry that the documentation will become disparate in any user-visible sense. All projects will have their documentation visible somewhere within twistedmatrix.com/, and we could even have a monolithic "docs" section that links to all the others if we find it becomes necessary (I think the current 'howto' page is an absolute mess, though).
Yeah the howto page is a mess (*and* misnamed as Glyph points out elsewhere). There's also very weird coverage in places (detailed things covered, basic things missing) that having subprojects may help with, because coverage of their subproject will be important to them. Goodness knows what happens to the tutorial in this model. It could perhaps split into pieces and be greatly expanded by each subproject. It definitely is useful to people: most docs bugs are filed against it. At a lower level, all I really want from the docs is two things: 1. Ability to find them in some kind of defined spot and to be able to edit the etc. You state this above. 2. Some kind of policy about certain things which I can't even begin to define yet, but which may include directory structure, referencing and the like. However, since doc refactors are easier than code refactors, I'm happy to make policy when required rather than do it in advance. 3. Cooperation from subproject developers: ie ability to hassle them about docs, nag them about broken and missing docs and that sort of thing. (At present, I file docs bugs against myself, that can't continue with X subprojects :) ) I think at the moment I don't really have 3, because a solid half of the project's developers aren't aware that there is someone who has (however tenuous) responsibility for the docs, especially since the twisted-web list appeared fairly early in that process. I may be misjudging people there. -Mary
On Sun, Apr 18, 2004, Mary Gardiner wrote:
I think at the moment I don't really have 3, because a solid half of the project's developers aren't aware that there is someone who has (however tenuous) responsibility for the docs, especially since the twisted-web list appeared fairly early in that process. I may be misjudging people there.
On reflection, this is probably most of the reason I like a separate docs project: if people have to commit docs into a separate project, they would probably be more aware of a cross-Twisted documentation policy (insofar as one exists, which it will, I hope). However, that's clearly not sufficient justification (because if you replace the words "docs" with "code" the statement becomes stupid). -Mary
On Apr 17, 2004, at 8:35 AM, Mary Gardiner wrote:
Since 3 looks like being the most achievable, this is basically an appeal to the twisted-web folk to keep in mind that they should, where possible, try and make twisted-web the Model of a Modern Twisted Subproject. And make it as easy as possible for other subprojects to copy whatever it is that they're going to do.
I think that we are going to need some of #2 as well. Without the Imposition From Above, the subprojects are likely to start in a similar place (modeled after Twisted.Web, as you suggest) and gradually diverge based on evolutionary needs, leaving them all completely different and weird in a year's time. Your focus on docs is correct, though, because what we really need is a shared policy document :). What we ALSO need, Christopher Armstrong, if you are reading this, is a release-twisted script which is its own, bootstrap-released project that can be run on ANY of the twisted subprojects to generate an instant release.
Glyph Lefkowitz wrote:
On Apr 17, 2004, at 8:35 AM, Mary Gardiner wrote:
Since 3 looks like being the most achievable, this is basically an appeal to the twisted-web folk to keep in mind that they should, where possible, try and make twisted-web the Model of a Modern Twisted Subproject. And make it as easy as possible for other subprojects to copy whatever it is that they're going to do.
I think that we are going to need some of #2 as well. Without the Imposition From Above, the subprojects are likely to start in a similar place (modeled after Twisted.Web, as you suggest) and gradually diverge based on evolutionary needs, leaving them all completely different and weird in a year's time.
Your focus on docs is correct, though, because what we really need is a shared policy document :).
What we ALSO need, Christopher Armstrong, if you are reading this, is a release-twisted script which is its own, bootstrap-released project that can be run on ANY of the twisted subprojects to generate an instant release.
Ahem, I've been working on release-twisted full-time for the past 2 or 3 days :-) Anyway, I see no point in making it bootstrapped. The only "bootstrapping" that would theoretically be necessary is for releasing Twisted itself, where r-t is contained, and that's been possible since its conception. All of the other projects can just rely on the presence of twisted during their release. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://radix.twistedmatrix.com/
participants (3)
-
Christopher Armstrong
-
Glyph Lefkowitz
-
Mary Gardiner