Re: Future of pipermail?
WebDAV works really, really well as an archiver. And it would be
particularly cool if the archive already had the MIME pieces exploded out.
Even cooler would be if the messages sent to the list were rewritten such that
all of the MIME pieces-- typically large, on some lists-- were actually URLs to
the archived pieces on teh WebDAV server.
If it had an interface that sent an HTTP message to some random
webserver/appserver/cgi-bin letting it know a new message was submitted so that
server could decide an appropriate time to rebuild an index file, that would
allow the presentation of the archive to be offloaded from Mailman.
Since WebDAV will be included with every copy of Apache 2.0 by default, there
really isn't that much additional "moving pieces" load on the configuration.
Mailman already requires diddling the web server configuration-- copy/paste a
few more lines isn't that big of a deal.
Anyone want to take this on?
Oh, wait a minute, that's right... it has already been done!
ftp://ftp.codefab.com/pub/unsupported/mailman_20000821_1123.tar.gz
It is based on a version of Mailman from sometime around May or so... so, it
needs to be updated to the latest version from the repository. However, it
DOES work and has been used in production systems [who will remain nameless to
protect this community from me venting] for quite some time. The company that
hired me to build this actually paid a bunch of $$$ for it to be built-- as
such, I actually had the freedom to thiink about doing it right and not just
hacking it on as fast as possible.
I don't know if I did do it right, but Barry provided quite a bit of guidance
and I don't think it is a totally offensive set of changes...
b.bum
At 12:11 PM -0500 11/21/00, Bill Bumgarner wrote:
WebDAV works really, really well as an archiver.
webDAV?
Oh, man. I guess I need to go look at another piece of technology. Wanna give those of us who have no clue what this is the 30 second executive overview?
We can't, however, assume users are running Apache, even if it's the overwhelming choice...
Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy
Chuq, et. al.,
See www.webdav.org for more information.
WebDAV is a standard that extends, but does not modify, HTTP to allow for
several additional operations against a web server. There are two levels to
the specification. Class 0 will be described before-- it is complete and
production quality. Class One is in the final stages of discussion and, as
such, is not anywhere near having a reference implementation (but I will
discuss briefly because it is very cool):
- supports PUT of resources [files] onto the server (i.e. upload a file to server)
- creation of collections (mkdir)
- renaming/moving/deleting/copying of resources or collections
- assigning properties to resources or collections. A set of properties is basically a chunk of XML associated with the collection or
resource. For the MailMan WebDAV extensions, I use properties to store the
mime-type and any extended headers associated with the attachment in question.
- searching the properties database for particular values/attributes
- locking; locking is limited to write locks because read locks would have required modification of the HTTP spec. Can have either shared or
exclusive write locks. However, it is trivial to throw together a cgi-bin/
that allows read access to a resource if and only if the client sends a long
the appropriate shared write lock token. This is used in the Mailman WebDAV
extensions to support lmited access to the archive of messages.
- locks can have a timeout. As such, it is trivial to generate an URL that allows access to a particular piece of the archive for, say, only the
next 30 minutes.
Most importantly, WebDAV is truly a standard and a very widely accepted
standard, at that.
- Apple ships a WebDAV client with Mac OS X. The builds of Apache with Mac OS X and Mac OS X Server both include WebDAV modules that are disabled
by default.
- Microsoft has full blown WebDAV support in Office and Windows 2000. In Office, you can open a document served by a WebDAV server and subsequently
hit ctrl-s to save (or save as) as if the document were on a local fileserver.
- IBM's WebSphere fully supports WebDAV
- Zope fully supports WebDAV
- GoLive CyberStudio and other HTML authoring packages either ship with WebDAV support or have announced that they will soon.
So, WebDAV is, by no means, an apache only solution! Because it is a simple
extension to HTTP-- and class 0 is relatively trivial in nature-- WebDAV
capabilities can be provided via cgi-bin/ without a problem.
The new class of DAV functionality is aimed at full support for version
control. This includes basic revisioning as well as tags, maps, workareas,
multiple users, etc... This revision to the spec is in the final stages of
discussion and Greg Stein-- author of mod_dav-- is actively working with the
group to create an implementation of the spec.
In terms of Mailman, there is no real reason why creating the archive should
be something that isn't completely abstracted within Mailman. Archival is
basically three tasks:
- storing data
- storing meta-infrormation
- creating some kind of index/viewDecoding of attachments and rewriting the messages is mostly just an extension
of the above concepts.
As such, the Mailman archival interface could be an abstract interface of
which we provide two concrete implementations; direct-to-filesystem and
WebDAV. The actual process of creating an index/view isn't really a Mailman
problem-- obivously, it is useful to ship with an out-of-the-box solution.
b.bum
From: Chuq Von Rospach chuqui@plaidworks.com Date: 2000-11-21 10:00:53 -0800 To: bbum@codefab.com, mailman-developers@python.org Subject: [Mailman-Developers] Re: Future of pipermail? In-Reply-To: 200011211711.eALHB1322170@bjork.codefab.com
At 12:11 PM -0500 11/21/00, Bill Bumgarner wrote:
WebDAV works really, really well as an archiver.
webDAV?
Oh, man. I guess I need to go look at another piece of technology. Wanna give those of us who have no clue what this is the 30 second executive overview?
We can't, however, assume users are running Apache, even if it's the overwhelming choice...
Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy
WebDAV: stands for "Web-based Distributed Authoring and Versioning", which sort of neatly sums up what Bill was saying about its capabilities. (It's new to me, too. There seem to be two new web-database "environments" a week these days.)
"BB" == Bill Bumgarner bbum@codefab.com writes:
BB> This revision to the spec is in the final stages of discussion
BB> and Greg Stein-- author of mod_dav-- is actively working with
BB> the group to create an implementation of the spec.I'll just note briefly that Greg is a long time Pythonista and Mailmaner (in fact he owns the list.org domain), so WebDAV support in Python ought to be very good.
I want to read and think more, but I've been interested in WebDAV as a technology for a while now.
-Barry
At 5:33 PM -0500 11/21/00, Barry A. Warsaw wrote:
I'll just note briefly that Greg is a long time Pythonista
excuse me if I quietly twich at the terminology (does anyone know where this -ista stuff came from? My first run-in iwth it was Guy Kawasaki and his "evangelistas", but was it used before that?
I want to read and think more, but I've been interested in WebDAV as a technology for a while now.
I'm starting to see serious pushes (and I'll self-admit to be one here!) towards serious, hard-core integration of technologies. Which is a double-edged sword of the worst and best kind. I've talked to Barry about building ways to allow for site-wide authentication issues, for instance, and now we're seeing WebDAV (which I find interesting, and given it's going to be part of Apache 2.0, it's got to be considered heir presumptive in this technology psace now, over things like, oh, Zope. And I was looking over a couple of other things today (MetaDot to name one.. http://www.metadot.com/metadot/index.pl?iid=1876) and going "hmm. you know, that stuff's got potential, too...)
There's HUGE synergy potential here. We certainly shouldn't avoid those things, but the more we start tying into (and/or depending on) other technologies, the more we lock users into a single technology suite, the more we start depending on specific technology suites the more we lock uers into our view of how things work, and the less flexibility we give them in building their sites. And, to some degree, the less ability we have to adapt to future changes in the net universe ourselves.
None of which is a reason to not do these things -- but it's reason to be wary. We really need to understand how these things interact and what the side effects are, including things as basic as "does this mean it'll never run on Windows NT again?" or "did we just set it up so they have to run apache?"
I think there's a lot here we want to do (and need to do) but -- not for the sake of doing it.
WebDAV looks fascinating. WebDAV -- as mailman's default archiver -- looks to me to be overkill at a quick evaluation. Yes, it'll be in there by default for apache 2.0, once Apache 2.0 ships and is out for a while and everyone upgrades. that's, what, 2 years out before 50% of the sites are running apache 2.x? Easily. Until then, what do we do? And what about non-apache sites? what's this do to mailman across all supported platforms?
I dunno -- but there's a lot of complexity here we need to make sure we deal with in these issues. And to some degree, I think it's another excuse for me to pull out the "lots of separately tracked modules with API's" schtick, because we can build a Mailman API and a WebDAV plug-in for it(along with Mhonarc and Pipermail plug-ins) and over time, as WebDAV becomes endemic, switch the priority of development in its direction, without locking into it. In two years, there'll be other stuff beyond WebDAV, too, I'll bet.
I know, I know... it's all details. Details are boring (grin). but necessary...
Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy
There's HUGE synergy potential here. We certainly shouldn't avoid those things, but the more we start tying into (and/or depending on) other technologies, the more we lock users into a single technology suite, the more we start depending on specific technology suites the more we lock uers into our view of how things work, and the less flexibility we give them in building their sites. And, to some degree, the less ability we have to adapt to future changes in the net universe ourselves.
I'd like to suggest that instead of integrating other technologies, that making Mailman open to them would be better. Such as offering the alternative to use Webdav, or making APIs for PHP, Perl, etc to interact with it.
This is a classic COTS integration issue. If you depend too heavily on a product, it is possible for a future release of that product to break yours and the other guys don't care because you're a small fry in the pond. Been there myself and ended up gutting a product I was working on.
My apologies for butting in - been lurking a long time and this is an interesting thread.
Ken Kyler
"KK" == Ken Kyler ken@kyler.com writes:
KK> I'd like to suggest that instead of integrating other
KK> technologies, that making Mailman open to them would be
KK> better. Such as offering the alternative to use Webdav, or
KK> making APIs for PHP, Perl, etc to interact with it.
KK> This is a classic COTS integration issue. If you depend too
KK> heavily on a product, it is possible for a future release of
KK> that product to break yours and the other guys don't care
KK> because you're a small fry in the pond. Been there myself and
KK> ended up gutting a product I was working on.Yes, but I think there's a difference in that we're going to be relying on open standards, not closed COTS technology. Mailman, of course, is already built on a number of standards: RFC 822, SMTP, NNTP, HTTP, CGI, etc., etc. Any of these could go away or be replaced or whatever, but we're doing a good thing by relying on these standards. Of course, you have to pick the right ones, and (IMO) build an architecture that lets you more easily move to new standards when they arise and become useful.
KK> My apologies for butting in - been lurking a long time and
KK> this is an interesting thread.The more the merrier!
-Barry
standards. Of course, you have to pick the right ones, and (IMO) build an architecture that lets you more easily move to new standards when they arise and become useful.
How true - what I would give for omniscience.
Keep up the good work guys! If I ever finish grad school I'll help.
Ken
At 6:55 PM -0500 11/21/00, Ken Kyler wrote:
Keep up the good work guys! If I ever finish grad school I'll help.
hey, this ain't HP, guy! we don't care about pieces of paper! (grin)
-- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy
"KK" == Ken Kyler ken@kyler.com writes:
>> standards. Of course, you have to pick the right ones, and
>> (IMO) build an architecture that lets you more easily move to
>> new standards when they arise and become useful.
KK> How true - what I would give for omniscience.Don't forget Guido's time machine! When he's away, we like to borrow the keys and take 'er for a spin. One thing I've learned though: NEVER touch the purple lever, no matter how loudly it pleads with you. And the three-pole beaver switch only works if your Pythonic spirit is pure.
KK> Keep up the good work guys! If I ever finish grad school I'll
KK> help.Thanks! -Barry
At 7:04 PM -0500 11/21/00, Barry A. Warsaw wrote:
One thing I've learned though: NEVER touch the purple lever, no matter how loudly it pleads with you.
Only one thing I can say about this -- if the *lever* is pleading, you need to switch to Decaf.
-- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com)
The vet said it was behavioral, but I prefer to think of it as genetic. It cuts down on the liability -- Get Fuzzy
On Tue, Nov 21, 2000 at 06:55:57PM -0500, Ken Kyler wrote:
Keep up the good work guys! If I ever finish grad school I'll help.
Shht, don't tell Barry, but I never finished grad school (or any form of highschool, for that matter) and he still put some of my fixes into Mailman, and adapted others ;)
-- Thomas Wouters thomas@xs4all.net
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
"CVR" == Chuq Von Rospach chuqui@plaidworks.com writes:
CVR> excuse me if I quietly twich at the terminology (does anyone
CVR> know where this -ista stuff came from? My first run-in iwth
CVR> it was Guy Kawasaki and his "evangelistas", but was it used
CVR> before that?Dunno, and I'm not sure who invented "Pythonista" or "Perlie" or whatever. I actually like the term "Pythoneer" because it's reminiscent of "Pioneer" and "Engineer". Maybe we should just call them "Bruces" (to avoid confusion :).
CVR> I'm starting to see serious pushes (and I'll self-admit to be
CVR> one here!) towards serious, hard-core integration of
CVR> technologies. Which is a double-edged sword of the worst and
CVR> best kind.I think the trick is to balance what is necessary and expedient for Mailman with long term flexibility and integration with standards. Given the size of the todo list, where we can leverage existing technology we probably want opt that way instead of inventing our own. But the flexibility is important because different sites have so many different requirements.
That's why I like the idea of moving toward a core set of classes and scripts that provide the basic Mailman functionality, building interfaces and hooks so that people can do the customization they want.
CVR> There's HUGE synergy potential here. We certainly shouldn't
CVR> avoid those things, but the more we start tying into (and/or
CVR> depending on) other technologies, the more we lock users into
CVR> a single technology suite, the more we start depending on
CVR> specific technology suites the more we lock uers into our
CVR> view of how things work, and the less flexibility we give
CVR> them in building their sites. And, to some degree, the less
CVR> ability we have to adapt to future changes in the net
CVR> universe ourselves.
CVR> None of which is a reason to not do these things -- but it's
CVR> reason to be wary. We really need to understand how these
CVR> things interact and what the side effects are, including
CVR> things as basic as "does this mean it'll never run on Windows
CVR> NT again?" or "did we just set it up so they have to run
CVR> apache?"I agree.
CVR> WebDAV looks fascinating. WebDAV -- as mailman's default
CVR> archiver -- looks to me to be overkill at a quick
CVR> evaluation. Yes, it'll be in there by default for apache 2.0,
CVR> once Apache 2.0 ships and is out for a while and everyone
CVR> upgrades. that's, what, 2 years out before 50% of the sites
CVR> are running apache 2.x? Easily. Until then, what do we do?
CVR> And what about non-apache sites? what's this do to mailman
CVR> across all supported platforms?
CVR> I dunno -- but there's a lot of complexity here we need to
CVR> make sure we deal with in these issues. And to some degree, I
CVR> think it's another excuse for me to pull out the "lots of
CVR> separately tracked modules with API's" schtick, because we
CVR> can build a Mailman API and a WebDAV plug-in for it(along
CVR> with Mhonarc and Pipermail plug-ins) and over time, as WebDAV
CVR> becomes endemic, switch the priority of development in its
CVR> direction, without locking into it. In two years, there'll be
CVR> other stuff beyond WebDAV, too, I'll bet.Agreed again.
CVR> I know, I know... it's all details. Details are boring
CVR> (grin). but necessary...Indeed. There's also the "worse is better" mantra, or put in other words: "All software sucks, let's just try to suck less". :)
-Barry
participants (6)
-
barry＠digicool.com -
Bill Bumgarner -
Chuq Von Rospach -
Dan Mick -
Ken Kyler -
Thomas Wouters