At 5:01 PM +0900 2005-04-30, Stephen J. Turnbull wrote:

Any subscriber might be keeping and publishing an archive of the list posts.

True enough.

     If the listmaster doesn't like that, he should be vetting each

subscription, and making sure that each subscriber understands the rules.

We do that as much as we can.  We don't really care about some of 

the lists, but there are others we care about a great deal.

In the particular case of Gmane, I think they either added our 

lists to their archives at a time when their policies weren't so clear, or they did so before I became the listmaster, and the people who were around for that role did not remember any conversations with the Gmane people.

                                This actually answers most of your

worries, Brad; ie, if Gmane gatewaying were part of the Mailman configuration process rather than at Gmane's option, your ntp lists would never have been gatewayed and archived, right?

If I could guarantee that I always had full control over that 

process, from my end and without requiring any intervention on the part of personnel at Gmane (or mail-archive.com, or whomever), then I'd be happy with that part.

The second is that this patch evidently constitutes a significant endorsement of The Mail Archive.

I had thought about that too, but I couldn't come up with 

anything more than a recognition of the fact, so I didn't mention it. My original response was unclear enough as it was, and I knew it -- I didn't want to muddy the waters further.

... if Mailman is going to endorse services that way. I don't really think it's a good idea in principle, though. What happens if The Mail Archive goes away or goes proprietary? What are people going to think if The Mail Archive's maintainers hire Barry or Brad? Etc, etc.

Good questions.  I don't think I have any answers.

-- Brad Knowles, <brad@stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."

 -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
 Assembly to the Governor, November 11, 1755

SAGE member since 1995. See <http://www.sage.org/> for more info.

Stephen J. Turnbull wrote:

The second is that this patch evidently constitutes a significant endorsement of The Mail Archive. As I understand Jeff's post, he went to the trouble of asking Lars if he would like a similar setup added for Gmane, patch to be coded by Jeff || Jeff. I have to admit that somebody who would go to that much trouble to implement the same feature for a close substitute sounds pretty endorsable to me!

... if Mailman is going to endorse services that way. I don't really think it's a good idea in principle, though. What happens if The Mail Archive goes away or goes proprietary? What are people going to think if The Mail Archive's maintainers hire Barry or Brad? Etc, etc.

On the pro side, there is the point that this would make such mirroring an opt-in on the listmaster's side, which is good. So it might be worth Mailman thinking about under what conditions it would be good to make such an endorsement, and adding anybody who satisfied those conditions.

I personally don't see it as being a significant endorsement. AIUI, it's a patch that allows 2 software programs to work well together.
Mailman already provides patches or directions to make mailman work well with qmail and postfix, but I don't believe this constitutes an "endorsement" of qmail and/or postfix. Patches/directions that make mailman work well with archivers are similar - both with local archiver software and with mirror "site" archiver software. The documentation for these patches (or directions) should simply make it clear that the other programs - *all* the other programs (qmail, postfix, MHonArc, gmane, Mail Archive, etc.) - are not Mailman programs, and that the patches (or directions) are included solely for those who want to make mailman work well with the indicated outside-provided software or service.

A simple patch philosophy that will address this and future patches: If the patch is incorporated into the main mailman build then the default state should be "off" so that the listmaster has to proactively turn the feature "on" for their mailman list server. If the patch remains as an add-on feature that has to be downloaded separately from the main build, then the default state could be "on" (since downloading the patch separately would be a clear indication that the listmaster desired to implement this feature on their mailman list server). This should be the same for all patches that specifically enable cooperation between mailman and any other outside-provided software or service.

IMHO, etc. Caveat: I'm not a developer so those of you who actually write code and build/install the software (I just run it after someone else installed it for me :-) have much more say on this than I do. So if you hate my idea, speak up!

jc

Stephen J. Turnbull wrote:

...

...

...

...

...

"JC" == JC Dill <lists05@equinephotoart.com> writes:

JC> I personally don't see it as being a significant endorsement. JC> AIUI, it's a patch that allows 2 software programs to work JC> well together.

My understanding is that the programs already work well together; you just type "archive@mail-archive.org" into "Mass Subscribe", hit "Submit" and you're there. The patch changes this to "one-click," ie, advertises the Mail Archive as part of the ordinary configuration process.

But *only* for lists on a server whose listmaster has explicitly:

A) patched mailman to add this feature or B) toggled the setting in a mailman install to turn this feature on.

so it's the *listmaster* who has to do something to make this available for the server users in the ordinary configuration process.

This is very different from patching the docs to explain that the Mail Archive is an alternative / supplement to pipermail, MHonArc, or Gmane.

Mailman is what the listmaster makes of it, isn't it? The listmaster makes various decisions about how to patch and install mailman on their particular server. Once installed, it's not "our mailman" it is "their mailman" with the features and customization the listmaster selects. A patch that defaults to "off" unless the listmaster does something explicit to turn it on (by downloading a separate patch or by specifically changing the setting to "on") is not an endorsement, IMHO.
It's just one more feature that allows the listmaster to easily customize mailman for their particular user needs.

...

From a technical standpoint, I'm all for it (subject to the "defaults to OFF" policy). Archiving is a sore point with all the MLMs; giving admins another easy-to-install option is a plus. But from a support and PR standpoint, I think it does constitute an endorsement.

Would it help if the patch documentation elaborated on this concern?
Example:

This patch is provided for those who wish to enable their list owners to have a one-click option to have their lists archived with The Mail Archive. This patch is not an endorsement for The Mail Archive, each mailman server installer needs to make their own determination if this feature is desired on their server or not.

jc

Brad>     Lars was nice enough to comply with our request to
Brad> remove our lists from gmane, but these kinds of operations
Brad> should not be done without a positive and explicit approval
Brad> from the listmaster.

Any subscriber might be keeping and publishing an archive of the list posts. If the listmaster doesn't like that, he should be vetting each subscription, and making sure that each subscriber understands the rules.

If the listmaster has a position, it ought to be pusblished in the
list rules, and the users should abide by that.

problem is, users generally don't read the rules, but if it's not
published, the owner has only himself to blame.

[Otherwise ... under copyright law, the listmaster has no interest in the posts.

That's arguable at best.

the 3rd party archiver.] I don't see why Gmane or the Mail Archive should have to obey special rules here.

because it's the list owners list, and they have the final say on
how its run. Not the users. If the users don't like it, they can go
start their own lists, not attempt to co-opt policy on someone else's
list.

The second is that this patch evidently constitutes a significant endorsement of The Mail Archive.

no more than adding in a HOWTO on running with postfix is an
endosement of postfix.

... if Mailman is going to endorse services that way. I don't really think it's a good idea in principle, though. What happens if The Mail Archive goes away or goes proprietary?

then we disable the patch. No biggie. As long as it's not mandatory,
I don't see a problem here.

I would see it that way, in the "Archiving" section of admin :

  Archiving on a foreign system :
  -------------------------------

    prevent archiving on foreign systems   yes []  no [x]
            (adds a X-NO-Archive: header)

    if no,
    send all mails to archive on the following address(es):
            [ ................ ]
            e.g. archiver@mail-archive.org, thingy@gmane.plop, and so on
            see http://mail.org/foreign_archives/ for more information

With this (and proper wording), I think you get the best of the functionality without the "political" problems.

-- Fil

The sequence of "prevent" "no" "if no" can be confusing.

You are not wrong! Indeed! I can't disagree :)

-- Fil

Barry Warsaw wrote:

On Tue, 2005-05-03 at 04:45, Fil wrote:

...

I would see it that way, in the "Archiving" section of admin :

 Archiving on a foreign system :
 -------------------------------

   prevent archiving on foreign systems   yes []  no [x]
           (adds a X-NO-Archive: header)

You'd have to make clear that this would just be a hint to the foreign system, of course.

Good point. It would probably be simpler and clearer to just say:

        Add "X-NO-Archive: Yes" header     yes [] no [x] 

Do we need to explain what this header does (and doesn't) do, or could we assume that if the list owner doesn't know that they can google to find out?

jc

...

...

...

...

"JC" == JC Dill <lists05@equinephotoart.com> writes:

JC>             Add "X-NO-Archive: Yes" header yes [] no [x]

JC> Do we need to explain what this header does (and doesn't) do,
JC> or could we assume that if the list owner doesn't know that
JC> they can google to find out?

It needs explanation. It's arguable that Mailman shouldn't add that header AFAIK, for one. So Mailman's rationale and caveats need to be presented.

-- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

At 5:37 PM +0900 2005-05-03, Stephen J. Turnbull wrote:

Sure, they _are_ different, in a relevant way---they exist to broaden distribution, including archiving. But I think that in the great majority of cases where random users can just sign up, that is a service to be encouraged. It's not a good idea to put the burden of proof on them, when either the list-owner can be more selective about membership, or they can use X-No-Archive.

The problem here is that Mailman should not be adding an 

"X-No-Archive:" header to messages that it is processing. This is something that should be controlled from the perspective of the user, and Mailman should not be stepping on their toes.

Moreover, if Mailman did add such a header, what would happen to 

the internal archiving system? Would Mailman ignore the header that it has added while honoring the same header that might have been put on the message by the user?

As a list admin, I can see a strong desire to keep your own 

archive, but to want to prevent anyone else from maintaining an archive, at least not without your express approval. Unless, that is, you gateway to USENET news, in which case Google and others have news archives that you could not control and could not even be aware of in most cases. But then if you gateway to USENET news, you should be aware of this issue, and be prepared for what might happen.

Again, I'm not really arguing against the patch. It's the people who might be doing extra releases (Barry and Tokio, right?) or answering the FAQs (Brad and Mark, primus inter pares) who should decide if it belongs in the Mailman distribution.

IMO, the ultimate decision is up to Barry -- it's his project, 

and he gets to decide how things go. However, he is currently focusing on Mailman3 right now, and Tokio is the release engineer for Mailman2, and in the past Barry has been open to comments and suggestions from others. So, I imagine he might make his feelings known, but then leave the ultimate decision to Tokio, who would hopefully also take input from others.

However, I don't see that Mark or I would necessarily have any 

more weight given to our comments during that discussion as a result of our work with the FAQ and answering the questions. I would hope that we would be heard along with the others commenting on the subject, and appropriate weight would be given to them by Barry and Tokio, but more based on their merits than on the work we do with the FAQ.

There are plenty of other knowledgeable people on mailman-users 

and mailman-developers who don't (or haven't recently) done much of anything with the FAQ, and I would hope that their voices would be given as much weight relative to their experience as would ours.

I do advocate some kind of public statement about the policy toward adding new facilities of this kind. One easy one would be "you write the patch, and show that you conform to certain rules such as 'patch defaults off' and 'service respects X-No-Archive as well as conforming to relevant RFCs', and we'll put it in to the next regular release that isn't already in feature freeze."

I'm not so sure this is a good idea.  At least, not so far as 

guaranteeing that it would be put in the next regular release. IMO, if the patch defaults to off, and the service conforms to the relevant RFCs and BCPs, then I think we should give it serious consideration, but I wouldn't want to guarantee anything more than serious consideration.

Or maybe it's worth encouraging such services, and being more helpful about it.

I would encourage more people to make patches, and to try to be 

more helpful about this process in general. But I wouldn't want to make any guarantees as to what would/would not get included -- everything should get the appropriate level of consideration, but no guarantees beyond that.

-- Brad Knowles, <brad@stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."

 -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
 Assembly to the Governor, November 11, 1755

SAGE member since 1995. See <http://www.sage.org/> for more info.

At 7:17 PM +0900 2005-05-06, Stephen J. Turnbull wrote:

Maybe you could make it part of the user's configuration. Then the list master could default it to X-No-Archive: yes; individual users could turn it to no if they want to, including on a message by message basis. A stretch, I know, but it would be really horrible to have to have separate archiving control headers for the user and the list, and to have to establish a precedence, etc.

I think the concept of having Mailman optionally add it's own 

"X-No-Archive:" header would greatly complicate the situation, and I would be opposed to spending a whole lot of time working on this unless someone else can provide us a pretty much complete feature-rich patch that allows for modifying where in the stream this process is done and where in the stream Mailman pays attention to this header itself, and on a per-user basis.

Even then, I would be opposed to spending a whole lot of time 

contemplating the patch.

The original patch is simple enough that I think we can argue the 

merits of trying to slip it in sooner rather than later. However, the more complex you make this concept and the more complex you make the patch, the more I would be unwilling to consider incorporating such a feature until the next major release of Mailman2.

I'm happy with your phrasing of "serious consideration." Nothing's guaranteed in a volunteer maintained project, of course.

Of course.

-- Brad Knowles, <brad@stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."

 -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
 Assembly to the Governor, November 11, 1755

SAGE member since 1995. See <http://www.sage.org/> for more info.

On Tue, 2005-05-03 at 04:37, Stephen J. Turnbull wrote:

Hrm. Maybe a way to turn on X-No-Archive should be part of this patch? (Or does Mailman offer that already? I don't see it.)

Not on the Mailman side.

Good trick, that. Mailman is free software; it will be months before the revised code propagates to all the major distros

Or years. It's bad enough I still get bug reports about Mailman 2.0, but there are sites still running <shudder> Mailman 1.x.

, let alone the small-time variants. How long would it take Apple to get a Software Update out? During that period, Mailman is offering a service that it can't support. Nor do I think Mailman's reaction would be particularly swift---isn't this the kind of thing that as a technical matter really should wait until the next scheduled release?

Again, I'm not really arguing against the patch. It's the people who might be doing extra releases (Barry and Tokio, right?) or answering the FAQs (Brad and Mark, primus inter pares) who should decide if it belongs in the Mailman distribution.

I do advocate some kind of public statement about the policy toward adding new facilities of this kind. One easy one would be "you write the patch, and show that you conform to certain rules such as 'patch defaults off' and 'service respects X-No-Archive as well as conforming to relevant RFCs', and we'll put it in to the next regular release that isn't already in feature freeze."

Or maybe it's worth encouraging such services, and being more helpful about it.

I think I've stated my general philosophy in a previous message. If WizzyMTA came with a new plug-in, documentation, and some promise of support help (if only to answer questions on mailman-users), we'd probably add the module in the next release. If we had a similar plug-in architecture for multiple 3rd-party archivers, we'd facilitate the same kind of thing for that service. I'd be all for that.

-Barry

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

... if Mailman is going to endorse services that way. I don't really think it's a good idea in principle, though. What happens if The Mail Archive goes away or goes proprietary? What are people going to think if The Mail Archive's maintainers hire Barry or Brad? Etc, etc.

(Disclosure: I'm the other archiving service, Gmane.)

I wonder if adding a layer of indirection here would help with these considerations.

That is -- Mailman could have a box saying "archive using external archivers" (without saying which external archivers) that list admins could tick/untick that resulted in a message being sent to a meta-archival site. This meta-archival site would then have a list of external archivers that be notified about the mailing list.

This meta-archive would be run by a third party, and would de-list any archivers that are naughty.

-- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen

On Tue, 2005-05-03 at 06:47, Lars Magne Ingebrigtsen wrote:

I wonder if adding a layer of indirection here would help with these considerations.

That is -- Mailman could have a box saying "archive using external archivers" (without saying which external archivers) that list admins could tick/untick that resulted in a message being sent to a meta-archival site. This meta-archival site would then have a list of external archivers that be notified about the mailing list.

This meta-archive would be run by a third party, and would de-list any archivers that are naughty.

My idea of having a framework in Mailman wouldn't preclude this, and it would give list owners the option of picking a specific archiver (because of compliance with whatever policies they care about), or just let the meta-archiver choose.

-Barry

what I did with my system was make the archiver completely separate
from Mailman -- to mailman , it's just another email address that the
admin subscribes, or not. Does archiving need to be integrated into
mailman in the first place? As opposed to, say, a place to specifiy
the URL of the archives for display?

and since we've never really had the time/energy/etc to maintain and
enhance pipermail, maybe spliting the code out and retiring it is a
good strategy.

On May 12, 2005, at 4:15 PM, Barry Warsaw wrote:

Let's say that's an important requirement. What I'd like to see in
that case is a way to specify multiple archivers, register these archivers with Mailman, and then give list owners the ability to select one or more of those registered archive methods. You could even
generalize it so that the internal Pipermail archiver was just another one of those methods.

...

...

...

...

"BAW" == Barry Warsaw <barry@python.org> writes:

BAW> On Thu, 2005-05-12 at 19:48, Chuq Von Rospach wrote:

>> Does archiving need to be integrated into mailman in the first
>> place? As opposed to, say, a place to specifiy the URL of the
>> archives for display?

BAW> The key is that you have to know what value to insert into
BAW> List-Archive.

But you need that anyway---unless you use pipermail, Mailman can't know where your archives are until you tell it. So that doesn't argue against changing pipermail's interface to generic archive-by-mail, Mailman can still know where pipermail will put stuff.

I see two problems with archive-by-mail, aside from "don't fix pipermail, which ain't broke". The first is that you have to be careful to set up that archive mail address to be secure, otherwise it's a back door for spam to get into your archive. This is going to be, uh, subtle for most of the users <coff>at least, I got bit</coff>. Even if we try to document it carefully I think this is FAQ-a-minute.

The second was suggested by the "who owns X-No-Archive, anyway?" issue. If archiving is done by a special handler, it can have special logic to (attempt to) inhibit 3rd party archivers, it can respect X-No-Archive itself, and you can move it around in the pipeline if you need special effects. All of that is hard to do if it's hiding among the general memebership.

NB, if the Archive handler supports the mail-to-archive protocol, then you don't have to worry about having an International Archiver Protocol with a World Archiver Organization to embargo archives who forget to obey X-No-Archive ... Mailman handles that.

-- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

...

• New third party archiving option that uses The Mail Archive. The implementation subscribes or unsubscribes the archive@mail-archive.com address from the subscriber list.

I find this whole discussion fascinating. I've been thinking about these types of possibilities for a while, and have been watching gmane and mail-archive off and on for inspiration for what Kabissa might do. Archiving is getting to be a big 'problem' at Kabissa currently, esp as our archives get older and bigger, and we desire to move to a more interactive and useful format for archiving discussions and making them more accessible via the web, i.e. in forums. Maybe the short-term answer is to encourage the listowners of our very busy lists to move their archives to mail-archive or gmane.

In my view, this sort of functionality belongs at the other end - the site that wants to provide the archiving service. Over the years I've been toying with the idea of developing a tool hosted at Kabissa that would enable our members (organizations working in Africa) to submit lists that they want to archive and make accessible via Kabissa... both lists hosted on Kabissa's own mailman and those hosted elsewhere. This would be beneficial because the list may not be well archived elsewhere (chronic problem with pipermail archives), may otherwise only be archived at Yahoogroups (advertising driven and not easy to export archive to move it later - also don't include attachments), or may not be archived at all. Part of the process of submitting the list for archiving on Kabissa (providing name, description, home page, subscribe instructions etc) would optionally include subscribing an email address at Kabissa to the list itself.

The described patch to Mailman is very interesting, though, and I'm glad it's been done. I haven't tried the patch, but from what I'm hearing the issue of it appearing to endorse one or other archives can be dealt with by making the feature customizable - i.e. have a control line where mailman admins can configure whatever external archiver they want, with completely configurable fields: the name/description of the archiver and the email address to automatically add. The mail-archive and gmane options would be included as examples but commented out.

Without opening a can of worms, hopefully, let me close with one last thought. I realize this has probably been discussed ad nauseum in other places, but there's a bit of a flaw in all this. Email addresses can be faked, and so an archiver based on email is going to be fraught with problems - or at least a whole lot of spam on the archives once the spammers figure out how it works. On our own system, we use Mailman's own archiver subsystem for gatewaying messages to our Fud Forum (http://www.fudforum.org). Another way I've tried successfully is through the use of an email address made up of random characters that gets delivered to Fud. That works fine, and since the list is on Kabissa (managed by me) and the Fud is on Kabissa, the likelihood of spammers getting in by spoofing addresses is pretty low.

Anyway, contact me off list if you want to discuss this further, esp if you can point me to resources/threads where this issue is dealt with to your satisfaction - it will help me to convince developers of various forums that it's worth opening the door to e-mail list/forum gatewaying. Here's a classic example of a thread on the Simple Machines Forum board: http://www.simplemachines.org/community/index.php?topic=10166.0

Cheers,

Tobias

-- Tobias Eigen Executive Director

Kabissa - Space for Change in Africa http://www.kabissa.org

• Kabissa's vision is for a socially, economically, politically, and environmentally vibrant Africa, supported by a strong network of effective civil society organizations. *

Tobias Eigen wrote:

...

...

• New third party archiving option that uses The Mail Archive. The implementation subscribes or unsubscribes the archive@mail-archive.com address from the subscriber list.

I find this whole discussion fascinating. I've been thinking about these types of possibilities for a while, and have been watching gmane and mail-archive off and on for inspiration for what Kabissa might do. Archiving is getting to be a big 'problem' at Kabissa currently, esp as our archives get older and bigger, and we desire to move to a more interactive and useful format for archiving discussions and making them more accessible via the web, i.e. in forums. Maybe the short-term answer is to encourage the listowners of our very busy lists to move their archives to mail-archive or gmane.

This looks like a good opportunity to mention that mailman-users and mailman-developers are now also archived at:

<http://archives.free.net.ph/splash/index.en.html>

See this particular thread at:

<http://archives.free.net.ph/message/20050429.220927.207b9a02.en.html>

They asked for permission to join -dev so that it could be archived on their site. At the time, their archives didn't have munged email addresses, and Barry and I decided that if they fixed their archives to not expose email addresses, he would give them permission to join -dev and to archive mailman lists..

I like gmane's feature of making lists accessible in a newsgroup fashion, and I like free.net's format of providing the list in a threaded fashion via the web. This is not an endorsement of either site, just a personal observation of some of the positive features each of them offers over traditional list archives. Some list admins might want to offer their lists to be mirrored/archived at more than one site.

While RFC 2369 only allows for one List-Archive: URL, I wonder if email client implementations that understand 2369 (is there a list of which ones are?) are generous in their response when a list email has more than List-Archive header... Be generous in what you accept and all that... :-)

jc