Hi list,
please let me start by presenting myself.
My name is Bernd "Siggy" Brentrup and I'm a longtime *nix user/programmer/admin, from '95 thru '04 I have been a DD, shortly before I vanished from Debian I was part of the 2 and 1/2 men show maintaining Debian's mailman package; during that time I wrote most of the debconf stuff and contributed postfix-to-mailman.py modelled after Bruce Perens's qmail-to-mailman.py.
Even then, one of my biggest concerns with mailman was that the archives are not searchable, I just can't recall whether I published this in '04.
Now trying to reenter the OSS community I observe that archives on mailman lists still aren't searchable.
Barry told me pipermail didn't see much love during the last years and some revamp is required.
In contrast to his suggestions (sth involving google search) I'm aiming at a solution akin to Debian's Xapian powered ML search.
cf http://lists.debian.org/cgi-bin/search
What do you think and is anybody else interested in working on this?
Thanks for listening Siggy
-- bsb-at-psycho-dot-informationsanarchistik-dot-de or: bsb-at-psycho-dot-i21k-dot-de O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Siggy> In contrast to his suggestions (sth involving google search) I'm
Siggy> aiming at a solution akin to Debian's Xapian powered ML search.I've never been all that pleased with Xapian search results. I'm one of those lazy people who feels that Google does a better job than I ever could.
-- Skip Montanaro - skip@pobox.com - http://www.smontanaro.net/ Getting old sucks, but it beats dying young
On Aug 7, 2009, at 7:11 AM, skip@pobox.com wrote:
Siggy> In contrast to his suggestions (sth involving google
search) I'm Siggy> aiming at a solution akin to Debian's Xapian powered ML
search.I've never been all that pleased with Xapian search results. I'm
one of those lazy people who feels that Google does a better job than I
ever could.
Unfortunately, that doesn't work for private mailing lists.
-Barry
Siggy Brentrup wrote:
In contrast to his suggestions (sth involving google search) I'm aiming at a solution akin to Debian's Xapian powered ML search.
cf http://lists.debian.org/cgi-bin/search
What do you think and is anybody else interested in working on this?
There are patches at <http://www.openinfo.co.uk/mm/index.html> that integrate the ht://Dig search engine with Mailman's archives. They work well.
The problem with integrating these or some of the other solutions into the source is they require installation of software over which we have no control.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, Aug 07, 2009 at 13:40 -0700, Mark Sapiro wrote:
Siggy Brentrup wrote:
In contrast to his suggestions (sth involving google search) I'm aiming at a solution akin to Debian's Xapian powered ML search.
cf http://lists.debian.org/cgi-bin/search
What do you think and is anybody else interested in working on this?
There are patches at <http://www.openinfo.co.uk/mm/index.html> that integrate the ht://Dig search engine with Mailman's archives. They work well.
The problem with integrating these or some of the other solutions into the source is they require installation of software over which we have no control.
From a Debian point of view this isn't much of a problem, I might create a patched mailman-searchable-archives package that depends on htdig, eventually factoring out common parts to yield:
mailman mailman-searchable-archives
\ /
mailman-common htdig
but it doesn't address concerns about pipermail. MM 3 seems to me to be an opportunity to address all these issues, we might learn from ht://Dig or Xapian (both written in perl and C iirc) to implement an enhanced pipermail.
These are only first thoughts, before deciding which way to go a lot of research is still necessary and I'm open to any suggestions.
Regards Siggy
bsb-at-psycho-dot-informationsanarchistik-dot-de
or: bsb-at-psycho-dot-i21k-dot-deO< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Bernd 'Siggy' Brentrup writes:
learn from ht://Dig or Xapian (both written in perl and C iirc) to implement an enhanced pipermail.
Xapian is written in C++ (I'm pretty sure about that) and has Python bindings. Python bindings for sure: it's used in Roundup (though like much in Roundup I can't recommend it as exemplary implementation).
On Aug 7, 2009, at 4:40 PM, Mark Sapiro wrote:
The problem with integrating these or some of the other solutions into the source is they require installation of software over which we have no control.
Agreed, which has always been my argument for including Pipermail in
the Mailman distribution. At least it gives people an archiver out of
the box. OTOH, it really makes more sense to make Pipermail a
separate project and let it live or die on its own. I'm not sure what
the right thing to do is, but it sure would help if someone adopted
Pipermail <wink>.
-Barry
On Fri, Aug 07, 2009 at 19:27 -0400, Barry Warsaw wrote:
On Aug 7, 2009, at 4:40 PM, Mark Sapiro wrote:
The problem with integrating these or some of the other solutions into the source is they require installation of software over which we have no control.
Agreed, which has always been my argument for including Pipermail in the Mailman distribution. At least it gives people an archiver out of the box. OTOH, it really makes more sense to make Pipermail a separate project and let it live or die on its own. I'm not sure what the right thing to do is, but it sure would help if someone adopted Pipermail <wink>.
<duck> :) No need for this broad hint.
I was thinking about this option too and as soon as the issues with the old account Ubuntu automatically created for me when importing my Debian packages are solved I will create a pipermail project on launchpad and announce it here.
Siggy
p.s. Please respect my .signature, I'd really like to reenable duplicate elimination.
Please don't Cc: me when replying, I might not see either copy. bsb-at-psycho-dot-informationsanarchistik-dot-de or: bsb-at-psycho-dot-i21k-dot-de O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Aug 7, 2009, at 9:03 PM, Bernd Siggy Brentrup wrote:
<duck> :) No need for this broad hint.
Tim Peters would be proud at the power of the <wink>!
I was thinking about this option too and as soon as the issues with the old account Ubuntu automatically created for me when importing my Debian packages are solved I will create a pipermail project on launchpad and announce it here.
That would be cool. I wonder if we should make ~mailman-coders the
owner of that project? If not, a different but parallel team
structure might make sense. Note too that since Pipermail comes from
Mailman, it's GNU so we need to get copyright assignments for all
Pipermail hackers.
I will volunteer to help separate the code, and to work out how best
to integrate it for packaging.
p.s. Please respect my .signature, I'd really like to reenable duplicate elimination.
You can do this for Mailman lists, at least when you are explicitly
CC'd with an address that's subscribed to the list. You'll get the
direct copy and not the list copy though. That's the best Mailman can
do (anything more takes a better MUA ;).
-Barry
On Fri, Aug 07, 2009 at 22:02 -0400, Barry Warsaw wrote:
On Aug 7, 2009, at 9:03 PM, Bernd Siggy Brentrup wrote:
<duck> :) No need for this broad hint.
Tim Peters would be proud at the power of the <wink>!
I was thinking about this option too and as soon as the issues with the old account Ubuntu automatically created for me when importing my Debian packages are solved I will create a pipermail project on launchpad and announce it here.
That would be cool. I wonder if we should make ~mailman-coders the owner of that project?
No objections from my side if you grant me admin rights. As a surplus we don't have to wait for lp ppl to merge my accounts; there are some restrictions that inhibit merging e.g. creating a PPA, after all I want to be known on lp as ~bsb and not as ~bsb-psycho :-P
If not, a different but parallel teamstructure might make sense. Note too that since Pipermail comes from Mailman, it's GNU so we need to get copyright assignments for all Pipermail hackers.
ack
I will volunteer to help separate the code, and to work out how best to integrate it for packaging.
Thanks for this, did you write down any issues that should be addressed when revamping pipermail? One thing that comes to mind is how to deal with mime types and attachments. I'd also like to see signatures validated. (On an aside, I hate these footers that make my messages not fully signed)
p.s. Please respect my .signature, I'd really like to reenable duplicate elimination.
You can do this for Mailman lists, at least when you are explicitly CC'd with an address that's subscribed to the list. You'll get the direct copy and not the list copy though. That's the best Mailman can do (anything more takes a better MUA ;).
I know that's because I mentioned it, I didn't recall the variable name though. In the first place I don't want to see 2 copies arriving, no MUA involved here.
Siggy (birds are greeting the day, time to go to bed :)
Please don't Cc: me when replying, I might not see either copy. bsb-at-psycho-dot-informationsanarchistik-dot-de or: bsb-at-psycho-dot-i21k-dot-de O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Bernd Siggy Brentrup writes:
[Barry Warsaw contributed the comment:]
You can do this for Mailman lists, at least when you are explicitly CC'd with an address that's subscribed to the list. You'll get the direct copy and not the list copy though. That's the best Mailman can do (anything more takes a better MUA ;).
They don't come *better* than mutt, though. At best, "different".
I know that's because I mentioned it, I didn't recall the variable name though. In the first place I don't want to see 2 copies arriving, no MUA involved here.
I suggest "/etc/init.d/$MTA stop" is the only way to guarantee that two copies of something will never arrive. The next best is to set Reply-To, which might actually have the desired effect most of the time. The next best is to set Mail-Followup-To, which won't have any effect but you will get a lot of sympathy from Dan Bernstein and Emacs/Gnus users.
P.S. You owe me $250/hr X 5 seconds / (3600 seconds/hour) = $0.35 for time & expense of fixing up the headers in this message. You pay the wire transfer and currency conversion costs.<wink>
On Sat, Aug 08, 2009 at 15:43 +0900, Stephen J. Turnbull wrote:
Bernd Siggy Brentrup writes:
[Barry Warsaw contributed the comment:]
You can do this for Mailman lists, at least when you are explicitly CC'd with an address that's subscribed to the list. You'll get the direct copy and not the list copy though. That's the best Mailman can do (anything more takes a better MUA ;).
They don't come *better* than mutt, though. At best, "different".
Oh no, no yamuad (Yet Another MUA Discussion) please.
I know that's because I mentioned it, I didn't recall the variable name though. In the first place I don't want to see 2 copies arriving, no MUA involved here.
I suggest "/etc/init.d/$MTA stop" is the only way to guarantee that two copies of something will never arrive. The next best is to set Reply-To, which might actually have the desired effect most of the time. The next best is to set Mail-Followup-To, which won't have any effect but you will get a lot of sympathy from Dan Bernstein and Emacs/Gnus users.
I'm subscribed to some MLs that mangle with Reply-To to keep subscribers with broken MUAs happy, mutt does the Mail-Followup-To for me w/o my intervention.
Do you know anybody who wants to see djb happy?
P.S. You owe me $250/hr X 5 seconds / (3600 seconds/hour) = $0.35 for time & expense of fixing up the headers in this message. You pay the wire transfer and currency conversion costs.<wink>
You actually read those winnegan.fake (my version of localnet) headers? Nobody does this! As kind of a reward IOU a beer should we ever meet.
Regards Siggy
Please don't Cc: me when replying, I might not see either copy. bsb-at-psycho-dot-informationsanarchistik-dot-de or: bsb-at-psycho-dot-i21k-dot-de O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Bernd Siggy Brentrup writes:
On Sat, Aug 08, 2009 at 15:43 +0900, Stephen J. Turnbull wrote:
They don't come *better* than mutt, though. At best, "different".
Oh no, no yamuad (Yet Another MUA Discussion) please.
Oh, fer shure. My point was to remind Barry that there are some things you can't ask of any MUA, 'cause none of them do it (well).<wink>
I'm subscribed to some MLs that mangle with Reply-To to keep subscribers with broken MUAs happy, mutt does the Mail-Followup-To for me w/o my intervention.
Indeed, that was quite my point. You just can't win on "no dupes, please," except by accepting them and handling them yourself.
You actually read those winnegan.fake (my version of localnet) headers? Nobody does this!
*chuckle* Dirty job, but somebody's gotta do it.
On Aug 8, 2009, at 9:33 AM, Stephen J. Turnbull wrote:
Oh, fer shure. My point was to remind Barry that there are some things you can't ask of any MUA, 'cause none of them do it
(well).<wink>
Writing the Best MUA In The World is on my list, right after I solve
the health care issue. :) But yep, it's just a specification of
Warsaw's Third Law.
-Barry
On Aug 7, 2009, at 10:49 PM, Bernd Siggy Brentrup wrote:
On Fri, Aug 07, 2009 at 22:02 -0400, Barry Warsaw wrote:
On Aug 7, 2009, at 9:03 PM, Bernd Siggy Brentrup wrote:
<duck> :) No need for this broad hint.
Tim Peters would be proud at the power of the <wink>!
I was thinking about this option too and as soon as the issues with the old account Ubuntu automatically created for me when importing my Debian packages are solved I will create a pipermail project on launchpad and announce it here.
That would be cool. I wonder if we should make ~mailman-coders the owner of that project?
No objections from my side if you grant me admin rights. As a surplus we don't have to wait for lp ppl to merge my accounts; there are some restrictions that inhibit merging e.g. creating a PPA, after all I
want to be known on lp as ~bsb and not as ~bsb-psycho :-P
I think right now I don't want to open up admin rights on the mailman-
coders team. If that means we should create a pipermail-coders team
for the pipermail project, that would be cool with me.
I will volunteer to help separate the code, and to work out how best to integrate it for packaging.
Thanks for this, did you write down any issues that should be addressed when revamping pipermail? One thing that comes to mind is how to deal with mime types and attachments. I'd also like to see signatures validated. (On an aside, I hate these footers that make my messages not fully signed)
Here are the top priorities for me:
- Stable and predictable URLs. We must guarantee that if the HTML
archive were blown away and regenerated from the raw messages, that
the urls to those messages will not change. URLs must also be
predictable given only the information in the original message.
Mailman must be able to calculate a URL to the message in the archive
without communicating to the archiver. See this wiki page for details
(already implemented on the Mailman side in MM3)
http://wiki.list.org/display/DEV/Stable+URLs
Move away from mbox format for raw messages. (maildir?)
Use a modern backend for persistence (pickles suck)
Use templates and make the archives easily customizable
Use modern CSS, but Javascript sparingly and then with progressive
degradation.IWBNI all content were vended dynamically, with caching, so that
messages could be removed from the archive, or the style could be
changed, without a hard regen.
p.s. Please respect my .signature, I'd really like to reenable duplicate elimination.
You can do this for Mailman lists, at least when you are explicitly CC'd with an address that's subscribed to the list. You'll get the direct copy and not the list copy though. That's the best Mailman can do (anything more takes a better MUA ;).
I know that's because I mentioned it, I didn't recall the variable name though. In the first place I don't want to see 2 copies arriving, no MUA involved here.
That's because I was nice the first time and removed your name from
the recipients. As they say "the first one's free" :)
-Barry
participants (9)
-
Barry Warsaw -
Bernd 'Siggy' Brentrup -
Bernd Siggy Brentrup -
Jordan Hayes -
Mark Sapiro -
Siggy Brentrup
-
skip＠pobox.com -
Stephen J. Turnbull -
Stephen J. Turnbull