Mail-to-news gatewaying?
Folks,
Okay, so we know about the issue of munging Message-ID: (if you don't know what I'm talking about, see <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.059.htp>).
The latest crisis that has me wondering is the subject prefix stuff. Looking in NewsRunner.py, starting at line 104, we have:
# Should we restore the original, non-prefixed subject for gatewayed
# messages?
origsubj = msgdata.get('origsubj')
if not mlist.news_prefix_subject_too and origsubj is not None:
del msg['subject']
msg['subject'] = origsubj
However, we still have a problem. If mail clients reply to the message to the list (which will include the subject prefix), then this will result in a new message with a subject prefix for the list already present. Now, Mailman is smart and avoids adding a second subject prefix, but it doesn't strip out the subject prefix if one already exists. For the mailing list, this subject prefix stripping isn't necessary. But for gatewaying to the newsgroup, it might be desirable.
This would also resolve the issue of restoring the original subject when gatewaying -- don't bother, and just let the subject prefix stripper deal with it.
If this were sed or Perl, I could quickly put together a one-line hack to implement this feature. For example, just doing something off the top of my head, you might be able to use something like:
subject=y/\w*$prefix\w*/ /;
But this is Python, and I don't really grok Python. Can someone help me out here?
Looking at <http://www.python.org/doc/2.3.4/lib/module-string.html>, I'm imagining some sort of permutation of find(), split(), either lstrip() or rstrip(), and then join(), should be sufficient to do the job. But I'm not quite able to figure out how to put them all together.
Of course, the reason why I ask is that I'm getting complaints about the mail-to-news gateway I'm running at another site, and I don't want this to blow up into an ugly situation.
Sigh....-- Brad Knowles, <brad@stop.mail-abuse.org>
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See <http://www.sage.org/> for more info.
Hi,
Brad Knowles wrote:
The latest crisis that has me wondering is the subject prefixstuff. Looking in NewsRunner.py, starting at line 104, we have:
# Should we restore the original, non-prefixed subject for gatewayed # messages? origsubj = msgdata.get('origsubj') if not mlist.news_prefix_subject_too and origsubj is not None: del msg['subject'] msg['subject'] = origsubj However, we still have a problem. If mail clients reply to themessage to the list (which will include the subject prefix), then this will result in a new message with a subject prefix for the list already present. Now, Mailman is smart and avoids adding a second subject prefix, but it doesn't strip out the subject prefix if one already exists. For the mailing list, this subject prefix stripping isn't necessary. But for gatewaying to the newsgroup, it might be desirable.
Sorry but I have no newsgroup forwarding environment here. Our system administrator (myself) determined long ago that newsgroups (usenet and fj) have extremely low S/N ratio and waste of storage and bandwidth. ;-)
This would also resolve the issue of restoring the original subjectwhen gatewaying -- don't bother, and just let the subject prefix stripper deal with it.
If this were sed or Perl, I could quickly put together a one-linehack to implement this feature. For example, just doing something off the top of my head, you might be able to use something like:
subject=y/\w*$prefix\w*/ /;
Things are not easy because there are MIME subjects and we also start numbering prefix in mailman 2.1.6. May be we can calculate 'stripped_subject' in CookHeaders.py and store in msgdata for later use. Or, strip in a magical ;-) way which I use in CookHeaders.py and in HyperArch.py, in NewsRunner.py again.
Anyway, I will try.
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
Bob Puff@NLE wrote:
Tokio Kikuchi wrote:
Things are not easy because there are MIME subjects and we also start numbering prefix in mailman 2.1.6.
Why are we doing this, again?
It's optional and the default is not to.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Right, but why are we providing the option in the first place? To try to show threading? Seems that it would cause more grief like that which Brad was alluding to earlier.
I'm just wondering if we're losing a good fix due to an incompatibility with a feature that might not be used.
Bob
Mark Sapiro wrote:
Bob Puff@NLE wrote:
Tokio Kikuchi wrote:
Things are not easy because there are MIME subjects and we also start numbering prefix in mailman 2.1.6.
Why are we doing this, again?
It's optional and the default is not to.
-- Mark Sapiro <msapiro@value.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/bob%40nleaudio.com
At 2:25 PM -0500 2005-02-08, Bob Puff@NLE wrote:
Right, but why are we providing the option in the first place? To try to show threading? Seems that it would cause more grief like that which Brad was alluding to earlier.
I'm not 100% certain, but I don't think that the subject numbering issue has any direct relation to the mail/news gateway process. I think it's intended for use in mail-only environments, perhaps such as an official support mailing list where things need to be tracked by ticket number, etc.... In this case, the subject numbering would presumably become the ticket number.
I'm just wondering if we're losing a good fix due to an incompatibility with a feature that might not be used.
So long as there's an optional way to cause that to be pulled back out when the message is gatewayed to a newsgroup, and the overall feature defaults to off, I don't see that there's much of a problem here. But it certainly can make life in other areas more complex.
-- Brad Knowles, <brad@stop.mail-abuse.org>
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See <http://www.sage.org/> for more info.
At 12:11 PM +0900 2005-02-08, Tokio Kikuchi wrote:
Things are not easy because there are MIME subjects
There's that damn pesky MIME problem again. ;)and we also startnumbering prefix in mailman 2.1.6.
Ouch! I had forgotten about that.May be we can calculate'stripped_subject' in CookHeaders.py and store in msgdata for later use. Or, strip in a magical ;-) way which I use in CookHeaders.py and in HyperArch.py, in NewsRunner.py again.
Hmm. Maybe we can have a library routine that all programs could use?Anyway, I will try.
Much appreciated. Thanks!-- Brad Knowles, <brad@stop.mail-abuse.org>
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See <http://www.sage.org/> for more info.
Hi Brad,
While we are at security bug fix. I also worked on this and checked in the CVS/2.1.6b3. I could only test it by logging the news message because I don't have inn server around. I hope you can test it by yourself.
Brad Knowles wrote:
At 12:11 PM +0900 2005-02-08, Tokio Kikuchi wrote:
Things are not easy because there are MIME subjects
There's that damn pesky MIME problem again. ;)and we also startnumbering prefix in mailman 2.1.6.
Ouch! I had forgotten about that.May be we can calculate'stripped_subject' in CookHeaders.py and store in msgdata for later use. Or, strip in a magical ;-) way which I use in CookHeaders.py and in HyperArch.py, in NewsRunner.py again.
Hmm. Maybe we can have a library routine that all programs could use?Anyway, I will try.
Much appreciated. Thanks!
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
At 2:30 AM +0100 2005-02-08, Brad Knowles wrote:
The latest crisis that has me wondering is the subject prefix stuff. Looking in NewsRunner.py, starting at line 104, we have:
# Should we restore the original, non-prefixed subject for gatewayed # messages? origsubj = msgdata.get('origsubj') if not mlist.news_prefix_subject_too and origsubj is not None: del msg['subject'] msg['subject'] = origsubj
Okay, I broke down and bought the three primary Python books -- Learning, Programming, and Nutshell. Looking at the string functions, and not accounting for MIME encoding, Unicode encoding, or subject numbering, to my naïeve perspective, it looks like we should be able to turn this into something like the following:
# Should we restore the original, non-prefixed subject for gatewayed
# messages?
origsubj = msgdata.get('origsubj')
if not mlist.news_prefix_subject_too and origsubj is not None:
stripped_subj = subject.replace(r'\s*'+subject_prefix+r'\s*', ' ')
del msg['subject']
msg['subject'] = stripped_subj
The s.replace() function should grab any leading whitespace and trailing whitespace surrounding the subject_prefix, and replace all of that with a single space character.
Now, to go off and look at CookHeaders.py and see just how far off I am....-- Brad Knowles, <brad@stop.mail-abuse.org>
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See <http://www.sage.org/> for more info.
participants (5)
-
Bob Puff＠NLE -
Brad Knowles -
Mark Sapiro -
Thomas Hochstein -
Tokio Kikuchi