Rejecting on size before content-filters
Is it possible to change the testing-order so that the list size limit is tested [and generates an appropriate reject] *before* the content filters are processed? [I just ran into one that was nearly double the maximum size but got kicked out because it also ran afoul of one of the content filters, and so what I ended up doing was ignoring the content filter and putting in "message too long...etc" into the 'Reason' on my bounce screen]. tnx!
/Bernie\
-- Bernie Cosell Fantasy Farm Fibers mailto:bernie@fantasyfarm.com Pearisburg, VA --> Too many people, too few sheep <--
Bernie Cosell wrote:
Is it possible to change the testing-order so that the list size limit is tested [and generates an appropriate reject] *before* the content filters are processed?
This is the way it's normally done. The usual request is to do it the other way around. See, e.g., <http://mail.python.org/pipermail/mailman-users/2008-December/064541.html>.
[I just ran into one that was nearly double the maximum size but got kicked out because it also ran afoul of one of the content filters, and so what I ended up doing was ignoring the content filter and putting in "message too long...etc" into the 'Reason' on my bounce screen]. tnx!
Do you really mean "content filter" as in "Content filtering" or do you mean something else, perhaps header_filter_rules? "Content filtering" doesn't hold messages. It just removes parts, and perhaps does filter_action if it removes the entire message, but "hold' is not one of the actions.
You can rearrange the pipeline similarly to the suggestion in the above referenced post so that whatever handler was responsible for this hold comes after the Hold handler which processes the message size limit amongst others.
It is also possible that Hold miscalculated the size of the message. My first contribution to Mailman was a patch to Hold to take preambles and epilogues into account when computing message size. Pre 2.1.7, this wasn't done. There may be other problems in computing the size of possibly malformed messages.
So, why was this message held. I.e., exactly what "content filter" did it "run afoul of"?
And, if there is something strange happening, do you have a copy of the original message, perhaps in a moderator notice?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 4 Jan 2009 at 8:18, Mark Sapiro wrote:
Bernie Cosell wrote:
Is it possible to change the testing-order so that the list size limit is tested [and generates an appropriate reject] *before* the content filters are processed?
This is the way it's normally done. The usual request is to do it the other way around. See, e.g., <http://mail.python.org/pipermail/mailman-users/2008-December/064541.html>.
Do you really mean "content filter" as in "Content filtering" or do you mean something else, perhaps header_filter_rules? "Content filtering" doesn't hold messages. It just removes parts, and perhaps does filter_action if it removes the entire message, but "hold' is not one of the actions.
My bad.. Sorry I'm new here and I don't get the terminology right. The filter that tripped was in Privacy options/spam filters".
You can rearrange the pipeline similarly to the suggestion in the above referenced post so that whatever handler was responsible for this hold comes after the Hold handler which processes the message size limit amongst others.
Ah, Ok -- I see the post and I'll try to figure it out and change the order...
So, why was this message held. I.e., exactly what "content filter" did it "run afoul of"?
It was a little "spammy" by our spam filter [Barracuda]. The listmembers are *astoundingly* spam averse but [of course] I don't want to reject a lot of false positives, so what I've done is set it up so that 'vaguely spam-like' submissions are, in essence, moderated. In addition to "hold"ing on "X-Barracuda-Spam-Status: Yes" I also hold on X-Barracuda- Spam-Score: [other than zero]. This gets a LOT of false positives [a spam score of 1.2 isn't very spammy, but it *might* be spam], so I just send-em through if they're OK. BUT: if the message is too large, then I don't want to send them through, even if the message was legit. so I want the "you're too big" filter to bounce the message before the spam- filter 'holds' it for me.
/Bernie\
Bernie Cosell Fantasy Farm Fibers mailto:bernie@fantasyfarm.com Pearisburg, VA --> Too many people, too few sheep <--
On 4 Jan 2009 at 8:18, Mark Sapiro wrote:
Bernie Cosell wrote:
Is it possible to change the testing-order so that the list size limit is tested [and generates an appropriate reject] *before* the content filters are processed?
This is the way it's normally done. The usual request is to do it the other way around. See, e.g., <http://mail.python.org/pipermail/mailman-users/2008-December/064541.html>.
I think I see what to do. Am I correct that this would work [I'm a Perl guy, and I don't know much python at the moment]:
GLOBAL_PIPELINE.remove('SpamDetect') GLOBAL_PIPELINE.insert(GLOBAL_PIPELINE.index('Hold'), 'SpamDetect')
/bernie\
Bernie Cosell Fantasy Farm Fibers mailto:bernie@fantasyfarm.com Pearisburg, VA --> Too many people, too few sheep <--
Bernie Cosell wrote:
I think I see what to do. Am I correct that this would work [I'm a Perl guy, and I don't know much python at the moment]:
GLOBAL_PIPELINE.remove('SpamDetect') GLOBAL_PIPELINE.insert(GLOBAL_PIPELINE.index('Hold'), 'SpamDetect')
Not quite.
First let me say for the benefit of others reading this, that in general, moving SpamDetect after Hold is not a good idea. However, in your case, as described in your immediately prior post, where all your header_filter_rules actions are 'hold' anyway, I think it's OK.
The lines above won't do what you want. They will move SpamDetect to a position immediately before Hold. You want it after Hold so you want
GLOBAL_PIPELINE.remove('SpamDetect') GLOBAL_PIPELINE.insert(GLOBAL_PIPELINE.index('Hold') + 1, 'SpamDetect')
This will make the initial part of GLOBAL_PIPELINE look like
[ 'Approve', 'Replybot', 'Moderate', 'Hold', 'SpamDetect', 'MimeDel',
So you will do things in the following order:
- Detect pre-approved message (Approved: password header)
- Send any configured Auto-responder responses
- Process member moderation and non-member actions
- Process miscellaneous holds (including bounce_matching_headers)
- Process header_filter_rules
- Apply Content filtering
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 4 Jan 2009 at 9:08, Mark Sapiro wrote:
First let me say for the benefit of others reading this, that in general, moving SpamDetect after Hold is not a good idea. However, in your case, as described in your immediately prior post, where all your header_filter_rules actions are 'hold' anyway, I think it's OK.
The lines above won't do what you want. They will move SpamDetect to a position immediately before Hold. You want it after Hold so you want
GLOBAL_PIPELINE.remove('SpamDetect') GLOBAL_PIPELINE.insert(GLOBAL_PIPELINE.index('Hold') + 1, 'SpamDetect')
Another dumb newbie question: as I understand it, if I edit these into my mm_cfg.py it will affect the processing of *ALL* of the mailing lists, not just the one I want the reversed order on. That happens to be OK at the moment, but feels wrong -- is there a way to make this processing- order change *ONLY* apply to the one list at hand?
Thanks!!
/Bernie\
-- Bernie Cosell Fantasy Farm Fibers mailto:bernie@fantasyfarm.com Pearisburg, VA --> Too many people, too few sheep <--
Bernie Cosell wrote:
Another dumb newbie question: as I understand it, if I edit these into my mm_cfg.py it will affect the processing of *ALL* of the mailing lists, not just the one I want the reversed order on. That happens to be OK at the moment, but feels wrong -- is there a way to make this processing- order change *ONLY* apply to the one list at hand?
Your understanding is correct and yes, there is a way to modify the pipeline for a specific list. If the list object has a 'pipeline' attribute, that replaces the GLOBAL_PIPELINE for that list. See the FAQ at <http://wiki.list.org/x/l4A9> for ways to implement this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Bernie Cosell -
Mark Sapiro