[ mailman-Bugs-1461279 ] Filtering content while allowing HTML messages impossible

Fri Mar 31 09:22:22 CEST 2006

Bugs item #1461279, was opened at 2006-03-30 12:41
Message generated for change (Comment added) made by psy-q
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1461279&group_id=103

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: 2.1 (stable)
>Status: Closed
>Resolution: Invalid
Priority: 2
Submitted By: - (psy-q)
Assigned to: Nobody/Anonymous (nobody)
Summary: Filtering content while allowing HTML messages impossible

Initial Comment:
I hope this is a genuine bug. I'll use 2.1.7 in these    
examples, though I verified this behavior in 2.1.5    
also.    

I have a mailing list configured as such:    

filter_content = 1    
filter_mime_types = ''   
pass_mime_types = """multipart/mixed   
multipart/alternative   
text/plain   
text/html"""   
collapse_alternatives = 0   
convert_html_to_plaintext = 0   
filter_action = 2   

And yet all HTML messages are automatically stripped   
of HTML, have their attachments removed and are    
immediately posted to the list.   

The originals are sent as multipart/alternative with  
two attachments, one the text/plain rendition of the  
message and one the text/html original. It seems to 
be okay, user agents can render this message as 
intended. All that remains when delivered by Mailman 
is the text/plain bit, though. 

Shouldn't this list configuration let HTML mails  
through? I can't see a reason why Mailman should 
remove the HTML silently with these settings. 

If filter_content is turned off, sending HTML 
messages works. 

(I know it's not polite to send HTML messages through 
mailing lists, but some of my users insist.) 

----------------------------------------------------------------------

>Comment By: - (psy-q)
Date: 2006-03-31 09:22

Message:
Logged In: YES 
user_id=244049

Seems Konqueror messed up the line breaks in my last reply.
These last two days are full of cursed software :)

I tried a vanilla installation of Mailman 2.1.7, configured
the list as described here and everything worked exactly as
expected (and advertised).

I can't explain where yesterday's behavior came from, and
I'm sorry if I caused any confusion. I can only guess: Could
it be that that something in the 2.1.5 to 2.1.7 upgrade
didn't go right, or that I did something wrong, and somehow
2.1.5's hardcoded collapse_alternatives had taken effect
even though 2.1.7's settings said not to collapse? I did
restart the qrunner with mailmanctl after the upgrade, but
on the other hand, I did install 2.1.7 into the same
location where 2.1.5 previously was. Something there might
have been the source of the error.

----------------------------------------------------------------------

Comment By: - (psy-q)
Date: 2006-03-31 09:04

Message:
Logged In: YES 
user_id=244049

Thanks for clearing that up, you've confirmed most of what      
I've only had vague suspicions about :)      

I found out about 2.1.5's hardcoded collapse_alternatives     
behavior about half an hour after posting my bug, but     
thanks for confirming it.     

I feel like a fool at the moment. I just retried this so I      
could give you example messages, and the behavior has      
stopped. I reproduced the unwanted filtering for over an      
hour yesterday, I have no idea why it's working correctly      
now. Yesterday, I tried about a dozen combinations of      
filter strings and settings, none of which produced the      
desired result. In the end, I returned to the      
configuration that I posted here and performed a final      
test to verify that the behavior is indeed confusing.      

Today, I switched my test MTA back on, switched Mailman      
back on and tried the very same test message as yesterday.      
Amazingly, it went through exactly as expected. Then I      
removed text/html from pass_mime_types, and again it     
behaved exactly as advertised: It let the     
multipart/alternative message through, kept the text/plain     
part intact but removed the text/html. This is completely     
different behavior than I saw yesterday.     

So now I'm even more confused than before -- In theory,     
this means that an upgrade to 2.1.7 using this list     
configuration would make everything work as it should at    
my site, and it would mean that there is no bug.   

I will test a bit more thoroughly, I'll remove Mailman and 
do a vanilla install of 2.1.7 to see if I can get the 
behavior to show again. 

----------------------------------------------------------------------

Comment By: Mark Sapiro (msapiro)
Date: 2006-03-31 03:00

Message:
Logged In: YES 
user_id=1123998

You can't really compare 2.1.5 and 2.1.7 in this respect
since 2.1.5 doesn't have the collapse_alternatives setting
and always operates as if collapse_alternatives is true
(non-zero). I.e., 2.1.5 will always replace a
multipart/alternative part with just the first alternative.

Also, your "immediately posted" remark makes me think that
you think filter_action applies whenever content is filtered
in any way. This is not true. filter_action only applies
when the message is empty after filtering.

You also say "yet all HTML messages are automatically
stripped of HTML, have their attachments removed and ...". I
think you may or may not have a misconception here. With the
pass_mime_types settings you have given, no attachments of
any type other than text/plain or text/html should be left
in the message. The types multipart/alternative and
multipart/mixed only allow such messages/parts to be further
processed looking for allowable types. I.e, with these
settings a message or part of type multipart/related will be
filtered completely (nothing left) regardless of what it
contains, because multipart/related is not accepted.

On the other hand, a message or part of type multipart/mixed
will have it's sub-parts examined for allowable types
because multipart/mixed is allowed.

So the only issue is why are text/html parts being removed
in 2.1.7 from multipart/mixed and/or multipart/alternative
parts. This may or may not be a bug. Please attach full
copies of a message both before and after filtering
including all headers to this report, so we can see if there
is a bug or what the problem might be.

Note that your description of the multipart/alternative
messages and results is the only way Mailman 2.1.5 works so
there is no bug there. In 2.1.7 with collapse_alternatives =
0 and text/html in pass_mime_types, the html should remain
in the outgoing message, so there is clearly a problem here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1461279&group_id=103