[Mailman-Users] Mime conversions - missing carriage returns andoddcharacters

[Ryan Steele]

Mark Sapiro wrote:
> Ryan Steele wrote:
>   
>> Mark Sapiro wrote:
>>     
>>> It is possible that something in Mailman's MimeDel (content filtering)
>>> is misrepresenting the character set and causing utf-8 encoded text to
>>> be declared as some other character set, but it could also be
>>> something else. 
>>>       
>> Sounds like a fair enough assumption.  I guess that could also account 
>> for the carriage returns not being evaluated properly?
>>     
>
>
> I don't think so, but I can't be sure.
>
>
>   
>>> Do you have convert_html_to_plaintext set to Yes?
>>>   
>>>       
>> Affirmative, I do.
>>     
>
>
> Based on what I see, I don't think that is the problem.
>
>
>   
>>> To diagnose this further, we need to see an original message as sent to
>>> the list (e.g. a Bcc: of a list post) and the message as received from
>>> the list. These need to be raw messages with all MIME headers intact.
>>>
>>>   
>>>       
>> Following is an original message as sent to the list, before being 
>> munged by MimeDel:
>>
>> ########################################################################################
>>
>>     
> <Received: headers snipped>
>   
>>  From: someaoluser at aol.com
>>  Message-ID: <2d3.3f8b576.313c8b61 at aol.com>
>>  Date: Sun, 5 Mar 2006 13:43:45 EST
>>  Subject: Re: [somelist] Building the Field of O.D. into a Profession
>>  To: list-of-users at domains.com
>>  CC: otheruser at otherdomain.org
>>  MIME-Version: 1.0
>>  Content-Type: multipart/alternative; 
>> boundary="-----------------------------1141584225"
>>  X-Mailer: 9.0 SE for Windows sub 5022
>>     
> <X-Spam-* headers snipped>
>   
>>  Dear David,
>>     
>
>
> This is incomplete. You have copied the message headers, and then
> apparently the body as displayed by your mail client. I.e., the
> message is multipart/alternative, but I don't see any part headers for
> the alternative parts and I only see one body part.
>
> <snip>
>
>   
>> ########################################################################################
>>
>> Here is that same message as received by the list, after being munged by 
>> MimeDel:
>>
>> ########################################################################################
>>
>>     
> <Received: headers snipped>
>   
>> From: someaoluser at aol.com
>> Message-ID: <2d3.3f8b576.313c8b61 at aol.com>
>> Date: Sun, 5 Mar 2006 13:43:45 EST
>> To: list-of-users at domains.com
>> MIME-Version: 1.0
>> X-Mailer: 9.0 SE for Windows sub 5022
>> X-Content-Filtered-By: Mailman/MimeDel 2.1.8
>> Cc: otheruser at otherdomain.org
>> Subject: Re: [somelist] Building the Field of O.D. into a Profession
>> X-BeenThere: somelist at lists.company.org
>> X-Mailman-Version: 2.1.8
>> Precedence: list
>> List-Id: Our email discussion for the godparents of our field
>>         <somelist.lists.company.org>
>> List-Unsubscribe: < http://lists.company.org/mailman/listinfo/somelist>,
>>         < mailto:somelist-request at lists.company.org?subject=unsubscribe >
>> List-Archive: < http://lists.company.org/mailman/private/somelist>
>> List-Post: < mailto:somelist at lists.company.org>
>> List-Help: < mailto:somelist-request at lists.company.org?subject=help>
>> List-Subscribe: < http://lists.company.org/mailman/listinfo/somelist>,
>>         < mailto:somelist-request at lists.company.org?subject=subscribe >
>> Content-Type: text/plain; charset="utf-8"
>> Sender: somelist-bounces at lists.company.org
>> Errors-To: somelist-bounces at lists.company.org
>>     
> <X-Spam-* headers snipped>
>   
>>  Dear  David, Thank you for your reply. Starting back in 1981  when I 
>>     
> <snip>
>   
>> re-writing what became âEURoeThe International O.D. Code  of 
>> EthicsâEUR?. It is based on comments from key O.D. people  from all over 
>>     
> <snip>
>   
>> ########################################################################################
>>     
>
>
> Here I may be seeing the whole message. It contains some possibly
> garbled utf-8 characters, but the Content-Type: header identifies it
> as utf-8 characher set, so I suspect your mail client is not properly
> rendering utf-8.
>
> It looks like (I can't be certain because these aren't the raw
> messages) Mailman has done the normal thing which is select the
> text/plain alternative and replace the multipart/alternative message
> body with a body consisting only of the text/plain part. I really need
> to see the full raw source of the original message to the list (the
> first message above). I.e., what you see when you select View->Message
> Source in Thunderbird.
>
>   

I appreciate your continued help on this Mark.  I apologize about the 
pre-garbled message - I simply copied it from the information I 
requested from the client without checking to see that it contained part 
boundaries within the body as well.  I'm not even sure it's possible in 
Outlook to get this information (I researched/looked for a little while 
to no avail), so I'm going to set up an Outlook installation, mimicking 
the UTF-8 stuff, and send it to a test list I've got and my own personal 
account, which is set up using Thunderbird.  Then I can just ctrl+u to 
get the message source pre-garbling.  I'll post to the list once I've 
got this all set up.  Thanks!

Ryan

-- 
Ryan Steele                         
Systems Administrator               steele at agora-net.com
AgoraNet, Inc.                      (302) 224-2475
314 E. Main Street, Suite 1         (302) 224-2552 (fax)
Newark, DE 19711                    http://www.agora-net.com