[Tracker-discuss] [issue147] Current email headers corrupt mail reader display

[Dennis Allison]

Dennis Allison added the comment:

These examples were pulled out of the mbox file directly using vi.  Since 
I copied them out, I have deleted them and unsubscribed from the list 
while this problem is being resolved.  I can resubscribe if necessary.

The Stanford mail system has a mail hub that gathers messages and then
redistributes them.  In this case, the mail went to mx1, one of the mail
hubs, then to the leland cluster of machines. On those machines I have a
.forward set to forward the mail to shasta.stanford.edu, a virtual server
running Linux using Xen or an equivalent.

At each stage, it's likely that some headers are appended and some 
translation may occur, but it is unlikely that anything is done to the 
From: header which appears to be what is broken.

When Pine grabs the From: header and tries to display it, the header 
behaves as if it has CRLFs in it.  The GUI is ncurses based, I believe, 
and so gets quite confused overwriting lines here and there and generally 
making navigation in the index complicated.

From: =?utf-8?q?=0A=09=09=09=09=09Hirokazu_Yamamoto=0A=09=09=09=09?=
        <report at bugs.python.org>

ends up being displayed as three lines.   

The From: header seems to be the only thing that is mucked up.

What I suspect is happending is that the header is in UTF-8 and the
charset information is added at some point in the mail transmission. =0A
is a newline, =09 is a tab, The various "?' characters are out of range
ASCII characters, probably utf-8 escapes which are inserted at some point
(possibly by Pine) when mapping utf-8 to ASCII.  Pine's support for
unicode is likely to be limited to doing a best guess remap to ASCII.

Applying the folding rules to unfold, the two lines in the From: header 
are joined, then the characters are translated from the encoded to 
unencoded form giving (modulo some special characters):

[utf-8]\n\t\t\t\tHirokazu_Yamamoto\n\t\t\t\t<report at bugs.python.org>

which would display as
-------------------
[utf-8]
				Hirokazu_Yamamoto
				<report at bugs.pythong.org>
--------------------
overwriting lines following it in the ncurses display.

I believe that the program that generates the email does not construct a 
proper From: header.

On Sat, 22 Sep 2007, [utf-8] Martin v. Löwis wrote:

> 
> Martin v. Löwis added the comment:
> 
> Dennis: unfortunately, these are still not the original messages. In example2,
> the text "Your display is set for the "ISO-8859-1" character set." appears,
> which was *not* in the original message. Please try to retrieve the original
> message from disk, e.g. from an mbox file, in the hopes that your email reader
> has not tampered with it yet (in case you are uncertain what the original text
> is: it also should not say "Unable to print this part.")
> 
> I can't stress enough the importance of getting the *original*, *unmodified*,
> *untampered*, *full* message that has been sent out to analyze the problem further.
> 
> _______________________________________________________
> Meta Tracker <metatracker at psf.upfronthosting.co.za>
> <http://psf.upfronthosting.co.za/roundup/meta/issue147>
> _______________________________________________________
> 

--

_______________________________________________________
Meta Tracker <metatracker at psf.upfronthosting.co.za>
<http://psf.upfronthosting.co.za/roundup/meta/issue147>
_______________________________________________________