Re: [Mailman-Developers] [Mailman-Users] [Fwd: TypeError: us-ascii with python2.4 and mailman 2.1.8-1 (debian)]

Hi developers,

This particular problem is caused by a bug in email 4.0.1 package which was fixed in the most recent subversion repository. http://svn.python.org/view/python/trunk/Lib/email/message.py?rev=54333&r1=50840&r2=54333

Maybe it's time to think of next bug fix release of mailman 2.1.10 as soon as the email 4.0.2(?) is out.

Justin Warren wrote:

Hi folks,

I hit the same sort of problem as a couple of other people apparently have, eg:

http://mail.python.org/pipermail/mailman-users/2006-November/054246.html

I hit this when Mailman attempts to send digests.

I thought I'd tracked the issue to the way Charset is used within Mailman/Scrubber.py, on line 193 (some modifications by me to print debug output):

charset = None                                                                                                     
lcset = Utils.GetCharSet(mlist.preferred_language)                                                                 
print "getting charset for preferred language:", repr(lcset)                                                       
lcset_out = Charset(lcset).output_charset or lcset                                                                 
print "lcset_out is:", repr(lcset_out) 

Which yields:

getting charset for preferred language: 'us-ascii' lcset_out is: u'us-ascii'

It appears that Charset returns a unicode string for its .output_charset, which python2.4 email.Message.set_charset() does not recognise as a string, and it isn't a Charset either, so a TypeError is raised.

I don't know enough about why .output_charset is being used, or why it is unicode, but it seems that changing line 193 to read:

lcset_out = Charset(lcset) or lcset

would prevent this error.

However, something else is also returning a unicode string, since after changing Scrubber, you end up with this error:

Traceback (most recent call last): File "/usr/lib/mailman/cron/senddigests", line 94, in ? main() File "/usr/lib/mailman/cron/senddigests", line 86, in main mlist.send_digest_now() File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now ToDigest.send_digests(self, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests send_i18n_digests(mlist, mboxfp) File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 339, in send_i18n_digests mcset = msg.get_content_charset('') File "/usr/lib/python2.4/email/Message.py", line 805, in get_content_charset charset = unicode(charset, 'us-ascii').encode('us-ascii') TypeError: decoding Unicode is not supported

I haven't worked out what this other thing is as yet.

Alternately, patching email/Message.py (line 803) to ignore a charset that is a unicode string seems to work, but is probably fragile or broken in other ways:

    # charset character must be in us-ascii range                                                                  
    try:                                                                                                           
        if not isinstance(charset, unicode):                                                                       
            charset = unicode(charset, 'us-ascii').encode('us-ascii')                                              
    except UnicodeError:                                                                                           
        return failobj 

I hope this helps someone more knowledgable about Mailman to work out what the underlying problem is.

-- Tokio Kikuchi, tkikuchi@is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/