[Mailman-Users] [Fwd: TypeError: us-ascii with python2.4 and mailman 2.1.8-1 (debian)]

[Justin Warren]

Hi folks,

I hit the same sort of problem as a couple of other people apparently
have, eg:

http://mail.python.org/pipermail/mailman-users/2006-November/054246.html

I hit this when Mailman attempts to send digests.

I thought I'd tracked the issue to the way Charset is used within
Mailman/Scrubber.py, on line 193 (some modifications by me to print
debug output):

    charset = None                                                                                                     
    lcset = Utils.GetCharSet(mlist.preferred_language)                                                                 
    print "getting charset for preferred language:", repr(lcset)                                                       
    lcset_out = Charset(lcset).output_charset or lcset                                                                 
    print "lcset_out is:", repr(lcset_out) 

Which yields:

getting charset for preferred language: 'us-ascii'
lcset_out is: u'us-ascii'

It appears that Charset returns a unicode string for
its .output_charset, which python2.4 email.Message.set_charset() does
not recognise as a string, and it isn't a Charset either, so a TypeError
is raised.

I don't know enough about why .output_charset is being used, or why it
is unicode, but it seems that changing line 193 to read:

    lcset_out = Charset(lcset) or lcset

would prevent this error.

However, something else is also returning a unicode string, since after
changing Scrubber, you end up with this error:

Traceback (most recent call last):
  File "/usr/lib/mailman/cron/senddigests", line 94, in ?
    main()
  File "/usr/lib/mailman/cron/senddigests", line 86, in main
    mlist.send_digest_now()
  File "/var/lib/mailman/Mailman/Digester.py", line 60, in send_digest_now
    ToDigest.send_digests(self, mboxfp)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 142, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/var/lib/mailman/Mailman/Handlers/ToDigest.py", line 339, in send_i18n_digests
    mcset = msg.get_content_charset('')
  File "/usr/lib/python2.4/email/Message.py", line 805, in get_content_charset
    charset = unicode(charset, 'us-ascii').encode('us-ascii')
TypeError: decoding Unicode is not supported

I haven't worked out what this other thing is as yet.

Alternately, patching email/Message.py (line 803) to ignore a charset
that is a unicode string seems to work, but is probably fragile or
broken in other ways:

        # charset character must be in us-ascii range                                                                  
        try:                                                                                                           
            if not isinstance(charset, unicode):                                                                       
                charset = unicode(charset, 'us-ascii').encode('us-ascii')                                              
        except UnicodeError:                                                                                           
            return failobj 

I hope this helps someone more knowledgable about Mailman to work out
what the underlying problem is.

-- 
Justin Warren <justin at seafelt.com>
seafelt.com: Making sense of a sea of data.