[Distutils] non-ascii characters in long description

[Reinout van Rees]

On 2009-09-17, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
> On Thu, Sep 17, 2009 at 2:46 PM, Reinout van Rees <reinout at vanrees.org> wrote:
>> I have non-ascii characters in my long description (as I just fixed a bug in
>> my package that had to do with non-ascii characters).  I did all the right
>> things like opening my changelog and readme with "codecs.open(...,
>> encoding='utf-8')" and so.  But I ran into the following setuptools problem:
>>
>> When calling "python setup.py --long-description", setuptools effectively does
>> a "print long_description", which works with a utf8 string, but fails with a
>> unicode string.
>>
>> When uploading to pypi, setuptools calls unicode() on my long description,
>> which fails with a utf8 string and works with a unicode string.
>
> Mmm, I've fixed that problem in Distutils code in 2.6+  IIRC

Ah, I'm still using 2.5 mostly.

> Are you sure this code is in setuptools and not in distutils ? can you
> provide a traceback ?

You're completely right: distutils (at least the upload to pypi bug).  Here's
the traceback:

Traceback (most recent call last):
  File "setup.py", line 47, in <module>
    'lasttagdiff = zest.releaser.lasttagdiff:main'],
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 986, in run_commands
    self.run_command(cmd)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 1006, in run_command
    cmd_obj.run()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/setuptools/command/register.py", line 9, in run
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 48, in run
    self.send_metadata()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 162, in send_metadata
    auth)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 257, in post_to_server
    value = unicode(value).encode("utf-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6030:
  ordinal not in range(128)



Reinout

-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"