[Distutils] non-ascii characters in long description

[Tarek Ziadé]

On Thu, Sep 17, 2009 at 3:31 PM, Reinout van Rees <reinout at vanrees.org> wrote:
> On 2009-09-17, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
>> On Thu, Sep 17, 2009 at 2:46 PM, Reinout van Rees <reinout at vanrees.org> wrote:
>>> I have non-ascii characters in my long description (as I just fixed a bug in
>>> my package that had to do with non-ascii characters).  I did all the right
>>> things like opening my changelog and readme with "codecs.open(...,
>>> encoding='utf-8')" and so.  But I ran into the following setuptools problem:
>>>
>>> When calling "python setup.py --long-description", setuptools effectively does
>>> a "print long_description", which works with a utf8 string, but fails with a
>>> unicode string.
>>>
>>> When uploading to pypi, setuptools calls unicode() on my long description,
>>> which fails with a utf8 string and works with a unicode string.
>>
>> Mmm, I've fixed that problem in Distutils code in 2.6+  IIRC
>
> Ah, I'm still using 2.5 mostly.
>
>> Are you sure this code is in setuptools and not in distutils ? can you
>> provide a traceback ?
>
> You're completely right: distutils (at least the upload to pypi bug).  Here's
> the traceback:
>
> Traceback (most recent call last):
>  File "setup.py", line 47, in <module>
>    'lasttagdiff = zest.releaser.lasttagdiff:main'],
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup
>    dist.run_commands()
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 986, in run_commands
>    self.run_command(cmd)
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 1006, in run_command
>    cmd_obj.run()
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg/setuptools/command/register.py", line 9, in run
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 48, in run
>    self.send_metadata()
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 162, in send_metadata
>    auth)
>  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/register.py", line 257, in post_to_server
>    value = unicode(value).encode("utf-8")
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6030:
>  ordinal not in range(128)

Yes that's the one I've fixed. You have to backport a fix in your
setup.py if python < 2.6

You can register.patch post_to_server to pre-process all values in the
"data" argument, so they are
all in unicode, then call the real one.

That's hackish, but at least unicode() won't break anymore when called
on already decoded strings eg unicode objects.

Tarek

>
>
>
> Reinout
>
> --
> Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
> Software developer at http://www.thehealthagency.com
> "Military engineers build missiles. Civil engineers build targets"
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Tarek Ziadé | http://ziade.org | オープンソースはすごい!