[Catalog-sig] reStructuredText whitespace lost when writing pkg-info

[stephen emslie]

This is something that I have already sent to the distutils-sig list.
Apologies for the cross-post, but seeing Tarek Ziadé's recent work on
the Plone software center and enhancing pypi got me thinking that this
might actually be a more appropriate place to ask, as I am working on
something similar.

I'd be grateful for any help or comments:

---------- Forwarded message ----------
From: stephen emslie <stephenemslie at gmail.com>
Date: Jan 10, 2008 5:20 PM
Subject: reStructuredText whitespace lost when writing pkg-info
To: distutils-sig at python.org


I've noticed that setuptools seems to flatten any indentation in
fields when generating the PKG-INFO metadata file. I noticed this
because PEP345 [1] suggests that the long description field can be
written in reStructuredText (which would be excellent for my current
project), but unfortunately whitespace is being lost that breaks
things like literal blocks that require indentation.

The trouble seems to be that distutils.util.rfc822_escape strips all
excess whitespace and adds 8 spaces of padding to the start of every
line. For example:

>>> rest = """
... a literal python block::
...   >>> import this
... """
>>> print distutils.util.rfc822_escape(rest)

        a literal python block::
        >>> import this

which generates a few warnings and no literal block when docutils
tried to render it.  Can anyone tell me if this behaviour is necessary
to be rfc822/MIME compliant, or is distutils being over-cautious in
flattening out all whitespace? The w3c discussion on multiple lines in
rfc822 [2] seems to suggest that whitespace can be 'unfolded' safely,
so it seems a shame to be throwing it away when it can have important
meaning.

[1] http://python.cofman.dk/peps/pep-0345.html
[2] http://www.w3.org/Protocols/rfc822/3_Lexical.html

Stephen Emslie