[Distutils] "Safe" Project Names and underscores in Project Names issue

[P.J. Eby]

At 08:02 PM 8/4/2011 -0700, Michael wrote:
>Forgive me if I ask a stupid question but what characters are 
>allowed in a project name on PyPI and by Distribute/Setuptools?

Setuptools allows names that have been processed through safe_name.


>As such, modifying the safe_name method to recognize undescore breaks pip.

Why are you modifying the safe_name function?  If you change it, 
it'll no longer be safe.  ;-)



>Doing "pip install pyramid_beaker" installs the package successfully 
>however any package tools (i.e pip) report the project as 
>"pyramid-beaker" and as such becomes unusable name if PyPI is 
>queried via RPC Interfaces. Trying to turn all the dashes back 
>underscores breaks packages that have underscores in them (i.e. 
>sphinxcontrib-programoutput). And if the package were to use both, 
>it would be impossible to determine what it is without running all 
>the permutations.
>
>I would appreciate if any one could advise me on this matter and let 
>me know if this is a bug underscore treatment with 
>Distribute/Setuptools, pip or PyPI.

It's a PyPI incompatibility - neither the distutils nor PyPI were 
originally designed for a world involving automatically-resolved 
dependencies, where names needed to be unambiguous.  That is, PyPI 
and distutils had an implicit assumption that 1) people would choose 
reasonable names and 2) be able to handle it when other people didn't.

Setuptools, on the other hand, needs unambiguous naming, and 
therefore has a canonicalization algorithm that's designed to work 
reasonably well with distutils' file naming conventions.  Distutils 
normally uses '-' to separate name parts in a filename, and 
(sometimes) escapes '-' using '_'.  So setuptools canonicalizes all 
punctuation to '-', and escapes it as '_'.

Some of PyPI's code has been changed to work with this approach, and 
some has not.


>[pyramid-beaker does not exist on PyPI, since its pyramid_beaker)

 From setuptools POV, the package name is pyramid-beaker, because '_' 
is reserved for escaping punctuation in filenames.

If you want to look up a setuptools package name using PyPI XML-RPC, 
you can't always use the name directly; you may have to perform a 
PyPI search operation to obtain the package's PyPI name first.