At 12:38 PM 3/11/2010 +0530, Baiju M wrote:
>On Thu, Mar 11, 2010 at 11:05 AM, Baiju M <mbaiju(a)zeomega.com> wrote:
> > If "_" is a valid project_name identifier, why it is replaces with "-" ?
In order to have a canonicalized name form which can be escaped in
filenames for unambiguous identification of an egg's project and version.
Egg filenames use '-' as a separator between name, version, python
version, and platform. A '-' in any of these components is escaped
as '_', so that the '-' remains a viable and unambiguous
separator. This means that '_' gets turned back into a '-' when
unescaped, so the mapping between '_' and '-' is part of the
safe_name canonical form.
>There nearly 300 packages in PyPI with "_" in the package name.
>For all the packages built using Setuptools, the "Name" field in
>the PKG-INFO file is replaced with "-".
>I checked some of the packages built with "distutils.core" 
>Distutils is not replacing "Name" field in PKG-INFO file
>Why Setuptools is behaving different from Distutils ?
Because distutils wasn't built in a world where: package names needed
to be uniquely and unambiguously machine-parseable from
filenames. The code that easy_install has for dealing with
distutils-named source distributions has to guess at possible
interpretations of those filenames, because distutils filenames don't
distinguish between a '-' in a name or version, and a '-' *between*
names and versions.
Ultimately, the simplest way to deal with this was to treat runs of
'_' (or any other non-alphanum, non-dot character), as being
identical to a single '-'.
>Buildout has a functionality to "pin-down" ("lock down"/"nail down") versions
>of eggs (distribution?). There is another functionality to enforce
>versions of all eggs used in a particular Buildout configuration. If we
>use "_" as the package name (distribution name?), this functionality is not
Your comparisons should be based on the 'key' attribute of
Distribution and Requirement objects, rather than relying on direct
string operations of your own. The 'key' attribute contains a form
of project name suitable for equality/inequality comparisons.
In other words, you should not take unparsed data from your
configuration and compare it against pkg_resources attributes. Use
constructors like using Requirement.parse() and
Distribution.from_filename() to create objects with 'key' attributes,
then compare keys, or just use Requirement.__contains__. For example:
if someDistribution in Requirement.parse(projname+'=='+exactversion):
# someDistribution is exactly version exactversion of projname
The pkg_resources API is there precisely so that you don't have to
know all the low-level details like syntax rules and escaping.
At 10:41 AM 2/25/2010 -0500, Olemis Lang wrote:
>PS: BTW, how could I trigger easy_install(ation) at a given point
>while implementing a distutils command, and let the command perform
>further actions if deps are installed correctly ?
Setuptools' "Distribution" object has a method for fetching
dependencies. See setuptools' "test" command for an example. (This
doesn't install the dependencies globally, just drops eggs into the
build directory. But they're there and available to be reused for
installation in a later phase, under normal circumstances.)
At 09:50 AM 3/10/2010 +0530, Baiju M wrote:
>I spend some time with Buildout and setuptools code to identify the issue.
>I will try to explain my findings.
>1. Buildout is relying on pkg_resources.Requirement.parse function to
> get the "project_name" like this:
> I can see from the code of `Requirement` class that, the `__init__`
> method is deprecated and recommend to use `parse`
It is undocumented, not deprecated. You are simply not supposed to
create instances via that (private) constructor.
> Does this mean that we should not use the attributes
> of an instance of `Requirement` class? This is very important as
> the `parse` function return a list of instances of `Requirement` class.
Requirement objects are documented; see:
> So, if it is acceptable to use the "project_name" attribute, then
> Buildout can rely on it, right ?
> According to this code, this will be the result:
> Is this behavior correct ?
Yes it is. All non-alphanumeric, non-dot characters are replaced with
'-' in a project name. This turns project names like e.g. "Foo's Bar
Factory" into their canonical form (i.e., "Foo-s-Bar-Factory").
> If you think what setuptools doing is fine, we will make changes
> in Buildout code to use the "safe_name" method where ever it directly
> get "project_name".
Why do you think you need that? (Most likely, you are mistaken,
since the only reason the unsafe_name attribute exists is to deal
with a limitation in older versions of the PyPI software, which did
not support doing "safe_name" redirection.)
If you can describe what you're actually trying to do with this
information, perhaps there is a safer/more documented way that I can suggest.
At 03:03 PM 3/9/2010 -0600, Brad Allen wrote:
>Today I was informed of an issue in which buildout (with the latest
>setuptools) is not resolving version numbers properly, causing the
>wrong package to be selected in some cases. The cause identified was
>having '_' in the package name.
I suspect there is a miscommunication or misunderstanding
somewhere. It is perfectly acceptable to have a '_' in a package
name or project name. This:
>| >>> a="jiva_interface-2.3.6-py2.6.egg"
>| >>> b="jiva_interface-2.3.8-py2.6.egg"
>| >>> pkg_resources.parse_version(a)
Is the wrong API to use to parse an egg filename, as parse_version()
is for parsing a version that's already extracted from a
filename. This is the right API for extracting a version from a filename:
And here's the correct one for extracting the parsed version from a filename:
('00000002', '00000003', '00000006', '*final')
('00000002', '00000003', '00000008', '*final')
('00000000', '00000001', '00000001', '*final')
('00000000', '00000001', '00000002', '*final')
As you can see, these APIs work just fine, so the example given is a
red herring, unless Buildout is using the APIs incorrectly (which I
really doubt it is).
Usually, the situation where people run into trouble with unusual
package names or filenames is when they produce a source distribution
manually, or by using something other than distutils/setuptools (that
has different filename escaping rules), or when they manually rename
a file before uploading, and expect it to still work the same.
It would be a good idea for you to check which of these things (if
any) is taking place, and provide details of the specific problem,
with steps to reproduce it, since the example given probably has
nothing to do with it. Thanks!
At 02:57 PM 3/9/2010 -0500, hari jayaram wrote:
>I am installing setuptools using the --prefix option it complains
>the directory does not exist even though it does. The install works
>however when I use the --instal-dir option as it recommends.
>macbook-pro-17:~ hari$ sudo sh setuptools-0.6c11-py2.6.egg
That's because the directory you gave is not a *prefix*, it's an
A prefix, in distutils terms, is generally the top-level directory
under which Python's lib, bin, and other directories are located.
So, the correct --prefix would be
"/Library/Frameworks/Python.framework/Versions/2.6" in this
case. (Notice the absence of lib/python2.6/site-packages.)
> [Errno 2] No such file or directory:
If you look closely at this path, you'll see that when the distutils
creates an install directory from your prefix (which is already an
installation directory), you end up trying to install to a
directory. That's because when you give a prefix, distutils decides
what subdirectories of the prefix to install things in, and it can't
tell that you already gave it a subdirectory of the prefix.
I am installing setuptools using the --prefix option it complains the
directory does not exist even though it does. The install works however when
I use the --instal-dir option as it recommends.
Thought I would pass this along
macbook-pro-17:~ hari$ sudo sh setuptools-0.6c11-py2.6.egg
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
[Errno 2] No such file or directory:
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
This directory does not currently exist. Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
New submission from Zooko O'Whielacronx <zooko(a)zooko.com>:
I am trying to get Tahoe-LAFS listed on the FSF/UNESCO directory of Free Software: http://directory.fsf.org/ . One of the sticking points is that Tahoe-LAFS includes a copy of setuptools (http://allmydata.org/trac/tahoe-lafs/browser/misc/dependencies/setuptools-0… ) and, as the FSF representative wrote:
Next, the following code misc/dependencies/setuptools/EGG-INFO/PKG-INFO
calls for either the Zope Public License or the Python Software
Foundation License neither of which is present. One of them needs to be
added in order to fix the problem. Both can be found through links on
To fix this, include a copy of the licence text with the distribution. (Or close this ticket as wontfix and I'll work-around it.)
title: include a copy of the licence text
Setuptools tracker <setuptools(a)bugs.python.org>
At 01:22 PM 3/2/2010 -0800, Walter Coole wrote:
>When I try to install with:
> sudo sh setuptools-0.6c11-py2.5.egg
>I get an error:
> error: invalid Python installation: unable to open
> /usr/lib64/python2.5/config/Makefile (No such file or directory)
>That I don't really understand. Yes, it's true that there is no
>/usr/lib64/python2.5/config directory, but I've gotten along nicely
>without one for quite a while.
You are probably missing your distribution's "python-dev" or
"python-devel" package, which is required to use the distutils at
all, let alone setuptools.
New submission from Dan Cotruta <dan.cotruta(a)artsalliancemedia.com>:
It seems that creating eggs with bdist_egg does not read egg_info writers for extra files to be placed in EGG-INFO. Running setup.py egg_info first fixes the problem, but it seems like this step should not be necessary.
Am I mistaken, or is this a real bug?
title: bdist_egg ignoring egg_info:writers
Setuptools tracker <setuptools(a)bugs.python.org>
At 02:52 PM 3/2/2010 -0500, Mathieu Jobin wrote:
>maybe you can help me. there is something wrong with building this
>rpm package ?
>maybe because I am running ....
>Python 2.2.3 (#1, May Â 1 2006, 12:33:49)
Setuptools requires Python 2.3, preferably 2.3.5 or higher.