At 02:50 PM 1/18/2007 +0100, =?ISO-8859-2?Q?Micha=B3_Kwiatkowski?= wrote:
>Related setuptools question: is there a way to get list of available
>versions from easy_install?
The setuptools.package_index.PackageIndex class knows about all
discoverable versions. However, this doesn't extend to versions that have
been hidden by the package author. To have more than one discoverable
version at the PyPI level, the author has to have made them visible in the
But, whichever ones *are* visible can be discovered by the PackageIndex
class, along with any that are linked from the Home Page or Download URLs,
or are linked on the package's PyPI page.
More precisely, PackageIndex doesn't track versions; it tracks available
*distributions*, which may include source distributions, SVN checkouts,
eggs, .exe's, etc. But each distribution object carries version info
determined from its filename.
Hello all. I'm the primary author of nose. Michal Kwiatkowski was kind
enough to point me to this thread, and I wanted to chime in with a few
questions and answers. Apologies for pulling things from various parts
of the thread together, I didn't want to send multiple replies all
covering the same ground.
Nose works fine with setuptools in source packages. It provides
nose.collector for use as a test_suite with the test command, and a
nosetests command that uses nose's own testrunner and supports loading
options from setup.cfg. So for today's use I think it works pretty
well. That's the good news. The bad:
> (I'm not sure if nose really works with eggs, though; ScanningLoader will
> discover tests in eggs as long as the source is included, but nose may be
> strictly file-based for all I know. Changing it probably wouldn't be too
> difficult, however.)
There's no explicit egg support and, unlike ScanningLoader, nose uses
naive os and os.path functions to access the filesystem, so I think
I'll need to supply a 2nd loader than uses pkg_resources.
> The loader must support a .loadTestFromName(name, obj) call, where name
> will be an empty string, and obj will be the object specified by a test
> entry point. The return value must be a unittest.TestSuite. And it must
> be possible to call the loader multiple times with different 'obj' values.
This is broken in nose 0.9.x -- I have the logic of loadTestsFromName
backwards, in that it only uses the module argument to determine the
package context of the name to be loaded; if the name is blank and the
module is 'foo' it will try to load tests from 'foo.' Not so good. It
also currently is a generator that yields tests as they are loaded,
rather finding them all and returning a TestSuite -- I don't know
whether that will also be a problem.
And nose currently uses imp.find_module and imp.load_module to import
test modules, since that seemed to be the easiest way to handle the
very common case of multiple test modules with the same name in
different (non-package) directories. I don't know whether those will
work correctly when importing from an egg.
I'm hoping to fix all of these loader problems in the 0.10 series.
I'll make whatever changes will be needed to support loading from eggs
and the final form of this proposal (so long as that doesn't break
anything that works now), but as you can see I'm going to need some
guidance about what I need to do.
[CC-ing this to distutils-sig, because it relates to setuptools as
well, but please keep discussion on the catalog-sig if possible.]
Maybe this is a stupid question, but how can I list all versions of a
package that were registered on PyPI? For example, when I go to
http://python.org/pypi/setuptools/ I'm redirected to the page of last
version of setuptools. How can I check earlier versions of setuptools
from there? I know this functionality is available for project admins,
what about users?
Another thing, some packages offer a development versions by giving a
link to their subversion repository. Is there a way to list these
extra install options on PyPI? By "extra options" I mean all
non-standard version strings that will work with `easy_install
package==version`, like common "dev".
Related setuptools question: is there a way to get list of available
versions from easy_install?
[Please stop taking this off-list; I don't do off-list support.]
At 08:17 AM 1/17/2007 -0600, Jon Nelson wrote:
>Phillip J. Eby wrote:
> > At 05:05 PM 1/9/2007 -0600, Jon Nelson wrote:
> >> Say, I've got a local patch to setuptools that adds rpm 'Requires'
> >> lines for all dependencies, which is a huge boon to us that use rpm for
> >> everything (and rely on /it/ for dependency management). Interested?
> >> It's a pretty small patch, and I'm probably doing it wrong anyway, but
> >> it works with everything I've thrown at it so far.
> > The main problem I see is that there is no guaranteed mapping between
> > RPM package names and PyPI package names. I'm also not clear on what
> > scope you're trying to deal with the issue in.
>Well, in theory, the change would only affect those using setuptools
>*and* bdist_rpm. If using bdist_rpm, then it only /adds/ dependency
As far as I can tell, unless these requirements are prefixed with somethign
to indicate their CheeseShop/setuptools nature, *and* a corresponding
Provides: is done in the RPM for what *that* package provides, I don't
really see how that would work.
The difficulty I see in doing that, especially as a default setting, is
that it means dependency RPMs might need to be rebuilt as setuptools RPMs.
Essentially, I don't see how this can work as a default behavior, and I'd
like some input from other people using RPMs before considering adding this
as a normal behavior.
Note too, by the way, that there is absolutely no need for you to patch
setuptools in order to get this behavior. just create a project directory
And define a subclass of setuptools' bdist_rpm in my_bdist_rpm.py, list it
in your 'distutils.commands' entry points (see setuptools' own setup.py for
examples), and then run "setup.py install". You will then have a new
command installed on your machine, that you can run with *any*
setuptools-based package, or by using this trick to run a non-setuptools'
package's setup.py with your command:
python -c "import setuptools;execfile('setup.py')" my_bdist_rpm ...
So, there's nothing stopping you from having this behavior without needing
to maintain a patch; just maintain a subclass instead. You can even
distribute your extended command on the Cheeseshop, so that other people
can install it with easy_install and be able to use the same behavior, if
they prefer it. Some popularity might then be convincing with respect to
including it in 0.7 as a default behavior.
>I'll send you the patch anyway. What would really be ideal, IMO, would
>be for it to default to on (again, it only changes bdist_rpm behavior)
>and for it to be switchable somehow. For environments that use RPM
>heavily (like all of the environments I've worked in for almost 10 years
>now) it's a huge bonus.
>diff -ur setuptools-0.6c5.orig/setuptools/command/bdist_rpm.py
>2007-01-09 13:20:24.000000000 -0600
>+++ setuptools-0.6c5/setuptools/command/bdist_rpm.py 2007-01-09
>@@ -4,7 +4,9 @@
> # finally, a kludge to track .rpm files for uploading when run on
> from distutils.command.bdist_rpm import bdist_rpm as _bdist_rpm
>-import sys, os
>+from distutils.sysconfig import get_python_version
>+import sys, os, re
>+from pkg_resources import *
> class bdist_rpm(_bdist_rpm):
>@@ -12,6 +14,21 @@
> self.no_egg = None
>+ def finalize_options(self):
>+ requires = list( yield_lines(self.distribution.install_requires
>or ()) )
>+ if not requires: return
>+ if not self.requires:
>+ self.requires = 
>+ # dang. RPM requires versions in a different format (w/spaces!)
>+ VERS_RE = re.compile(r"\s*(<=?|>=?|==|!=)\s*((\w|[-.])+)")
>+ for req in requires:
>+ parts = VERS_RE.split(req)[:3]
>+ parts = safe_name(parts)
>+ if len(parts) > 1:
>+ parts = safe_version(parts)
>+ self.requires.append(' '.join(parts))
> if sys.version<"2.5":
> # Track for uploading any .rpm file(s) moved to self.dist_dir
> def move_file(self, src, dst, level=1):
I've got few questions about defining setup.py
My project consits of intricated packages :
-- -- __init__.py
-- -- sources_pack2
The package pack2 uses librairies defined in Pack1;
My setup.py looks like :
pack1_srcs = glob.glob ("pack1/*.cc")
pack1_ext = Extension (
sources = pack1_srcs,
libraries = ['boost_python'],
include_dirs = [include_dir],
pack2_srcs = glob.glob ("Pack1/Pack2/*.cc")
pack2_ext = Extension (
sources = Pack2_srcs,
libraries = ['boost_python'],
include_dirs = [include_dir],
version = '1.0',
author = '***',
author_email = '***',
url = '***',
description = "****",
packages = ['Pack1,
ext_modules = [pack1_ext,
The compilation ./setup.py works well, but it seems that there is a problem
with the linking of the librairies. When i launch a script using pack2,
trying to import things from pack1, i've got undefined symbols (of symbols
defined in pack1)
On the other hand, if i succeed in compiling/installing pack1, then adding
Pack1 to extra_objects of Pack2 makes it work !
Then i'm wondering if there is a mean to compile pack1, then telling to
setup.py that it should use that library compiled before.
Probably it is not possible and i should define two separated directories
so that the first library is compiled and then , the compilation of the
second one uses the first one ... i don't know.
Thanks for your help !
At 05:05 PM 1/9/2007 -0600, Jon Nelson wrote:
>Say, I've got a local patch to setuptools that adds rpm 'Requires'
>lines for all dependencies, which is a huge boon to us that use rpm for
>everything (and rely on /it/ for dependency management). Interested?
>It's a pretty small patch, and I'm probably doing it wrong anyway, but
>it works with everything I've thrown at it so far.
The main problem I see is that there is no guaranteed mapping between RPM
package names and PyPI package names. I'm also not clear on what scope
you're trying to deal with the issue in.
Jos Yule wrote:
> I didn't know where else to address this - is there a mailing list or
> group which would be better suited to this email?
The distutils-sig group (copied) might be good. Especially when I'm
unsure of the answer like now ;)
> First - thanks for workingenv! Very helpful.
> I've run into a bug when using it to install Pylons on a WinXP box,
> using easy_install.exe.
> When doing a "easy_install Pylons", after creating a new workingenv and
> activating it, it gets hung up on a missing cli.exe from "<workingenv
> dir>/lib/python2.4/setuptools", which on my computer is simply a
> "__init__.py" and "__init__.pyc" file. I've copied the cli.exe from
> "<workingenv dir>/\lib\python2.4\setuptools-0.6c5-py2.4.egg\setuptools".
> Anyway, it still seems to work, this is more of an FYI.
Hi Jos. I'm not really sure what cli.exe is, but then I don't really
use Windows. Maybe someone here can explain more? Hrm... or maybe my
monkeypatch of setuptools is breaking the attempt to find cli.exe... :(
(I still really wish I didn't have to monkeypatch setuptools, but I do)
Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org
At 08:20 PM 1/15/2007 +0100, Arve Knudsen wrote:
>So I tried 'include_package_data' instead, after having extended
>setuptools with support for the Mercurial VCS. This made sdist include
>also each and every version-controlled Python file however. Is this the
Yes. Why would you *not* include them?
>If there's any interest I can provide the source for my Mercurial
>extension (simple as it may be).
May I suggest that you offer it as a Cheeseshop package instead? Then
anyone who wants to use it can just install it. (I'm assuming, of course,
that you wrote the extension as a "setuptools.file_finders" entry point.)
At 02:16 PM 1/15/2007 +0100, Arve Knudsen wrote:
>I'm experiencing a problem when using setuptools (0.6c5) to generate a
>source tarball, using the 'sdist' command. What happens is that data files
>(PNGs) are omitted even though I include them in package_data (keyword to
>setup), and they are correctly included when installing. Is it the
>intended behaviour of 'sdist' to ignore package_data, or is this a bug?
They should be included in the sdist. Are you using CVS or Subversion, and
if so, are the .png files under revision control? That information will
help me narrow down what the problem is.
I was wondering why there is no --version/-V option available for
easy_install to get the setuptools version currently installed (I would
welcome that option). Is there any specific reason for this or is there
an easier, "better" way to get that information? And by easier, I do not
mean digging into the site-packages directory or starting a Python
interactive shell and reading the `setuptools.__version__` attribute.
Also, the --help-commands option, as advertised by easy_install,
Common commands: (see '--help-commands' for more)
is not recognized.
This was experienced with setuptools 0.6c5 and Python 2.5 on Windows XP.