Mailman 3 October 2007 - Distutils-SIG

Ignoring site-packages when running zc.buildout tests
by Ross Patterson 31 Oct '07

31 Oct '07

I have python-setuptools installed through my OS and zc.buildout installed with easy_install. I'd like to be able to ignore these when running the zc.buildout tests (or tests for other projects that handle distributions) so that the system packages don't interfere with the testing environment. Any ideas? Should the zc.buildout tests be modified or should my system? Ross

2 5

patch: Unused import of 'shutil'
by Ben Finney 30 Oct '07

30 Oct '07

Howdy all, As of version 0.6c7 (and likely earlier), 'ez_setup.py' causes 'pyflakes' to emit two warnings:: $ pyflakes ./ez_setup.py ./ez_setup.py:117: 'shutil' imported but unused ./ez_setup.py:181: redefinition of unused 'main' from line 179 The latter is due to 'pyflakes' not handling an ImportError-based import fallback, as documented in ticket 1499 in the pyflakes Trac <URL:http://divmod.org/trac/ticket/1499>. The former is fixed with this patch: --- ez_setup.py 2007-10-29 13:36:24.000000000 +1100 +++ ez_setup.py 2007-10-29 13:36:37.000000000 +1100 @@ -114,7 +114,7 @@ with a '/'). `to_dir` is the directory where the egg will be downloaded. `delay` is the number of seconds to pause before an actual download attempt. """ - import urllib2, shutil + import urllib2 egg_name = "setuptools-%s-py%s.egg" % (version,sys.version[:3]) url = download_base + egg_name saveto = os.path.join(to_dir, egg_name) -- \ "Reichel's Law: A body on vacation tends to remain on vacation | `\ unless acted upon by an outside force." -- Carol Reichel | _o__) | Ben Finney

2 1

Object loading spec
by Ian Bicking 30 Oct '07

30 Oct '07

I mentioned some time ago that it would be nice if everyone could agree how to configure object references. Mostly I just had in mind having Paste Deploy and zc.buildout use the same syntax, but it's something that comes up in ad hoc situations often as well. I wrote up a small implementation to push this idea forward a bit. I'm hoping to get some feedback here. In an unfortunate mishmash of syllables (name suggestions welcome) I named it obconfloader. You can read the description here: http://svn.pythonpaste.org/ObConfLoader/trunk/docs/index.txt I've also copied that description into this email. Status & License ---------------- ObConfLoader is under an `MIT-style permissive license <http://svn.pythonpaste.org/ObConfLoader/trunk/docs/license.txt>`_. Discussion should occur on `distutils-sig <http://www.python.org/community/sigs/current/distutils-sig/>`_; please be sure to put "ObConfLoader" in the subject line. The package is available in a `subversion repository <http://svn.pythonpaste.org/ObConfLoader/trunk#egg=ObConfLoader>`_, and the trunk can be installed with ``easy_install ObConfLoader==dev``. You can get a checkout with:: svn co http://svn.pythonpaste.org/ObConfLoader/trunk ObConfLoader Introduction ------------ ObConfLoader allows you to load objects from strings, typically for use in config files (or from the command line, or other locations where object references are necessary but the format is constrained). It also allows you to utilize `Setuptools entry points <http://peak.telecommunity.com/DevCenter/setuptools#entry-points>`_ to make public objects that can be referenced. The concept of an entry point group, or API, allows backward compatibility when using entry points. The basic usage is very simple:: from obconfloader import load_ob obj, ep_group = load_ob(a_string, ['mypackage.api']) assert ep_group is None or ep_group == 'mypackage.api' This loads the object from the given string, returning the object and the API it supports. The second argument is a list of entry point groups that you support. You can support multiple entry point groups, or no groups (by leaving out the second argument, or using None). If you leave out the second argument ``ep_group`` will always be None. There are several possible exceptions: ``LoadError``: The parent of all other exceptions; a generic error. ``ConfigSyntaxError``: The string itself is malformed. ``BadGroupError``: The group given doesn't match up with an acceptable input group. ``NotFoundError``: The referenced object can't be found (something like DistributionNotFound, ImportError, AttributeError). ``PythonError``: Some error with a Python expression, Python syntax, etc. Specification Format -------------------- There are several formats for strings: ``file /path/to/file.py`` This loads the given file, and returns the module object. ``file /path/to/file.py:dotted.name`` This loads the given file, and returns the object ``dotted.name`` inside it. ``file /path/to/file.py:dotted.name [entry.point.group]`` This loads the object, and indicates that the object supports the group ``entry.point.group``. This can be used when multiple APIs are supported, and the object doesn't use the default API. ``python module.name:dotted.name`` This loads the module ``module.name``, and returns the ``dotted.name`` object. Of course you can leave out ``dotted.name`` to return the module, and also add an entry point group. ``egg Distribution`` This loads the distribution (the package) ``Distribution``, and gets the ``main`` entry point, with the given entry point group. ``egg Distribution:entry_point_name`` Instead of the ``main`` entry point, this gets ``entry_point_name``. Other Schemes ------------- You can pass in a dictionary of extra schemes (besides ``file``, ``python``, and ``egg``) with the ``extra_schemes`` argument. All schemes are converted to lower case. The signature of a loader is:: def scheme_loader(string, groups, orig_string, position, group): if problem: raise LoadError("message", orig_string, position) return loaded_obj, group The meaning of the arguments: ``string``: The string you should load, like ``"/path/to/file.py:dotted.name"``. This leaves out the trailing group and the scheme. ``groups``: The entry point groups accepted. Schemes besides ``egg`` probably will ignore this. ``orig_string``: The full string, with scheme. Used primarily with any ``LoadError`` exceptions, to give the user back the full string they entered. ``position``: The position where the string was found; also used with exceptions. ``group``: If the string contained an explicit group, this will be that group (otherwise None). If you don't have any explicit notion of groups then you can just do the return as in the example. You should raise LoadError exceptions when at all possible; failures generally *are* expected. Open Issues ----------- * Just using a different data source (like a database) for Python code instead of a file isn't super-easy (you'd have to provide some kind of extra scheme). Maybe a little refactoring of ``load_file`` would make it easier. * Possibly ``load_file`` itself should have some resource-finding pluggability. This could be used to restrict the files that can be accessed. * Disabling a scheme by setting ``extra_schemes=dict(file=None)`` would be handy, but isn't clean now. * The form ``python module.name.object.name`` *looks* reasonable, but it has to be ``python module.name:object.name``. The error message isn't very helpful either (basically an import error). Probably it should Just Work. * I don't scan the system for extra kinds of schemes. No pluggability. I am shying away from doing complete entry point scans (e.g., with ``pkg_resources.iter_entry_points``) because of efficiency concerns. * There's no way to indicate an egg entry point name without distribution. Maybe ``egg *:ep_name`` should be allowed? This is inefficient, but the ``*`` makes it *look* inefficient too. * Should an implied scheme be allowed? E.g., something like Buffet (a template abstraction layer) basically needs an object reference to the renderer. Right now this *must* be an egg/entry point, but there's no reason it couldn't use other kinds of references. But the ideal specification would just be ``Distribution``, not ``egg Distribution``. A default scheme of egg would fix this. * There's nothing for non-Python-object resources, like files or templates. This is probably fine. * There's no built-in indirection. Ideally such indirection would happen in the config file or elsewhere in the system. * Some exceptions are swallowed, though an attempt is made to swallow only the most boring exceptions. * There are some FIXME's in the code.

1 0

Distributing doc files w/ distutils src pkg?
by Skip Montanaro 28 Oct '07

28 Oct '07

I have a simple setup.py file: #!/usr/bin/env python from distutils.core import setup setup(name='lockfile', author='Skip Montanaro', author_email='skip <at> pobox.com', maintainer='Skip Montanaro', maintainer_email='skip <at> pobox.com', url='http://www.webfast.com/~skip/python/', download_url='http://www.webfast.com/~skip/python/lockfile.tar.gz', version='0.1', py_modules=['lockfile'], data_files=[('doc', ['lockfile.rst'])], ) I've used the data_files arg to setup to note where the docs live. Alas, the sdist command doesn't include this in the tar file: % tar tfz dist/lockfile-0.1.tar.gz lockfile-0.1/ lockfile-0.1/lockfile.py lockfile-0.1/PKG-INFO lockfile-0.1/README lockfile-0.1/setup.py I'm a complete distutils novice. I can't for the life of me understand why the doc directory isn't included in the tar file. As far as I can tell distutils has no explicit provision at all for distributing documentation, which seems very weird. What am I missing?

4 3

zc.buildout installing parts not asked for
by TIm Terlegård 28 Oct '07

28 Oct '07

When using zc.buildout I discovered that it installed a part that I didn't specify in the 'parts' option. This happened because I referenced this part somewhere else. Is this how it's supposed to be? I would prefer if it only installs the parts I specify in parts. I don't see anything wrong with reusing options from a part that I don't necessarily want to install. This is a buildout.cfg that will reproduce it. The 'notusedinparts' part is installed even if not specified in 'parts'. It's installed because of the 'dummy' option in the 'tok' section. Is this how it was meant to be? [buildout] develop = recipes parts = tok [tok] recipe = recipes:debug dummy = ${notusedinparts:someoption} [notusedinparts] recipe = recipes:debug someoption = why /Tim

2 1

Re: [Distutils] Setuptools pydoc problem
by Phillip J. Eby 26 Oct '07

26 Oct '07

At 11:13 AM 10/25/2007 -0500, Ian Bicking wrote: >I'm getting this error with setuptools trunk and pydoc/help(): > >>>>>import unittest >>>>>help(unittest.main) >>Traceback (most recent call last): >> File "<stdin>", line 1, in ? >> File >> "/usr/lib/python2.4/site-packages/setuptools-0.6c7-2.4.egg/site.py" >> , line 339, in __call__ >> >> File >> "/home/ianb/env/webdev/lib/python2.4/setuptools-0.7a1dev_r58373-py2.4.egg/pydoc.py", >> line 1645, in __call__ >> self.help(request) >> File >> "/home/ianb/env/webdev/lib/python2.4/setuptools-0.7a1dev_r58373-py2.4.egg/pydoc.py", >> line 1689, in help >> else: doc(request, 'Help on %s:') >> File >> "/home/ianb/env/webdev/lib/python2.4/setuptools-0.7a1dev_r58373-py2.4.egg/pydoc.py", >> line 1468, in doc >> desc = describe(object) >> File >> "/home/ianb/env/webdev/lib/python2.4/setuptools-0.7a1dev_r58373-py2.4.egg/pydoc.py", >> line 1413, in describe >> if inspect.isgetsetdescriptor(thing): >>AttributeError: 'module' object has no attribute 'isgetsetdescriptor' > >I'm guessing its something in Python 2.5's inspect but not 2.4. Fixed. (Please don't send setuptools bug reports to me in private email; the distutils-sig is where they go, to minimize duplicates, have a public record, and at least entertain the possibility that someone else might know of a fix, workaround, or be able to contribute a patch.)

1 0

buildout and console scripts in dependencies
by Martin Aspeli 25 Oct '07

25 Oct '07

Hi, I have a buildout which looks like this: [buildout] parts = repoze find-links = http://dist.repoze.org eggs = repoze.plone develop = [repoze] recipe = zc.recipe.egg eggs = ${buildout:eggs} Here, repoze.plone is an egg depending on a bunch of other eggs. Some of those dependencies, such as PasteScript and repoze.zope2, have console_scripts entry points. With the buildout above, I don't get any console scripts though. To get that, I have ot list the eggs with dependencies explicitly in the zc.recipe.egg part: [buildout] parts = repoze find-links = http://dist.repoze.org eggs = repoze.plone develop = [repoze] recipe = zc.recipe.egg eggs = ${buildout:eggs} repoze.zope2 PasteScript Is this a bug or feature? It's certainly a bit annoying, since I now have to work out which packages have the console scripts I need. :) Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

2 1

Re: [Distutils] [Enthought-dev] Building Fedora RPMs
by Stanley A. Klein 23 Oct '07

23 Oct '07

On Mon, 2007-10-22 at 18:15 -0500, Dave Peterson wrote: > Stanley A. Klein wrote: > > Other failures included: > > > > endo - unpackaged pyc and pyo files (looks like a setup.py issue) > > > > I get rpms built for this just fine. Are you sure you have no local changes? > The problem with this one may be a setuptools issue. The failure in doing the bdist_rpm is on a file endo.py, which is identified to setuptools in the setup.py as a script. The file is originally in enthought/endo/scripts. The pyc and pyo files get built there properly because of the [install] optimize=1 option in setup.cfg. However, the INSTALLED_FILES also lists a /usr/bin/endo.py without the pyc and pyo files. The Fedora rpm system causes those optimization files to be built unbeknownst to setuptools, which then causes the installed but unpackaged files error. I tried declaring the pyc and pyo files in the script statement, but at that point they don't exist and I got an error to that effect. There is probably a workaround, like commenting out the scripts statement in the setup.py and providing rpm a post-install script to copy endo.* into /usr/bin. However, it would create separate setup.py files for rpm and non-rpm packaging. Stan Klein

3 4

Dependency extensions
by Tres Seaver 23 Oct '07

23 Oct '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have use cases for a couple of extensions to how requirements are spelled in a setuptools-enabled project. Overriding Requirements - ----------------------- I would like to be able to require package 'bar', but override one or more of its dependencies (typically, 'bar' has over-strict version qualifiers, which I need to relax). I can imagine spelling such an override by including the overridden project with a qualifier, e.g.: in foo/setup.py: install_requires=['bar', '!baz==1.3', #override bar's spec of 1.2.4 ] The "bang" here is borrowed loosely from CSS; other spellings might be clearer. Another spelling might be to pass 'nodeps' or something as a suffix to the overly-strict package, e,g,:: install_requires=['bar (nodeps=baz)', 'baz==1.3', ] In general, I think the principal that the "most dependent" package wins would be a reasonable policy, although not backward compatible at this point. Pinning Download Pages - ---------------------- I would like to be able to "pin" a given package to a particular index / download page, rather than searching the standard locations. I could imagine either supplying the "pinned" find_url as a suffix, e.g.: install_requires=['baz (http://example.com/download-baz-here/)', ] Or else supplying a mapping which is used for the pinned URLs:: install_requires=['baz'], pinned_urls={'baz': 'http://example.com/download-baz-here'} The second might be easier to implement, but feels clunkier. Thoughts? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHDoKC+gerLs4ltQ4RAqR0AJ99D44ob2Od0WecAW05E+MGcDruOgCgo7zV q7XZxbyhFeh9engIsX0p/Rs= =9bFP -----END PGP SIGNATURE-----

3 5

"activate" script for virtualenv
by Matt Good 21 Oct '07

21 Oct '07

I just set up a little script so that I can activate a particular virtualenv and easily exit it. Basically it just invokes a new bash subshell and sets the PATH to include the current bin dir: mgood@sjl-dev1:~/python-envs/trac$ cat activate bash --init-file .bash_init mgood@sjl-dev1:~/python-envs/trac$ cat .bash_init . ~/.bash_profile export PATH="`pwd`/bin:$PATH" So running ./activate will set up your path in the subshell, and exiting will restore your normal PATH. I don't really like running ~/.bash_profile again b/c I actually end up with it re-adding some stuff that's already in the PATH, but otherwise I lost my normal prompt coloring. Someone with more bash-fu know how to fix that? mgood@sjl-dev1:~/python-envs/trac$ which python /usr/local/bin/python mgood@sjl-dev1:~/python-envs/trac$ ./activate mgood@sjl-dev1:~/python-envs/trac$ which python /home/mgood/python-envs/trac/bin/python mgood@sjl-dev1:~/python-envs/trac$ exit mgood@sjl-dev1:~/python-envs/trac$ which python /usr/local/bin/python -- Matt

2 2

Distutils-SIG October 2007