Mailman 3 April 2007 - Distutils-SIG

Re: [Distutils] Determining system packages
by Phillip J. Eby 13 Apr '07

13 Apr '07

At 10:24 AM 4/13/2007 -0700, Rob wrote: >Hi, > >I'm trying to determine which packages are "system" packages as >opposed to development or regular packages for yolk[1]. > >An example of what I mean would be wsgiref which comes included with >Python 2.5. > >The closest thing I can find that would help in a pkg_resources >Distribution object is the precedence attribute, but system and >development seem to be the same and the precedence would only help if >there were more than one in the environment. > >If I use get_python_lib from distutils.sysconfig and remove the >trailing "site-packages" from the reult You mean 'get_python_lib(standard_lib=1)'? > and check a distribution'slocation attribute to see if its installed > there, will it be reliable >or might system packages be in other locations? System packages are just develop eggs installed in "system" locations. System locations may be Python-defined or OS-defined. For example, some distros have non-standard directory layouts, such that there is both a "/usr/lib/python2.x/site-packages" and a "/usr/local/lib/python2.x/site-packages", and *either* could contain system packages. There is a trick you could use to identify "develop" distributions, though... if there is a "SOURCES.txt" metadata file associated with an egg, you can look for a setup.py in a parent directory whose directory layout corresponds to that described by the contents of SOURCES.txt. This could *potentially* produce a false positive, though, if the package in question puts all its modules in the project root and doesn't have any other files besides setup.py listed in the project root. (The long-term plan for 0.7 is that packages that are actually "installed" will have some kind of installation manifest ala PEP 262.) By the way, under Python 2.5, packages installed by the distutils include an .egg-info *file*, so that is a giveaway that a package was installed using distutils (and maybe a system packaging tool), and is definitely *not* a "develop" package.

2 1

Determining system packages
by Rob 13 Apr '07

13 Apr '07

Hi, I'm trying to determine which packages are "system" packages as opposed to development or regular packages for yolk[1]. An example of what I mean would be wsgiref which comes included with Python 2.5. The closest thing I can find that would help in a pkg_resources Distribution object is the precedence attribute, but system and development seem to be the same and the precedence would only help if there were more than one in the environment. If I use get_python_lib from distutils.sysconfig and remove the trailing "site-packages" from the reult and check a distribution's location attribute to see if its installed there, will it be reliable or might system packages be in other locations? Thanks, Rob [1] http://tools.assembla.com/yolk

1 0

setuptools and packages with dots
by Benji York 13 Apr '07

13 Apr '07

While working on a package I decided to do some old-school revision control and make a copy of the directory. I recursively copied the package to PACKAGENAME.bak1. I then ran setup.py develop (indirectly though zc.buildout) and received an intriguing error message. Here are reproduction steps that don't use buildout. % mkdir foo % cd foo % mkdir bad.dir % touch bad.dir/__init__.py create setup.py as such: from setuptools import setup, find_packages setup( name = 'test', packages = find_packages('.'), install_requires = ['setuptools'] ) % python setup.py develop running develop running egg_info writing requirements to test.egg-info/requires.txt writing test.egg-info/PKG-INFO writing top-level names to test.egg-info/top_level.txt writing dependency_links to test.egg-info/dependency_links.txt error: package directory 'bad/dir' does not exist It looks like somewhere in setuptools paths are being normalized to package names and back. The dot doesn't survive the round trip. -- Benji York Senior Software Engineer Zope Corporation

2 3

Please reply to confirm:1176369433.1060_11866.navsta message and allow delivery
by Friends system 12 Apr '07

12 Apr '07

This message was automatically generated for Murray <murray(a)muzmo.com>: The email system that we are using only accept emails from email addresses on personalized 'Lists of Friends' and you are not currently included on my 'List of Friends'. Your ommission from the list is probably due to an oversight on my part. We have added this new mechanism to reduce the amount of unwanted email by attempting to verify that the sender is a human. In the meantime, we are regularly adding new addresses to the 'List of Friends' If you know that I would really want to hear from you: Please reply to this message without changing the subject line, or even adding any text. I will then receive your email and be able to add you to my 'List of Friends' so that, hopefully*, you will not be bothered by this automated reply again, and I will receive your all your emails in the future. * If you have multiple email addresses that you would like me to add to my 'List of Friends', then please send me an email from each account. Thanks, Murray <murray(a)muzmo.com> For more information see http://netwinsite.com/surgemail/friends.htm

1 0

spurious warning when multiple ez_setup instances are present
by Antoine Pitrou 11 Apr '07

11 Apr '07

Hi, I occasionally get this warning when using a setuptools-enabled setup.py from a third-party package: /usr/lib/python2.5/site-packages/setuptools/command/sdist.py:3: UserWarning: Module ez_setup was already imported from /home/antoine/nose/config-file/ez_setup.pyc, but /home/antoine/XXX is being added to sys.path It seems to me that this warning conveys no useful information (yes, several independent packages can have their own copy of ez_setup.py... so what?), and that it would be better to remove it. Regards Antoine.

2 4

Buildout site-dirs and develop eggs
by Robin Bryce 10 Apr '07

10 Apr '07

Hi, When buildout installs develop eggs how does it setup --site-dirs ? I've tried having a rummage in buildout.py but I don't understand what the Buildout._develop method is doing; It appears to be preparing a environ dict - presumably in preparation for execing the moral equivalent of 'python setup.py develop' - but it does not pass this to zc.buildout.easy_install.develop. It _does_ pass in dest. I can see this comes from buildout['buildout']['develop-eggs-directory']; but it doesn't appear to do anything special to propagate this to --site-dir. I have sometimes works sometimes doesn't symptoms (see below) - Is there current working dir nonsense going on here ? The relevant versions are: zc.buildout.easy_install.picked: zc.buildout = 1.0.0b23 zc.buildout.easy_install.picked: setuptools = 0.6c5 I'm getting this error from running "buildout -vvv install": buildout: Develop: /home/robin/devel-asycamore/recipes zc.buildout.easy_install: in: /home/robin/devel-asycamore/recipes ['/tmp/tmpzqO9yC', '-q', 'develop', '-mxN', '-d', '/home/robin/devel-asycamore/./tmpxiPaPrbuild'] buildout: Develop: /home/robin/devel-asycamore/asycamore-trunk zc.buildout.easy_install: in: /home/robin/devel-asycamore/asycamore-trunk ['/tmp/tmp06R7ze', '-q', 'develop', '-mxN', '-d', '/home/robin/devel-asycamore/./tmppOiykWbuild'] error: /home/robin/devel-servers (in --site-dirs) is not on sys.path While: Installing Processing develop directory /home/robin/devel-asycamore/asycamore-trunk An internal error occured ... This is my first pass through working with zc.buildout and I think my buildout.cfg suspect. I have a recipe that does an svn co/up, a part that 'installs' sources using it, and dependent part that builds a release egg. At the very top I'm also listing the 'part' produced svn checkout as a 'develop' egg. As develop eggs get built before non develop eggs I dont think this will work in a single pass but even so the --site-dirs error was not what I was expecting. Is there a 'right way' tm to have a buildout install a develop link based on a part produced checkout ? I got some hints from this thread: http://mail.python.org/pipermail/distutils-sig/2007-January/007160.html but didn't see any clear recommendations. My buildout - such as it is - can be found here: http://svn.wiretooth.com/svn/open/asycamore.developer/trunk/ Cheers, Robin

1 1

spurious warning when multiple ez_setup instances are present
by Antoine Pitrou 10 Apr '07

10 Apr '07

Hi, I occasionally get this warning when using a setuptools-enabled setup.py from a third-party package: /usr/lib/python2.5/site-packages/setuptools/command/sdist.py:3: UserWarning: Module ez_setup was already imported from /home/antoine/nose/config-file/ez_setup.pyc, but /home/antoine/XXX is being added to sys.path It seems to me that this warning conveys no useful information (yes, several independent packages can have their own copy of ez_setup.py... so what?), and so that it would be better to remove it. Regards Antoine.

1 0

My Cheeseshop mirror
by Jon Rosebaugh 10 Apr '07

10 Apr '07

As Matt Good mentioned in the "Time for a static cheeseshop mirror for easy_install?", I started working on a cheeseshop mirror during one of the downtimes last week. It's now up and running in a limited capacity: http://cheeseboard.inklesspen.com/ The immediate goal of the "cheeseboard" is to enable people to install setuptools, Pylons, Mako, and other packages in the Pylons family when the cheeseshop is down, without having to jump through more than one or two hoops. It's a dynamic webapp written in Pylons, but if the load gets to be too big, it's easy to cache almost all of it. The files themselves are stored on Amazon S3; I'll have to see if that's cost-effective for this. Currently it is a manually-updated partial mirror. You have to add a 'package', which consists of the name and optionally SVN info and homepage, and then feed it the cheeseshop url for a specific release, such as <http://cheeseshop.python.org/pypi/Pylons/0.9.4.1>. It then copies all eggs, tarballs and zip files to S3 and enters the relevant info in the database. I've written it in a manner that should be usable by other people, but the SVN repository is not currently world-readable, and I haven't gotten around to uploading it to the cheeseshop yet. But if there's interest, I could do these things. In the future, I expect to add functionality for adding a package and all its dependencies automatically. I don't expect to be adding functionality for mirroring the complete cheeseshop, though, partially due to not having a good idea for how to do that automatically. Jon

1 0

Time for a static cheeseshop mirror for easy_install?
by Phillip J. Eby 09 Apr '07

09 Apr '07

For what easy_install does, there really isn't any dynamic API usage, so a static mirror for easy_install could take a good bit of load off the cheeseshop. I don't know whether this will actually solve any problems the cheeseshop itself is having; it may be that ill-behaved web spiders are at fault, or something else altogether. However, since the downtime mostly creates issues for people using easy_install, creating a solution for those people certainly seems worthwhile. Since easy_install was designed to be able to use simple directory indexes and HTML pages as a package index, it should be possible to create a simple directory tree of HTML pages, using PyPI's public XML-RPC API. The mirror could use PyPI's RSS feed to know when a package's information is out of date, although I'm not sure that the RSS includes all modifications, such as when packages are deleted, releases are hidden, files uploaded, etc. However, assuming that there's a scalable way to receive change notifications, it should be straightforward to implement a mirror script for easy_install, and have it run on one or more volunteered hosts, perhaps with round-robin DNS (maybe easy-install.python.org?) I'll be happy to assist anyone who wants to work on this, including updates to easy_install itself, of course. I'd actually be hacking on this now, if the cheeseshop weren't down (i.e., I can't download any XML-RPC data to do prototyping at the moment!)

6 8

Re: [Distutils] [PATCH] Add Monotone support to setuptools
by Phillip J. Eby 07 Apr '07

07 Apr '07

At 11:11 PM 4/6/2007 -0400, Dale Sedivec wrote: >On Fri, Apr 06, 2007 at 10:57:12PM -0400, Phillip J. Eby wrote: > > At 08:02 PM 4/6/2007 -0400, Dale Sedivec wrote: > > > Greetings. Per the setuptools docs, below is patch to add > > >support for the Monotone revision control system[1] to setuptools. > > > > May I suggest, that instead of distributing this as a patch, you simply > > create a package that contains this support? Then, anyone using Monotone > > can simply easy_install your package. I would prefer not to have to > > support this code in the main setuptools distribution. > > That was actually my original intent, but then I saw the >setuptools documentation suggested "submitting a patch to the >setuptools.command.sdist module," Oops -- I missed taking that out when I added a plugin interface. (Originally, patching was the only way to add support for a new source control system.) Thanks for pointing this out; I'll fix the docs pronto. :)

1 0