Mailman 3 September 2005 - Distutils-SIG

setuptools unittests on Mac OS X
by Ashley Walsh 29 Sep '05

29 Sep '05

Just a small fix to make the setuptools.tests.test_resources.DistroTests.testCollection pass on OSX (and other non-win32 platforms): diff -r1.25 test_resources.py 26c26 < ad = Environment([], python=None) --- > ad = Environment([], platform='win32', python=None) Without this the Environment uses the get_platform() result, and Distribution.from_filename("FooPkg-1.4-py2.4-win32.egg") is not added to the environment. (Maybe some can_add() tests would help?) Cheers, Ashley

2 1

Re: [Distutils] Recursive package data
by Phillip J. Eby 28 Sep '05

28 Sep '05

At 01:25 PM 9/28/2005 -0700, Ben Bangert wrote: >As I'm looking at setuptools to handle distribution and usage of web >application's, being able to package arbitrarily deep static content >that belongs with the webapp package (of various extension types) >starts cropping up. Having no way to just say, "everything under this >dir is package data, keep it" becomes a real killer in this context. > >I think I heard somewhere that this feature is being discussed/ >implemented, when will it be available? (or is it not being developed?) I originally thought I could just slap in detection via revision control metadata, but it turned out to be problematic. Doing "sdist" this way is not a problem, because the generated sdist doesn't need to know how its contents were generated - you don't generally build another sdist from an sdist. But doing this for data files would mean the information used to generate the list would be lost to source distributions, which would be Very Bad, because you then couldn't generate a bdist_egg from an sdist. So, the current status is that I'm a bit stumped as to how to implement the feature sanely, and was hoping for some type of brainstorm that would make it easier. :) Really, I suppose one way to do it would be to just include *everything* in package directories (minus CVS/.svn subdirs or other specialized exclusions), but this quickly seems to be duplicating the MANIFEST and MANIFEST.in mechanism of the distutils. In short, for right now I recommend simply writing your own routine(s) to generate file lists or a package_data dictionary, and I'd be interested in seeing different approaches and trying to get some kind of synthesis together in setuptools, assuming I don't come up with a clean way to unify the manifest stuff with this.

2 1

pkg_resources: feature request
by Ian Bicking 28 Sep '05

28 Sep '05

I just added a WSGI app in paste.urlparser.PkgResourceParser (http://svn.pythonpaste.org/Paste/branches/mainline-refactor/paste/urlparser…) to serve static files using pkg_resources's resource API. Two new methods would be useful there: resource_mtime and resource_size. Alternately, resource_stat, which would return an os.stat-compatible object. But a lot of what os.stat returns doesn't apply well to a zip file entry, even if mtime and size do. I'm not sure if mtime should be the mtime of the zip file itself, or the date_time of the entry. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

2 1

Improving distutils' script and GUI app handling
by Phillip J. Eby 27 Sep '05

27 Sep '05

Every so often, there's a perennial debate on the distutils-sig about script filenames. Should they have .py extensions or not? What about .pyw apps on Windows? What about Mac OS? and so on. It occurred to me this morning that we now have a new tool for resolving this issue in setuptools' "entry points" feature. For various reasons, it's common practice to write scripts as Python modules with a "main" function of some kind. These modules are then run directly with '-m', or use "if __name__=='__main__'", or have a short script that imports the main function and runs it. So, if these "main" functions were simply declared as entry points in the project's setup script, then EasyInstall could automatically generate stub scripts for them, in a platform-appropriate fashion, with no need for '-m', "__name__=='__main__'", or fiddling with file extensions. For example, if PyUnit were distributed as an egg, with the following entry points: [distutils.console_apps] unittest = unittest:main Then EasyInstall could create a 'unittest' script with a #! line on Unix-like OSes, and a 'unittest.py', 'unittest.bat', or 'unittest.exe' on Windows. In each case, the generated program would simply load and run the entry point with no arguments. Similarly, there could be a "distutils.gui_apps" entry point group, which could be handled differently according to the target platform. (For example, by creating desktop or menu shortcuts on Windows.) And, tools that create standalone applications or installers could use this information to create other kinds of wrappers around the same entry points. In order for this to work well, there are obviously some details to be worked out. Unix-like OSes are pretty obvious: #! line and no extension should work fine for all app types' executables. But adding desktop icons or menu items for GUI apps is very platform-specific (e.g. Gnome vs. KDE). So, whatever mechanism is used needs to be extensible and configurable. Ideally, it should be possible for the platform's Python installation to provide the necessary hooks to do this, because I personally don't want to have to write and maintain all that code. :) As for Mac OS, I have almost no experience with it, so I'm not sure what GUI applications there need. Does everything need py2app? If you have a wx-based app, would you just make a #! script? Bob Ippolito previously mentioned that you don't "install" applications there, that people just drag applications wherever they want them rather than using shortcuts, so at least that part isn't a problem. :) On Windows, I'd say that applications are pretty much always better as .exe's, whether console or GUI. The .py/.pyw form is dependent on a single global consistent version of Python, but it's possible and reasonable to have multiple Python versions installed. An .exe also has a lot more control over how Python is initialized, and that can be particularly important for applications. On the other hand, in the short run I can also see using .bat or .cmd files for console apps, and .pyw for GUI apps, just to have something that works and wait for the path management use cases for various .exe options to work themselves out. Anyway, my idea here is that when using setuptools, you would define entry points instead of creating scripts and listing them in setup(). Then, using either EasyInstall or "setup.py install" would automatically create platform-appropriate scripts. Some of the open questions: * What groups (if any) should exist besides "console_apps" and "gui_apps"? * How can we allow easy control of installation options on the target system? (e.g., I only want Windows Programs Menu items, you only want desktop shortcuts for KDE, etc., but we can't have six billion command line options, and all of this stuff can't go into setuptools anyway) * Should launchers hardcode their sys.path? This would prevent breakage caused by installing new and incompatible versions of their dependencies, but require an explicit upgrade action to update them, and make the installation process more complex. Thoughts, anyone?

8 22

How use Distutils for plugins ?
by william＠opensource4you.com 27 Sep '05

27 Sep '05

I've discovered the Distutils mailing list after my post to comp.lang.python http://groups.google.be/group/comp.lang.python/browse_thread/thread/b66d6ea… Does any one have an idea how to use distutils to install sub-elements (f.e. plugins) of a main packages (already installed). Thanks. -- William: http://www.opensource4you.com

2 1

easy_install question
by Thomas Heller 26 Sep '05

26 Sep '05

- When I run 'easy_install ctypes', the behaviour depends on the current directory. If I run this when the current dir contains a 'ctypes' subdirectory, it doesn't search pypi^H^H^H^HCheeseShop for this name, instead it seems to attempt to install from source in that directory. Is that intended? Can I override this behaviour? - When I install a package which requires another package is there a way to specify a download url for the required package? Thomas

2 2

Non-root install for testing
by Ian Bicking 26 Sep '05

26 Sep '05

I've created a script to set up a non-root installation of Python, suitable for testing installation procedures. It follows the instructions located here: http://peak.telecommunity.com/DevCenter/EasyInstall#non-root-installation The script is at: http://svn.colorstudy.com/home/ianb/non_root_python.py To do testing I've created a new user on my system, so I don't mix things up with my normal development, and so I have nothing to lose ;) I use it like: /usr/bin/python non_root_python.py --no-site-packages --clear I doubt it will work on a Windows system, but it should work on a Mac. You can run it with --clear to reset the system quickly, so you can restart your installation process from zero. --no-site-packages doesn't copy anything from the standard site-packages dir (unlike in the instructions), so you can start out even more bare. But you can use that or not, it shouldn't cause a problem either way. (Well, it *can* cause problems, but interesting problems; like I have cheetah installed with a Debian package, so if I copy site-packages I get an error related to that -- I'm not sure what to do about that at this point). Just make sure ~/bin is first in your $PATH. Ian

2 2

setuptools 0.6a4 bug fix release
by Phillip J. Eby 25 Sep '05

25 Sep '05

I've released 0.6a4, to fix the require() bug in 0.6a3.

1 0

setuptools: bug in run_script
by Ian Bicking 25 Sep '05

25 Sep '05

Ben Bangert encountered a problem where pkg_resources.WorkingSet.run_script didn't work; from line 407: self.require(requires)[0].run_script(script_name, ns) If all the packages are already loaded for some reason, then self.require() returns []. I don't really understand how we got in this state -- that apparently packages are available before they are being required. But so it is; it might be from a script wrapper created with 0.6a2?

2 4

Re: [Distutils] setuptools: bug in run_script
by Phillip J. Eby 25 Sep '05

25 Sep '05

At 01:00 PM 9/25/2005 -0500, Ian Bicking wrote: >Ben Bangert encountered a problem where >pkg_resources.WorkingSet.run_script didn't work; from line 407: > > self.require(requires)[0].run_script(script_name, ns) > >If all the packages are already loaded for some reason, then >self.require() returns []. > >I don't really understand how we got in this state -- that apparently >packages are available before they are being required. That's actually normal, for packages that are already on sys.path (via easy-install.pth). What's not normal is that require() should never return []. As it turns out, it's a bug in an optimization I added for 0.6a3 that avoids creating an Environment if all requirements can be satisfied from the working set. It ends up not adding the relevant packages to the list it returns. Here's a patch to fix it until I get a bugfix release out: Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.72 diff -u -r1.72 pkg_resources.py --- pkg_resources.py 24 Sep 2005 19:48:28 -0000 1.72 +++ pkg_resources.py 25 Sep 2005 18:57:07 -0000 @@ -463,7 +463,7 @@ requirements = list(requirements)[::-1] # set up the stack processed = {} # set of processed requirements - best = dict([(d.key,d) for d in self]) # key -> dist + best = {} # key -> dist to_activate = [] while requirements: @@ -471,20 +471,20 @@ if req in processed: # Ignore cyclic or redundant dependencies continue - dist = best.get(req.key) if dist is None: # Find the best distribution and add it to the map - if env is None: - env = Environment(self.entries) - dist = best[req.key] = env.best_match(req, self, installer) + dist = self.by_key.get(req.key) if dist is None: - raise DistributionNotFound(req) # XXX put more info here + if env is None: + env = Environment(self.entries) + dist = best[req.key] = env.best_match(req, self, installer) + if dist is None: + raise DistributionNotFound(req) # XXX put more info here to_activate.append(dist) elif dist not in req: # Oops, the "best" so far conflicts with a dependency raise VersionConflict(dist,req) # XXX put more info here - requirements.extend(dist.requires(req.extras)[::-1]) processed[req] = True

1 0