Mailman 3 July 2009 - Distutils-SIG

[issue84] setup.py install: make sys.executable overridable
by Iwan Vosloo 28 Jul '09

28 Jul '09

New submission from Iwan Vosloo <iwan(a)reahl.org>: When running setup.py install --single-version-externally-managed in order to package an egg in, eg, a .deb, the scripts generated from entrypoints contain a line: #!/path/to/python This path is taken from sys.executable. However, this command is often run from inside a virtualenv, in which case sys.executable is not what you want. It would be nice to either: 1) be able to specify a path for the python executable on the commandline, similar to the --prefix option or 2) strip off the contents of the $VIRTUAL_ENV variable out of sys.executable when running with --single-version-externally-managed ---------- messages: 332 nosy: IwanVosloo priority: wish status: unread title: setup.py install: make sys.executable overridable _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue84> _______________________________________________

1 0

setup.py install: overriding sys.executable for scripts?
by Iwan Vosloo 27 Jul '09

27 Jul '09

Hi there, We are running: python setup.py install --single-version-externally-managed in order to package an egg in a .deb. However, we run this inside a virtualenv environment, which means that all scripts generated by setuptools start with: #!/path/to/python/in/virtualenv I see that sys.executable is used for generating this line. Is there any way to override it, similarly to how you can override the installation directory with --prefix? Regards - Iwan

2 1

Including data files with setuptools
by Adeel Ahmad Khan 25 Jul '09

25 Jul '09

I am having some trouble including data files in the egg installed by setuptools. I've listed the data files in MANIFEST.in and set include_package_data = True. % ls LICENSE README ez_setup.py timed.py MANIFEST.in cmdapp.py setup.py % cat setup.py from ez_setup import use_setuptools use_setuptools() from setuptools import setup setup( # ... py_modules=['timed', 'cmdapp'], entry_points={'console_scripts': ['timed = timed:main']}, include_package_data=True, install_requires=['PyYAML'], zip_safe=False, ) % cat MANIFEST.in include *.py include README include LICENSE % sudo python setup.py install running install running bdist_egg running egg_info creating timed.egg-info writing requirements to timed.egg-info/requires.txt writing timed.egg-info/PKG-INFO writing top-level names to timed.egg-info/top_level.txt writing dependency_links to timed.egg-info/dependency_links.txt writing entry points to timed.egg-info/entry_points.txt writing manifest file 'timed.egg-info/SOURCES.txt' reading manifest file 'timed.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'timed.egg-info/SOURCES.txt' installing library code to build/bdist.macosx-10.3-i386/egg running install_lib running build_py creating build creating build/lib copying timed.py -> build/lib copying cmdapp.py -> build/lib creating build/bdist.macosx-10.3-i386 creating build/bdist.macosx-10.3-i386/egg copying build/lib/cmdapp.py -> build/bdist.macosx-10.3-i386/egg copying build/lib/timed.py -> build/bdist.macosx-10.3-i386/egg byte-compiling build/bdist.macosx-10.3-i386/egg/cmdapp.py to cmdapp.pyc byte-compiling build/bdist.macosx-10.3-i386/egg/timed.py to timed.pyc creating build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/PKG-INFO -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/SOURCES.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/dependency_links.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/entry_points.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/not-zip-safe -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/requires.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/top_level.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO creating dist creating 'dist/timed-0.11-py2.6.egg' and adding 'build/bdist.macosx-10.3-i386/egg' to it removing 'build/bdist.macosx-10.3-i386/egg' (and everything under it) Processing timed-0.11-py2.6.egg removing '/usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg' (and everything under it) creating /usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg Extracting timed-0.11-py2.6.egg to /usr/local/lib/python2.6/site-packages timed 0.11 is already the active version in easy-install.pth Installing timed script to /usr/local/bin Installed /usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg Processing dependencies for timed==0.11 Searching for PyYAML==3.08 Best match: PyYAML 3.08 Processing PyYAML-3.08-py2.6-macosx-10.3-i386.egg PyYAML 3.08 is already the active version in easy-install.pth Using /usr/local/lib/python2.6/site-packages/PyYAML-3.08-py2.6-macosx-10.3-i386.egg Finished processing dependencies for timed==0.11 % ls /usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg EGG-INFO cmdapp.py cmdapp.pyc timed.py timed.pyc Why isn't README there? What do I need to do to include it?

1 0

Data files with setuptools
by Adeel Ahmad Khan 25 Jul '09

25 Jul '09

I am having some trouble including data files in the egg installed by setuptools. I've listed the data files in MANIFEST.in and set include_package_data = True. % ls LICENSE README ez_setup.py timed.py MANIFEST.in cmdapp.py setup.py % cat setup.py from ez_setup import use_setuptools use_setuptools() from setuptools import setup setup( # ... py_modules=['timed', 'cmdapp'], entry_points={'console_scripts': ['timed = timed:main']}, include_package_data=True, install_requires=['PyYAML'], zip_safe=False, ) % cat MANIFEST.in include *.py include README include LICENSE % sudo python setup.py install running install running bdist_egg running egg_info creating timed.egg-info writing requirements to timed.egg-info/requires.txt writing timed.egg-info/PKG-INFO writing top-level names to timed.egg-info/top_level.txt writing dependency_links to timed.egg-info/dependency_links.txt writing entry points to timed.egg-info/entry_points.txt writing manifest file 'timed.egg-info/SOURCES.txt' reading manifest file 'timed.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'timed.egg-info/SOURCES.txt' installing library code to build/bdist.macosx-10.3-i386/egg running install_lib running build_py creating build creating build/lib copying timed.py -> build/lib copying cmdapp.py -> build/lib creating build/bdist.macosx-10.3-i386 creating build/bdist.macosx-10.3-i386/egg copying build/lib/cmdapp.py -> build/bdist.macosx-10.3-i386/egg copying build/lib/timed.py -> build/bdist.macosx-10.3-i386/egg byte-compiling build/bdist.macosx-10.3-i386/egg/cmdapp.py to cmdapp.pyc byte-compiling build/bdist.macosx-10.3-i386/egg/timed.py to timed.pyc creating build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/PKG-INFO -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/SOURCES.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/dependency_links.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/entry_points.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/not-zip-safe -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/requires.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO copying timed.egg-info/top_level.txt -> build/bdist.macosx-10.3-i386/egg/EGG-INFO creating dist creating 'dist/timed-0.11-py2.6.egg' and adding 'build/bdist.macosx-10.3-i386/egg' to it removing 'build/bdist.macosx-10.3-i386/egg' (and everything under it) Processing timed-0.11-py2.6.egg removing '/usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg' (and everything under it) creating /usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg Extracting timed-0.11-py2.6.egg to /usr/local/lib/python2.6/site-packages timed 0.11 is already the active version in easy-install.pth Installing timed script to /usr/local/bin Installed /usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg Processing dependencies for timed==0.11 Searching for PyYAML==3.08 Best match: PyYAML 3.08 Processing PyYAML-3.08-py2.6-macosx-10.3-i386.egg PyYAML 3.08 is already the active version in easy-install.pth Using /usr/local/lib/python2.6/site-packages/PyYAML-3.08-py2.6-macosx-10.3-i386.egg Finished processing dependencies for timed==0.11 % ls /usr/local/lib/python2.6/site-packages/timed-0.11-py2.6.egg EGG-INFO cmdapp.py cmdapp.pyc timed.py timed.pyc Why isn't README there? What do I need to do to include it?

1 0

Re: [Distutils] Including DLLs in binary distributions
by P.J. Eby 25 Jul '09

25 Jul '09

At 11:09 AM 7/21/2009 -0400, Jean-Paul Calderone wrote: >On Tue, 21 Jul 2009 09:47:46 -0400, Jean-Paul Calderone ><exarkun(a)divmod.com> wrote: >>On Tue, 21 Jul 2009 09:02:16 -0400, Jean-Paul Calderone >><exarkun(a)divmod.com> wrote: >>>On Mon, 20 Jul 2009 20:22:41 -0400, "P.J. Eby" >>><pje(a)telecommunity.com> [snip] >>>> >>>>Use package_data instead; it should do the right thing with both >>>>distutils and setuptools. (It is available in the distutils as >>>>of Python 2.4; for 2.3 you'd have to use setuptools.) >>> >>>Can package_data include files which aren't in the source tree? I naively >>>tried >>> package_data = { >>> 'OpenSSL': ['C:/OpenSSL/ssleay32.dll', >>> 'C:/OpenSSL/libeay32.dll']} >> >>Hmm. Actually, I then tried various other spellings, using only relative >>paths, and was unable to get these DLLs into the egg using any of them. I >>wonder if I am missing something fundamental about how package_data is >>interpreted. >> >>A couple other values I tried: >> >> package_data = {'': ['ssleay32.dll', 'libeay32.dll']} >> >> package_data = {'OpenSSL': ['ssleay32.dll', 'libeay32.dll']} >> >>Any tips? > >I tracked down the problem to a requirement I wasn't aware of. If no >value is passed for the "packages" argument to setup, it seems that the >package_data value is ignored. Once I added `packages = ["OpenSSL"]Â´, >the former of the above package_data definitions worked. > >I'm still curious about whether it's possible to specify absolute paths >rather than relative paths. No, it's not supported by the distutils or by setuptools. The concept of "package data" is data in the same directory as the package's Python code. Using it to install libraries is a bit of a hack in the first place.

5 11

setuptools for win32 py2.6
by Eric Swenson 24 Jul '09

24 Jul '09

Hello, I notice that the download page doesn't include a win32 installer for setuptools and python 2.6. I built one from sources but wonder if we should have one up on the download page. Do you have one available or can I give you one I built to include on the download page? THanks. -- Eric

1 0

Distutils History
by anatoly techtonik 24 Jul '09

24 Jul '09

Hello, I've added a chapter on Distutils history to http://wiki.python.org/moin/Distutils to provide an overview why current Distutils in not sufficient anymore and needs refactoring. I would like to ask for review to see if that information is more appropriate for Distutils/Critics --anatoly t.

3 2

Re: [Distutils] Colour this bikeshed: Name for setuptools fork
by P.J. Eby 23 Jul '09

23 Jul '09

At 01:05 AM 7/19/2009 +0200, Hanno Schlichting wrote: >On Sat, Jul 18, 2009 at 9:50 PM, P.J. Eby<pje(a)telecommunity.com> wrote: > > Also, this entire discussion may be moot if, as I'm currently understanding > > it, this is simply a matter of other people doing maintenance and bug fixes > > to the 0.6 branch... Â in which case there is a very strong > possibility that > > I'll just bless various versions of it as official 0.6 versions of > > setuptools. > > > > (One obstacle to this actually happening, though, is the part where they're > > stripping out my release tools, like wikiup.cfg, version+version.dat, etc., > > as I would need to add these back in order to actually do a release.) > >In order to make it easier for you to bless things, I created a branch >of our work, which very closely resembles the official setuptools 0.6 >branch, including the correct distribution name and including all your >build and release tools. > >It's a "hg up -C setuptools-compatible-0.6" away after cloning the >repo. It does need some more tweaks to version identifiers (is it 0.6 >or 0.6.0 or 0.6-final ...) and lacks the md5 checksums for release >files, since these are obviously not produced yet. > >I'd be happy to produce the Windows binary installers for an official release. Good to know. I'll take a look at it soon. >I'm sure we do have some disagreements about how to evolve setuptools >after the 0.6 branch. Perhaps. But it occurs to me that there is something I can do to help that in the long run. After giving it some thought today, it seems that it would be a good idea to take the "good" (i.e. relatively uncontroversial) parts of setuptools and split them out into another package, which I am tentatively naming "Discovery". Rather than a set of executable tools, it would just be a collection of libraries to be called by package management tools, and other Python code needing to discover packages either installed on the system, on the package index, be notified when they're imported, etc. etc. The package itself would look something like this: discovery.resources -- basically pkg_resources under a different name discovery.sandbox -- basically setuptools.sandbox, less distutils dependency discovery.unpack -- setuptools.archive_util, less distutils dependency discovery.downloads -- a hugely refactored version of parts of setuptools.package_index, redesigned for pluggable extendibility (e.g. to handle Sourceforge crap, checkouts from other revision control systems, etc.) discovery.install -- selected subroutines from easy_install, refactored for narrow functionality and minimal coupling, combined with PEP 376 support code discovery.importing -- basically peak.util.imports under a different name, but with a more modern/decorator-friendly API discovery.plugins -- basically peak.util.plugins, refactored to drop dependencies on other PEAK modules Then, easy_install could be refactored to consist mostly of calls to these other modules, along with whatever "legacy" logic needs to be retained for compatibility reasons, and setuptools proper could evolve towards just being build support, and become more pluggable generally. It wouldn't be possible to switch setuptools to do this right away, since there would remain the problem of bootstrapping the installation of the discovery package prior to setuptools installation. But setuptools' source distribution could perhaps bundle a copy of discovery within itself, and the ez_setup.py script could perhaps then be changed to download *discovery* first -- and then ask discovery to download setuptools to a temporary location. However, now that somebody else is effectively looking out for 0.6 maintenance, this lets me bump up the priority of working on 0.7, and has given me ideas for how to get out from under the rest of the "legacy" problems of setuptools, while also making reuse easier for competing and/or complementary projects (such as buildout, pip, etc.)

6 20

Possible to import from a zipfile retrieved from within an .egg using pkg_resource's ResourceManager API?
by Dave Peterson 22 Jul '09

22 Jul '09

Background: The Mayavi project generates wrapper code around VTK during it's build process. This gets bundled into a zip file called tvtk_classes.zip and included in a binary distribution. Currently, we mark built eggs as non-zip-safe so that they get unpacked during installation, which makes it possible to find the zip file and add it directly to sys.path. However, this mechanism won't work when someone tries to distribute an app containing Mayavi by using py2app, py2exe, etc. because all of site-packages usually gets bundled in a single zip. Problem: I'm wondering if there is some way to use pkg_resources to retrieve a reference to the embedded zip file within py2app's generated site-packages.zip (or within an unpacked egg install) and somehow add that to sys.path? The only obvious thing I see in pkg_resource's ResourceManager API docs is to extract the embedded zipfile via "resource_filename". However, it isn't clear to me whether calling that will extract *everything* within the containing zip, or just unpack the requested resource itself. The former could be a rather large performance hit in the case of py2app. Does anyone have any feedback on the performance of using this resource_filename API? Or is there a better way to do this? Perhaps there is a way to force py2app/py2exe to avoid putting Mayavi within the site-packages zipfile? (Right now, I'm more concerned about the py2app usecase than I am the zip-within-an-egg usecase.) Thanks in advance for any advice! -- Dave

2 3

[Distribute] Distribute release preview - testers needed
by Tarek Ziadé 22 Jul '09

22 Jul '09

Hello I've uploaded a "preview" of the distribution in the repository website. That's not the official release but I've added there so people can help me debug it before it gets out for real ;) There's a script called "new_ez_setup.py" you can download and use to install Distribute : http://bitbucket.org/tarek/distribute/downloads/new_ez_setup.py It's not called ez_setup.py because there's a bug in bickbucket not allowing me to override the old ez_setup.py file we've uploaded a while ago (the old content keeps showing up if the new file is called ez_setup.py, looks like a cache issue) Anyways, the process is : $ wget http://bitbucket.org/tarek/distribute/downloads/new_ez_setup.py $ python new_ez_setup.py What happens is : - if the setuptools distribution is detected, it's replaced with a fake 0.6c9 egg - Distribute then installs its own egg containing its setuptools package + pkg_resources.py module. In theory all programs that use setuptools will continue to work without any problem at this point, In practice I need to do a lot of tests to see how it works when there's other setuptools versions installed with -m. There's some work to do too on zc.buildout side to see how it behaves. I have started to write functional tests for this script but it's not easy to test it under all environements. but at this stage any help is already welcome ! my updated list of tasks : - more tests - working on zc.buildout side - documenting the install process, what is does, and how to remove it, In any case, there are good chances at this stage that some bugs will occur; if you do find a bug, please add an issue in the tracker, with "ez_setup" in the title with all the details about your environment. Thanks! Cheers Tarek -- Tarek Ziadé | http://ziade.org

2 3