Mailman 3 January 2006 - Distutils-SIG

setuptools site-package pollution
by Anthony Tarlano 10 Oct '06

10 Oct '06

Hi all.. Firstly, I would like to say thanks for the setuptools package, which I was introduced to after reading about the RuleDispatch package on the IBM developerworks charming python series. Oh btw. RuleDispatch is the most useful python package that I have seen in the last year in the python world, so thanks for that also ;) Now that the congrats and hugs are out of the way, I would like to ask a question. How can I tell setuptools not to put packages, such as dispatch, protocols, and setuptools, in a subdirectory named the same as the egg and to just put the package name. For example from pydoc I currently get: c:\python\lib\site-packages\ruledispatch-0.5a0.dev_r2100-py2.4-win32.egg dispatch (package) c:\python\lib\site-packages\pyprotocols-1.0a0dev_r2082-py2.4-win32.egg dispatch protocols (package) c:\python\lib\site-packages\setuptools-0.6a8-py2.4.egg easy_install pkg_resources setuptools (package) site what I would like to see is just: ..\site-packages dispatch (package) ..\site-packages protocols (package) ..\site-packages setuptools (package) Maybe it is just a pet-peeve, but I like to keep a nice tidy site-packages directory, and these long directory names just seem to me to be pollution of my site-packages directory in my command shell. If there is an option to have just the packages, I would really appreciate someone telling me what it is. If not maybe it could be considered, and to put whatever meta-data the directory names are providing somewhere else.. Thanks, Anthony

3 2

using bdist_wininst --install-script
by Stefan Seefeld 27 Jan '06

27 Jan '06

Hi there, I'm working on a python application which uses a number of data files. In order to figure out the path to the data files I used to use some hacks, such as letting the application file inject a 'prefix' variable into the root module (package) for other modules to use. This was, IIRC, required since on some systems (notably windows) the actual prefix could differ from any variable set during the packaging stage (since it's the user's choice to override it). Now I read about the '--install-script' option that sounds as if it could solve my problem (on windows). Can anybody comment on this ? How can I use it to fix up 'prefix' variables at installation time ? I'v found the documentation at http://www.python.org/doc/2.4.1/dist/postinstallation-script.html, but that doesn't mention what parameters are available to the script (beside the 'install' / 'remove' flag). Any help is highly appreciated ! Stefan

2 7

static linking in distutils/setuptools?
by Christopher Fonnesbeck 27 Jan '06

27 Jan '06

Is there any way to specify static linking to particular libraries when building installers with distutils or setuptools? I want to be able to include the shared libraries that I am linking to in my builds. I did not see any info on this in the docs. Thanks, C. -- Christopher J. Fonnesbeck Population Ecologist, Marine Mammal Section Fish & Wildlife Research Institute (FWC) St. Petersburg, FL Adjunct Assistant Professor Warnell School of Forest Resources University of Georgia Athens, GA T: 727.235.5570 E: chris at trichech.us

9 14

Sourceforge changed HTML again...
by Phillip J. Eby 26 Jan '06

26 Jan '06

Hi all. Ian Bicking reported an issue with Sourceforge's download process changing again; I've updated setuptools in SVN but haven't made a new release yet. If you need the fixed version, update to the development version via "ez_setup.py setuptools==dev" for now. Thanks.

5 9

py2exe and setuptools w/ install_requires
by Chris Lambacher 26 Jan '06

26 Jan '06

Hi, I have several apps that I distribute to customers using py2exe. I have also written modules which these apps share. Up to now I have not had any modules written in C so to build my py2exe apps I just set the PYTHONPATH environment variable to point to the module's location in CVS and run setup.py py2exe. This is convenient because release builds are done via Buildbot on a dedicated computer. All buildbot has to do is check out the source and run setup.py. I now have my first C module and have been trying to work out how to deal with building the module. I would prefer to build the module and 'release' it for use with my two apps. What I would like to do is build up an egg for the library and copy it to a known location. Then have the apps use the setuptools require_install directive to allow me to get the egg and 'install' it into the build directory somewhere and then let the rest of the py2exe magic work. I think I know where to add my call into setuptools from py2exe, but I cannot figure out what setup_tools function to call in order to do the 'installation'. Any help in executing my plan or in telling me I am a morron because there is some easier way to do this would be much appreciated. -Chris

2 4

(extension) module dependencies in setuptools
by Andrew Straw 26 Jan '06

26 Jan '06

I would like to propose a feature for setuptools: runtime enforcement of dependencies specified at build time by setup.py. I appreciate that "pkg_resources.require('foo==1.0')" works, but this requires a tedious update of version numbers in affected source files every time you upgrade foo and rebuild the target package. I'm thinking, in particular, of extension modules built on a particular version of another package with its own C interface. Think matplotlib.backends._ns_backend_agg depending on numpy. It would be really nice, in matplotlib's setup.py file, to say something like: from setuptools import setup, Extension import numpy numpy_include_dirs = numpy.get_numpy_include() setup(name='matplotlib', ext_modules=Extension('matplotlib.backends._ns_backend_agg', sources='src/backends/_ns_backend_agg.cpp', include_dirs=numpy_include_dirs, runtime_requires=[pkg_resources.get_distribution('numpy').as_requirement()], ) ) Alternatively, the whole package (not just the extension module) might depend on a particular version: from setuptools import setup import numpy numpy_include_dirs = numpy.get_numpy_include() setup(name='matplotlib', ext_modules=Extension('matplotlib.backends._ns_backend_agg', sources='src/backends/_ns_backend_agg.cpp', include_dirs=numpy_include_dirs, ) runtime_requires=[pkg_resources.get_distribution('numpy').as_requirement()], ) Now, if I installed an additional, newer numpy, a couple of things could happen: if my application imports matplotlib first, setuptools puts the appropriate (older) numpy into the global working_set and I get this older numpy when I do "import numpy". If my application imports (the newer) numpy first and then matplotlib, an exception is raised saying that matplotlib depends of versions such and such but version so and so is already imported. Because I'm thinking primarily of extension modules, there are additional reasons why I don't want to specify a runtime check using a hardcoded pkg_resources.require() in the package itself. First, the actual requirement may be a C-interface issue leading to segfaults and other nastiness if ignored or forgotten, thus justifying this easier way to specify dependencies. Second, an extension module, by definition, is not Python, so it will take more programming effort to write the call to pkg_resources.require(). Third, I really don't want to have to convince all the projects out there to modify multiple files to use setuptools. I'm already attempting to dispel enough anti-egg sentiment (for reasons I don't understand) resulting from slight changes to setup.py. What do folks think about this idea? Would such a feature be possible and desirable in setuptools? Cheers! Andrew

2 2

allowing $py_version in ~/.pydistutils.cfg
by Andrew Straw 26 Jan '06

26 Jan '06

Hi, Attempting to use variable subsitition in my ~/.pydistutils.cfg file (which I'd like to use across both Python 2.3 and Python 2.4), I find that [install] prefix=~/py$py_version_short-$PLAT works fine while the $variables in [easy_install] site_dirs=~/py$py_version_short-$PLAT/lib/python$py_version_short/site-packages don't get expanded. These $variables seem to be an undocumented distutils feature (at least they aren't in the online docs for distutils), but they're nevertheless handy. Would it be possible to include them in setuptools? They are implemented in distutils.util subst_vars(). Cheers! Andrew

2 1

Setting distutils options from sitecustomize
by Ian Bicking 26 Jan '06

26 Jan '06

Has anyone tried anything that involves setting the distutils options (e.g., where to install libraries) from sitecustomize or some other Python location? I want to put in logic that is more complex than can be expressed in a configuration file. Setuptools-specific is a-ok too. Monkeypatching distutils doesn't particularly bother me in this case either. Though that's not great either (if it goes in sitecustomize), since it means that there's an overhead to every Python startup to load distutils and patch it, even if distutils wouldn't have otherwise loaded. Jim Fulton asked about something similar on IRC too -- actually about having an easy_install binary that installed to a different location than normal (to the Zope instance path). Similar in that however such a script might work it might need to do similar things, setting distutils options at runtime. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

3 2

Re: [Distutils] Sourceforge changed HTML again...
by Phillip J. Eby 26 Jan '06

26 Jan '06

At 08:27 AM 1/23/2006 -0500, Charlie Moad wrote: >On 1/18/06, Ian Bicking <ianb(a)colorstudy.com> wrote: > > Phillip J. Eby wrote: > > > I checked what the NetBSD "pkgsrc" system does, and it uses the fact > > > that there is a .dl.sourceforge.net subdomain for mirrors. I > > > investigated further and found that dl.sourceforge.net is a round-robin > > > (or random?) DNS for each of the mirrors in the subdomain, so there's a > > > simple transformation from the user-visible download pages to the actual > > > download address. You can see this in the new fix_sf_url() function I > > > added to setuptools.package_index. > > > > I just got a Connection Refused error, but it worked on the second try. > > As I remember, the port system typically has a set of links, and > > frequently fails over from one link to the next, so I expect that this > > error should be expected. When it is encountered, setuptools should > > just try again until it works. I have noticed in the past that SF > > mirrors go up and down quite frequently, probably too fast for DNS to > > keep up. > >Does the pypi download url need to be updated to be compatible with >svn setuptools? I am assuming no, but I am constantly getting 404's >from easy_install, yet I can paste the dl.sf.net link into a browser >and it works?! These files have been on sf long enough to be >propgated around, and constantly ~= 20 attempts on the command line. >I am seeing this with matplotlib, btw. Interesting. I just tried "easy_install matplotlib" and it worked the first time. I actually have never yet gotten a 404 or other problem using the dl.sf.net URLs on any package. However, a little experimentation shows that my Windows PC caches the DNS reply for an extended period, so I guess that means that if you get a bad mirror IP, you're stuck with it until the cache clears or you run "ipconfig /flushdns". (In contrast, my Linux machine returns a different IP every time the name is looked up.) A little experimentation with the socket module shows that I can get the full list of mirror IPs from Python, so I've changed setuptools in SVN to just randomly select one to use, which should fix the sticking problem on Windows (and any other platform where it occurs). It's not a perfect solution, since of course you can still end up with a bad mirror for some download, but I'm not sure what else to do, short of having some kind of option/configuration to control mirror selection.

2 2

expected non-zero return values when spawning subprocesses
by Stefan Seefeld 26 Jan '06

26 Jan '06

Hi there, I'm using distutils.spawn.spawn() to run subprocesses during my build commands. It now happens that one of those subprocesses will return a non-zero value that doesn't strictly correspond to an error, and thus I'd like to catch it from within my python code. It appears spawn() itself will raise a message containing a stringified version of the exit code, but not the exit code itself. This seems to imply that, in order to discriminate on the exit value, I have to catch DistutilsExecError, and then parse the string. This seems rather unelegant. Is there some better way ? Could the exception be modified to contain the original code (as an integral value) ? Thanks, Stefan

2 1