Mailman 3 January 2006 - Distutils-SIG

Problems with svn in windows
by Sharky On PTNet April 29, 2018

April 29, 2018

Hi, I´m trying to do "easy_install setuptools==dev" in a windows box but have an error with svn command. """... 'svn' is not recognized as an internal or external command, ...""" I have installed TortoiseSVN but it seems that don't have command line, is all with the Explorer. Can you tell me what client I may use in windows? Thanks for all, Helio Pereira Sharky @ PTNet

3 2

local development scenarios
by tgreenwood April 29, 2018

April 29, 2018

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm new to distutils/and the egg system. I have not researched the archives as much as I should, so please excuse me if this is familiar territory... Q: Is it possible for dependency checks to work for non pipy deployed eggs? - A.egg - B.egg (setup_requires and install_requires A.egg) - email A and B eggs to friend - tell friend to : easy_install B - but B.egg does not install A...in fact, in my test case, it fails b/c it cannot find A in pipy - in this example, what happens if friend does not have net connection on target machine? how can they install from the eggs alone and get the dep checking to work? Q: How does my friend run the unittests on the egg that they have just installed? As I am developing, I frequently run the unittests and after every bdist_egg build, I also run the unit tests that way, too. How can the consumer do the same? Q: One of my apps is a command line app, say foo.py is a command line app that takes arg1, arg2,... How can the user run this app when it is encased in an egg? Q: How do I define my own pipy? While I like the idea of deploying there for the overall community, I don't want to send my half baked code into the wild. But I'd like to try the general functionality of having a deploy site for my beta customers. How can I set this up? Thanks in advance. - -Todd -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDl9Hfz6uXX4lQc/URAnY4AJ47DFHL/cpEltmqoUyGqzhsK5FjRACgt01v akc3Wh3IOXVv2jiS8TszhuE= =Rv/x -----END PGP SIGNATURE-----

5 12

Choice of dependencies?
by John J Lee April 29, 2018

April 29, 2018

Is it possible for a package to depend on one of several packages, with the user having the option to pick? For example, my package P might depend on package A, plus either package B or package C. I know I could (ab)use extras_require, but in my case this is a real dependency rather than an optional extra. The other option I have is to just pick one of the dependencies (B or C in my example above), and thereby force people to install it even though it's not strictly required. Neither option seems particularly attractive. John

3 6

setuptools doc patch
by John J Lee April 29, 2018

April 29, 2018

The comma removed by the patch below implies that users working from a Subversion checkout should not use EasyInstall to periodically fetch the latest version. That's not what you intended to say, I assume? http://peak.telecommunity.com/DevCenter/setuptools You should also inform your users of the need to run this command, if they -are working from a Subversion checkout, rather than using EasyInstall to +are working from a Subversion checkout rather than using EasyInstall to periodically fetch the latest version. Hmm, probably much better than the patch above: -You should also inform your users of the need to run this command, if they -are working from a Subversion checkout, rather than using EasyInstall to -periodically fetch the latest version. +If your users are working from a Subversion checkout rather than using +EasyInstall, you should also inform them of the need to run this command +to periodically fetch the latest version. John

2 2

Re: [Distutils] distutils install probs on suse 9.0
by Mats Wichmann April 29, 2018

April 29, 2018

At 05:43 AM 5/18/2004, David Handy wrote: >On Tue, 18 May 2004, Floris van der Tak wrote: > >> my attempt to install the distutils on suse 9.0 (which provides python >> 2.3) fails: > >Why are you installing distutils on a system that already has Python 2.3 >installed? Python 2.3 normally comes with distutils as part of the Python >standard library. > >Did suse distribute a version of Python that had distutils removed? It should come from python-devel

2 1

Sample setup.py for Numeric Python
by Greg Ward April 29, 2018

April 29, 2018

Hi again -- [cc'd to Paul Dubois: you said you weren't following the distutils sig anymore, but this directly concerns NumPy and I'd like to get your input!] here's that sample setup.py for NumPy. See below for discussion (and questions!). ------------------------------------------------------------------------ #!/usr/bin/env python # Setup script example for building the Numeric extension to Python. # This does sucessfully compile all the .dlls. Nothing happens # with the .py files currently. # Move this file to the Numerical directory of the LLNL numpy # distribution and run as: # python numpysetup.py --verbose build_ext # # created 1999/08 Perry Stoll __rcsid__ = "$Id: numpysetup.py,v 1.1 1999/09/12 20:42:48 gward Exp $" from distutils.core import setup setup (name = "numerical", version = "0.01", description = "Numerical Extension to Python", url = "http://www.python.org/sigs/matrix-sig/", ext_modules = [ ( '_numpy', { 'sources' : [ 'Src/_numpymodule.c', 'Src/arrayobject.c', 'Src/ufuncobject.c' ], 'include_dirs' : ['./Include'], 'def_file' : 'Src/numpy.def' } ), ( 'multiarray', { 'sources' : ['Src/multiarraymodule.c'], 'include_dirs' : ['./Include'], 'def_file': 'Src/multiarray.def' } ), ( 'umath', { 'sources': ['Src/umathmodule.c'], 'include_dirs' : ['./Include'], 'def_file' : 'Src/umath.def' } ), ( 'fftpack', { 'sources': ['Src/fftpackmodule.c', 'Src/fftpack.c'], 'include_dirs' : ['./Include'], 'def_file' : 'Src/fftpack.def' } ), ( 'lapack_lite', { 'sources' : [ 'Src/lapack_litemodule.c', 'Src/dlapack_lite.c', 'Src/zlapack_lite.c', 'Src/blas_lite.c', 'Src/f2c_lite.c' ], 'include_dirs' : ['./Include'], 'def_file' : 'Src/lapack_lite.def' } ), ( 'ranlib', { 'sources': ['Src/ranlibmodule.c', 'Src/ranlib.c', 'Src/com.c', 'Src/linpack.c', ], 'include_dirs' : ['./Include'], 'def_file' : 'Src/ranlib.def' } ), ] ) ------------------------------------------------------------------------ First, what d'you think? Too clunky and verbose? Too much information for each extension? I kind of think so, but I'm not sure how to reduce it elegantly. Right now, the internal data structures needed to compile a module are pretty obviously exposed: is this a good thing? Or should there be some more compact form for setup.py that will be expanded later into the full glory we see above? I've already made one small step towards reducing the amount of cruft by factoring 'include_dirs' out and supplying it directly as a parameter to 'setup()'. (But that needs code not in the CVS archive yet, so I've left the sample setup.py the same for now.) The next thing I'd like to do is get that damn "def_file" out of there. To support it in MSVCCompiler, there's already an ugly hack that unnecessarily affects both the UnixCCompiler and CCompiler classes, and I want to get rid of that. (I refer to passing the 'build_info' dictionary into the compiler classes, if you're familiar with the code -- that dictionary is part of the Distutils extension-building system, and should not propagate into the more general compiler classes.) But I don't want to give these weird "def file" things standing on the order of source files, object files, libraries, etc., because they seem to me to be a bizarre artifact of one particular compiler, rather than something present in a wide range of C/C++ compilers. Based on the NumPy model, it seems like there's a not-too-kludgy way to handle this problem. Namely: if building extension "foo": if file "foo.def" found in same directory as "foo.c" add "/def:foo.def" to MSVC command line this will of course require some platform-specific code in the build_ext command class, but I figured that was coming eventually, so why put it off? ;-) To make this hack work with NumPy, one change would be necessary: rename Src/numpy.def to Src/_numpy.def to match Src/_numpy.c, which implements the _numpy module. Would this be too much to ask of NumPy? (Paul?) What about other module distributions that support MSVC++ and thus ship with "def" files? Could they be made to accomodate this scheme? Thanks for your feedback -- Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

4 5

Can now compile simple extensions
by Greg Ward April 29, 2018

April 29, 2018

Hi all -- at long last, I found the time to hack in the ability to compile extension modules to the Distutils. Mainly, this meant adding a 'build_ext' command which uses a CCompiler instance for all its dirty work. I also had to add a few methods to CCompiler (and, of course, UnixCCompiler) to make this work. And I added a new module, 'spawn', which takes care of running sub-programs more efficiently and robustly (no shell involved) than os.system. That's needed, obviously, so we can run the compiler! If you're in the mood for grubbing over raw source code, then get the latest from CVS or download a current snapshot. See http://www.python.org/sigs/distutils-sig/implementation.html for a link to the code snapshot. I'm still waiting for more subclasses of CCompiler to appear. At the very least, we're going to need MSVCCompiler to build extensions on Windows. Any takers? Also, someone who knows the Mac, and how to run compilers programmatically there, will have to figure out how to write a Mac-specific concrete CCompiler subclass. The spawn module also needs a bit of work to be portable. I suspect that _win32_spawn() (the intended analog to my _posix_spawn()) will be easy to implement, if it even needs to go in a separate function at all. It looks from the Python Library documentation for 1.5.2 that the os.spawnv() function is all we need, but it's a bit hard to figure out just what's needed. Windows wizards, please take a look at the 'spawn()' function and see if you can make it work on Windows. As for actually compiling extensions: well, if you can figure out the build_ext command, go ahead and give it a whirl. It's a bit cryptic right now, since there's no documentation and no example setup.py. (I have a working example at home, but it's not available online.) If you feel up to it, though, see if you can read the code and figure out what's going on. I'm just hoping *I'll* be able to figure out what's going on when I get back from the O'Reilly conference next week... ;-) Enjoy -- Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Two fixes (at last)
by Greg Ward April 29, 2018

April 29, 2018

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker April 29, 2018

April 29, 2018

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller April 29, 2018

April 29, 2018

3 2

setuptools site-package pollution
by Anthony Tarlano Oct. 10, 2006

Oct. 10, 2006

Hi all.. Firstly, I would like to say thanks for the setuptools package, which I was introduced to after reading about the RuleDispatch package on the IBM developerworks charming python series. Oh btw. RuleDispatch is the most useful python package that I have seen in the last year in the python world, so thanks for that also ;) Now that the congrats and hugs are out of the way, I would like to ask a question. How can I tell setuptools not to put packages, such as dispatch, protocols, and setuptools, in a subdirectory named the same as the egg and to just put the package name. For example from pydoc I currently get: c:\python\lib\site-packages\ruledispatch-0.5a0.dev_r2100-py2.4-win32.egg dispatch (package) c:\python\lib\site-packages\pyprotocols-1.0a0dev_r2082-py2.4-win32.egg dispatch protocols (package) c:\python\lib\site-packages\setuptools-0.6a8-py2.4.egg easy_install pkg_resources setuptools (package) site what I would like to see is just: ..\site-packages dispatch (package) ..\site-packages protocols (package) ..\site-packages setuptools (package) Maybe it is just a pet-peeve, but I like to keep a nice tidy site-packages directory, and these long directory names just seem to me to be pollution of my site-packages directory in my command shell. If there is an option to have just the packages, I would really appreciate someone telling me what it is. If not maybe it could be considered, and to put whatever meta-data the directory names are providing somewhere else.. Thanks, Anthony

3 2

using bdist_wininst --install-script
by Stefan Seefeld Jan. 27, 2006

Jan. 27, 2006

Hi there, I'm working on a python application which uses a number of data files. In order to figure out the path to the data files I used to use some hacks, such as letting the application file inject a 'prefix' variable into the root module (package) for other modules to use. This was, IIRC, required since on some systems (notably windows) the actual prefix could differ from any variable set during the packaging stage (since it's the user's choice to override it). Now I read about the '--install-script' option that sounds as if it could solve my problem (on windows). Can anybody comment on this ? How can I use it to fix up 'prefix' variables at installation time ? I'v found the documentation at http://www.python.org/doc/2.4.1/dist/postinstallation-script.html, but that doesn't mention what parameters are available to the script (beside the 'install' / 'remove' flag). Any help is highly appreciated ! Stefan

2 7

static linking in distutils/setuptools?
by Christopher Fonnesbeck Jan. 27, 2006

Jan. 27, 2006

Is there any way to specify static linking to particular libraries when building installers with distutils or setuptools? I want to be able to include the shared libraries that I am linking to in my builds. I did not see any info on this in the docs. Thanks, C. -- Christopher J. Fonnesbeck Population Ecologist, Marine Mammal Section Fish & Wildlife Research Institute (FWC) St. Petersburg, FL Adjunct Assistant Professor Warnell School of Forest Resources University of Georgia Athens, GA T: 727.235.5570 E: chris at trichech.us

9 14

Sourceforge changed HTML again...
by Phillip J. Eby Jan. 26, 2006

Jan. 26, 2006

Hi all. Ian Bicking reported an issue with Sourceforge's download process changing again; I've updated setuptools in SVN but haven't made a new release yet. If you need the fixed version, update to the development version via "ez_setup.py setuptools==dev" for now. Thanks.

5 9

py2exe and setuptools w/ install_requires
by Chris Lambacher Jan. 26, 2006

Jan. 26, 2006

Hi, I have several apps that I distribute to customers using py2exe. I have also written modules which these apps share. Up to now I have not had any modules written in C so to build my py2exe apps I just set the PYTHONPATH environment variable to point to the module's location in CVS and run setup.py py2exe. This is convenient because release builds are done via Buildbot on a dedicated computer. All buildbot has to do is check out the source and run setup.py. I now have my first C module and have been trying to work out how to deal with building the module. I would prefer to build the module and 'release' it for use with my two apps. What I would like to do is build up an egg for the library and copy it to a known location. Then have the apps use the setuptools require_install directive to allow me to get the egg and 'install' it into the build directory somewhere and then let the rest of the py2exe magic work. I think I know where to add my call into setuptools from py2exe, but I cannot figure out what setup_tools function to call in order to do the 'installation'. Any help in executing my plan or in telling me I am a morron because there is some easier way to do this would be much appreciated. -Chris

2 4

(extension) module dependencies in setuptools
by Andrew Straw Jan. 26, 2006

Jan. 26, 2006

I would like to propose a feature for setuptools: runtime enforcement of dependencies specified at build time by setup.py. I appreciate that "pkg_resources.require('foo==1.0')" works, but this requires a tedious update of version numbers in affected source files every time you upgrade foo and rebuild the target package. I'm thinking, in particular, of extension modules built on a particular version of another package with its own C interface. Think matplotlib.backends._ns_backend_agg depending on numpy. It would be really nice, in matplotlib's setup.py file, to say something like: from setuptools import setup, Extension import numpy numpy_include_dirs = numpy.get_numpy_include() setup(name='matplotlib', ext_modules=Extension('matplotlib.backends._ns_backend_agg', sources='src/backends/_ns_backend_agg.cpp', include_dirs=numpy_include_dirs, runtime_requires=[pkg_resources.get_distribution('numpy').as_requirement()], ) ) Alternatively, the whole package (not just the extension module) might depend on a particular version: from setuptools import setup import numpy numpy_include_dirs = numpy.get_numpy_include() setup(name='matplotlib', ext_modules=Extension('matplotlib.backends._ns_backend_agg', sources='src/backends/_ns_backend_agg.cpp', include_dirs=numpy_include_dirs, ) runtime_requires=[pkg_resources.get_distribution('numpy').as_requirement()], ) Now, if I installed an additional, newer numpy, a couple of things could happen: if my application imports matplotlib first, setuptools puts the appropriate (older) numpy into the global working_set and I get this older numpy when I do "import numpy". If my application imports (the newer) numpy first and then matplotlib, an exception is raised saying that matplotlib depends of versions such and such but version so and so is already imported. Because I'm thinking primarily of extension modules, there are additional reasons why I don't want to specify a runtime check using a hardcoded pkg_resources.require() in the package itself. First, the actual requirement may be a C-interface issue leading to segfaults and other nastiness if ignored or forgotten, thus justifying this easier way to specify dependencies. Second, an extension module, by definition, is not Python, so it will take more programming effort to write the call to pkg_resources.require(). Third, I really don't want to have to convince all the projects out there to modify multiple files to use setuptools. I'm already attempting to dispel enough anti-egg sentiment (for reasons I don't understand) resulting from slight changes to setup.py. What do folks think about this idea? Would such a feature be possible and desirable in setuptools? Cheers! Andrew

2 2

allowing $py_version in ~/.pydistutils.cfg
by Andrew Straw Jan. 26, 2006

Jan. 26, 2006

Hi, Attempting to use variable subsitition in my ~/.pydistutils.cfg file (which I'd like to use across both Python 2.3 and Python 2.4), I find that [install] prefix=~/py$py_version_short-$PLAT works fine while the $variables in [easy_install] site_dirs=~/py$py_version_short-$PLAT/lib/python$py_version_short/site-packages don't get expanded. These $variables seem to be an undocumented distutils feature (at least they aren't in the online docs for distutils), but they're nevertheless handy. Would it be possible to include them in setuptools? They are implemented in distutils.util subst_vars(). Cheers! Andrew

2 1

Setting distutils options from sitecustomize
by Ian Bicking Jan. 26, 2006

Jan. 26, 2006

Has anyone tried anything that involves setting the distutils options (e.g., where to install libraries) from sitecustomize or some other Python location? I want to put in logic that is more complex than can be expressed in a configuration file. Setuptools-specific is a-ok too. Monkeypatching distutils doesn't particularly bother me in this case either. Though that's not great either (if it goes in sitecustomize), since it means that there's an overhead to every Python startup to load distutils and patch it, even if distutils wouldn't have otherwise loaded. Jim Fulton asked about something similar on IRC too -- actually about having an easy_install binary that installed to a different location than normal (to the Zope instance path). Similar in that however such a script might work it might need to do similar things, setting distutils options at runtime. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

3 2

Re: [Distutils] Sourceforge changed HTML again...
by Phillip J. Eby Jan. 26, 2006

Jan. 26, 2006

At 08:27 AM 1/23/2006 -0500, Charlie Moad wrote: >On 1/18/06, Ian Bicking <ianb(a)colorstudy.com> wrote: > > Phillip J. Eby wrote: > > > I checked what the NetBSD "pkgsrc" system does, and it uses the fact > > > that there is a .dl.sourceforge.net subdomain for mirrors. I > > > investigated further and found that dl.sourceforge.net is a round-robin > > > (or random?) DNS for each of the mirrors in the subdomain, so there's a > > > simple transformation from the user-visible download pages to the actual > > > download address. You can see this in the new fix_sf_url() function I > > > added to setuptools.package_index. > > > > I just got a Connection Refused error, but it worked on the second try. > > As I remember, the port system typically has a set of links, and > > frequently fails over from one link to the next, so I expect that this > > error should be expected. When it is encountered, setuptools should > > just try again until it works. I have noticed in the past that SF > > mirrors go up and down quite frequently, probably too fast for DNS to > > keep up. > >Does the pypi download url need to be updated to be compatible with >svn setuptools? I am assuming no, but I am constantly getting 404's >from easy_install, yet I can paste the dl.sf.net link into a browser >and it works?! These files have been on sf long enough to be >propgated around, and constantly ~= 20 attempts on the command line. >I am seeing this with matplotlib, btw. Interesting. I just tried "easy_install matplotlib" and it worked the first time. I actually have never yet gotten a 404 or other problem using the dl.sf.net URLs on any package. However, a little experimentation shows that my Windows PC caches the DNS reply for an extended period, so I guess that means that if you get a bad mirror IP, you're stuck with it until the cache clears or you run "ipconfig /flushdns". (In contrast, my Linux machine returns a different IP every time the name is looked up.) A little experimentation with the socket module shows that I can get the full list of mirror IPs from Python, so I've changed setuptools in SVN to just randomly select one to use, which should fix the sticking problem on Windows (and any other platform where it occurs). It's not a perfect solution, since of course you can still end up with a bad mirror for some download, but I'm not sure what else to do, short of having some kind of option/configuration to control mirror selection.

2 2

expected non-zero return values when spawning subprocesses
by Stefan Seefeld Jan. 26, 2006

Jan. 26, 2006

Hi there, I'm using distutils.spawn.spawn() to run subprocesses during my build commands. It now happens that one of those subprocesses will return a non-zero value that doesn't strictly correspond to an error, and thus I'd like to catch it from within my python code. It appears spawn() itself will raise a message containing a stringified version of the exit code, but not the exit code itself. This seems to imply that, in order to discriminate on the exit value, I have to catch DistutilsExecError, and then parse the string. This seems rather unelegant. Is there some better way ? Could the exception be modified to contain the original code (as an integral value) ? Thanks, Stefan

2 1

SOURCES not working for me
by Ian Bicking Jan. 25, 2006

Jan. 25, 2006

I'm stumped as to why this package is not working properly: http://svn.pythonpaste.org/RhubarbTart/trunk The package data isn't being included when I build an egg (everything in rhubarbtart/paste_templates/). But SOURCES.txt lists all the proper files. I'm quite baffled. -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org

2 1

Re: [Distutils] Problem upgrading docutils with setuptools
by Phillip J. Eby Jan. 25, 2006

Jan. 25, 2006

At 06:46 PM 1/25/2006 +0100, Maciek Starzyk wrote: >Is there a way to tell setup.py to do the equivalent of easy_install -m >while upgrading its dependencies ? No, not really. You could always just bundle the "roman" module in your project, I suppose, or create a package for it on PyPI and make it a requirement of your project. These approaches wouldn't change docutils' behavior, but it'd ensure that *your* projects using docutils would work.

3 4

Re: [Distutils] Problem upgrading docutils with setuptools
by Phillip J. Eby Jan. 25, 2006

Jan. 25, 2006

At 02:53 PM 1/25/2006 +0100, Maciek Starzyk wrote: >Then upgrade to docutils 0.4. >Note that setup says "roman" module is already there. It was installed >as a part of docutils 0.3.9. At the moment when setup checks for >module roman - docutils 0.3.9 egg is still on the PYTHONPATH and therefore >"roman" is not installed. A simple fix is to do "easy_install -m docutils" before the upgrade, as this will remove the existing version from the .pth file. > But later on docutils 0.3.9 egg is removed from >PYTHONPATH and docutils 0.4 fails: > >EasyInstall docs say: > > installing a package automatically replaces > > any previous version in the easy-install.pth file > >So maybe this replacing should be done earlier - ensuring that the >previous version's code >is not in path - before actually running setup ? > >Or is it a bug in docutils' setup.py ? Well, docutils' setup.py is clearly trying to support Python versions older than 2.3, or else it would not be checking for textwrap and optparse. Setuptools only supports 2.3 and higher. So those modules shouldn't be an issue. If docutils were intended to use setuptools, then it would not make sense to check for 'roman', either, since an external dependency should be just that, a dependency. So in that case I would consider it a bug in their setup.py. However, since they clearly are trying to support 2.2 and maybe older versions as well, it's hard to call it that. Rather, I'd have to say it's a limitation of setuptools' ability to be backward-compatible with highly-customized setup scripts. Unfortunately, there is no way to remove the existing version from sys.path before the setup script runs, as before setup() is called, there's no way to know precisely which project needs to be removed. EasyInstall accepts URLs and directory names as well as project names, so it's not guaranteed that it will know what project it's building. EasyInstall also doesn't change sys.path at all under normal circumstances; it only changes the .pth file so that sys.path will be changed the next time Python is started. If I were to propose a change to docutils' setup.py, I would suggest that roman.py should either be bundled as part of the docutils package (i.e. always installed, but inside the docutils package directory), or else be treated as an external dependency (never installed by the setup script directly, only by the user or by setuptools' dependency handling). Either of these approaches is a robust way to deal with the issue. Checking for modules' presence at installation time, however, isn't a good practice at all; I found it to be fragile and error-prone for PEAK (which used to rely on zope.interface, among other things) before setuptools even existed.

3 4

setuptools include_package_data ignore data under a 'package'?
by bbangert＠groovie.org Jan. 25, 2006

Jan. 25, 2006

I've been having some strange issues when trying to get include_package_data to actually include package data, when said data is under a module. Though it seems to behave unpredictably. For example, several people have installed Pylons and its been missing pylons/media/style/orange.css (a package file), though I haven't had this happen myself. However, when I make a new template project, then bundle it into a distribution and unpack it (to see what setuptools thought was package data). It's missing several directories. Easy way to reproduce: paster create --template=pylons test cd test python setup.py sdist cd dist tar xzvf test-0.0.0dev.tar.gz Then compare: ls test-0.0.0dev/test/ To ../test/ The new one is missing the docs, public, templates directories which are package data directories containing content. If these aren't included on purpose, where can I get more information on exactly what include_package_data=True does actually include? BTW, I'm using setuptools 0.6a9 Thanks, Ben

4 6

Problem upgrading docutils with setuptools
by Maciek Starzyk Jan. 25, 2006

Jan. 25, 2006

Hi. Recently I had some troubles with clean update of docutils. 0.3.9 -> 0.4 It looks to me like a setuptools problem. See below: First install docutils 0.3.9: ----------------------- 8< ---------------------------------- D:\download\python\docutils>easy_install docutils-0.3.9.tar.gz Processing docutils-0.3.9.tar.gz Running docutils-0.3.9\setup.py -q bdist_egg --dist-dir c:\docume~1\mast\ustawi~1\temp\easy_install-cxyfp2\docutils-0.3.9\egg-dist-tmp-rcp8-e "optparse" module already present; ignoring extras/optparse.py. "textwrap" module already present; ignoring extras/textwrap.py. zip_safe flag not set; analyzing archive contents... Adding docutils 0.3.9 to easy-install.pth file Installing rst2latex.py script to C:\Python24\Scripts Installing rst2html.py script to C:\Python24\Scripts Installed c:\python24\lib\site-packages\docutils-0.3.9-py2.4.egg Processing dependencies for docutils==0.3.9 D:\download\python\docutils>python -c "import docutils.parsers.rst; print 'OK'" OK ----------------------- 8< ---------------------------------- Then upgrade to docutils 0.4. Note that setup says "roman" module is already there. It was installed as a part of docutils 0.3.9. At the moment when setup checks for module roman - docutils 0.3.9 egg is still on the PYTHONPATH and therefore "roman" is not installed. But later on docutils 0.3.9 egg is removed from PYTHONPATH and docutils 0.4 fails: ----------------------- 8< ---------------------------------- D:\download\python\docutils>easy_install -U docutils-0.4.tar.gz Processing docutils-0.4.tar.gz Running docutils-0.4\setup.py -q bdist_egg --dist-dir c:\docume~1\mast\ustawi~1\ temp\easy_install-hnciga\docutils-0.4\egg-dist-tmp-y4g2fn "optparse" module already present; ignoring extras/optparse.py. "textwrap" module already present; ignoring extras/textwrap.py. "roman" module already present; ignoring extras/roman.py. zip_safe flag not set; analyzing archive contents... docutils.parsers.rst.directives.misc: module references __file__ docutils.writers.html4css1.__init__: module references __file__ docutils.writers.newlatex2e.__init__: module references __file__ docutils.writers.pep_html.__init__: module references __file__ docutils.writers.s5_html.__init__: module references __file__ Removing docutils 0.3.9 from easy-install.pth file Adding docutils 0.4 to easy-install.pth file Installing rst2html.py script to C:\Python24\Scripts Installing rst2latex.py script to C:\Python24\Scripts Installing rst2newlatex.py script to C:\Python24\Scripts Installing rst2pseudoxml.py script to C:\Python24\Scripts Installing rst2s5.py script to C:\Python24\Scripts Installing rst2xml.py script to C:\Python24\Scripts Installed c:\python24\lib\site-packages\docutils-0.4-py2.4.egg Processing dependencies for docutils==0.4 D:\download\python\docutils>python -c "import docutils.parsers.rst; print 'OK'" Traceback (most recent call last): File "<string>", line 1, in ? File "c:\python24\lib\site-packages\docutils-0.4-py2.4.egg\docutils\parsers\rst\__init__.py", line 77, in ? from docutils.parsers.rst import states File "c:\python24\lib\site-packages\docutils-0.4-py2.4.egg\docutils\parsers\rst\states.py", line 110, in ? import roman ImportError: No module named roman ----------------------- 8< ---------------------------------- EasyInstall docs say: > installing a package automatically replaces > any previous version in the easy-install.pth file So maybe this replacing should be done earlier - ensuring that the previous version's code is not in path - before actually running setup ? Or is it a bug in docutils' setup.py ? Cheers, Maciek

1 0

Re: [Distutils] no access to resources unpacked as root
by Phillip J. Eby Jan. 23, 2006

Jan. 23, 2006

At 12:42 AM 1/24/2006 +0000, Matt Goodall wrote: >The Twisted process starts up as root to open low ports, etc and then, >once up and running, sheds privileges and continues to run as a normal >user. Many of the resources that are buried in the eggs are located with >resource_filename at module import time; some when the application is >still running as root. > >The resources are unpacked from the egg into /root/.python-eggs/ and >(correctly, I think) given 600 (rw-------) file access permissions. The >problem is that the user the application ends up running as cannot access >the unpacked resource files and fails with a PermissionDenied error. > >Now, I can move some uses of resource_filename to happens later but I >don't think I can replace all of them without changing APIs. > >Has anyone else tackled this problem or can anyone offer any advice on how >to avoid the problem in the first place? The simplest approach is probably to mark the eggs as not zip-safe, or to configure easy_install to install them unzipped.

1 0

no access to resources unpacked as root
by Matt Goodall Jan. 23, 2006

Jan. 23, 2006

Hi, I'm using eggs a fair amount to deploy Twisted-based server applications. (This problem is not specific to Twisted so please don't stop reading!) The Twisted process starts up as root to open low ports, etc and then, once up and running, sheds privileges and continues to run as a normal user. Many of the resources that are buried in the eggs are located with resource_filename at module import time; some when the application is still running as root. The resources are unpacked from the egg into /root/.python-eggs/ and (correctly, I think) given 600 (rw-------) file access permissions. The problem is that the user the application ends up running as cannot access the unpacked resource files and fails with a PermissionDenied error. Now, I can move some uses of resource_filename to happens later but I don't think I can replace all of them without changing APIs. Has anyone else tackled this problem or can anyone offer any advice on how to avoid the problem in the first place? Thanks in advance, - Matt

1 0

using setup.py to uninstall
by John M. Gabriele Jan. 20, 2006

Jan. 20, 2006

Can distutils (setup.py) uninstall packages as well? If not, is this a feature that's being worked on, or is there some reason why it may not be necessary? Thanks, ---J __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

3 2

Re: [Distutils] namespace_packages
by Phillip J. Eby Jan. 20, 2006

Jan. 20, 2006

At 12:44 PM 01/20/2006 -0500, Charlie Moad wrote: >Here are the eggs if you want to unzip and look at them. I installed >both unzipped. > >http://euclid.uits.iupui.edu/~cmoad/mpleggs/matplotlib-0.86.2cvs-py2.4-linu… >http://euclid.uits.iupui.edu/~cmoad/mpleggs/basemap-0.8-py2.4-linux-i686.egg There are two issues that I saw. First, the matplotlib egg above does not declare a namespace in its matplotlib/__init__.py. Second, the basemap egg does not contain a matplotlib/__init__.py at all. Please note that you *must* include an __init__.py file for every namespace package and parent package thereof, in *every* egg that's part of the namespace package, and that these __init__.py files must all contain a namespace declaration. Missing even one can cause problems. For example, the basemap egg is simply not importable, because as far as Python is concerned it does not contain a matplotlib package (due to the missing __init__.py).

2 1

Re: [Distutils] namespace_packages
by Phillip J. Eby Jan. 20, 2006

Jan. 20, 2006

At 11:20 AM 01/20/2006 -0500, Charlie Moad wrote: >On 1/19/06, Robert Kern <robert.kern(a)gmail.com> wrote: > > Charlie Moad wrote: > > > Well here are th cvs links to them. > > > > > > > http://cvs.sourceforge.net/viewcvs.py/matplotlib/matplotlib/setup.py?rev=1.… > > > > http://cvs.sourceforge.net/viewcvs.py/matplotlib/toolkits/basemap/setup.py?… > > > > > > The matplotlib.toolkits module does not exist in the mpl cvs and it is > > > not in the setup.py file. I have tried adding both. In basemap > > > however, the matplotlib.toolkits module does exist and is listed in > > > the setup.py file. > > > > Both of these setup.py files are buggy. The packages list needs to have > entries > > in dotted form ("matplotlib.toolkits"), not filesystem form > > ("matplotlib/toolkits"). Correcting them, and adding an empty > > lib/matplotlib/toolkits/__init__.py to the matplotlib checkout allows me to > > build eggs with namespace_packages set appropriately. > > > >I finally got time to try this, and it didn't help much. Changing all >the packages to dot notation works fine. I have added the >toolkits/__init__.py file. Setuptools warns about not having >declare_namespace in the init files. I have added this: > >try: > __import__('pkg_resources').declare_namespace(__name__) > print __name__, __file__ >except ImportError: > pass # must not have setuptools Note that if you don't have pkg_resources, namespace packages simply won't work, so there's little point in the try/except. >to __init__.py in mpl's and basemap's matplotlib and >matplotlib.toolkits module, hence in 4 places. >Upon starting python I get this output: > >matplotlib >/usr/lib/python2.4/site-packages/matplotlib-0.86.2cvs-py2.4-linux-i686.egg/matplotlib/__init__.pyc >matplotlib.toolkits >/usr/lib/python2.4/site-packages/matplotlib-0.86.2cvs-py2.4-linux-i686.egg/matplotlib/toolkits/__init__.pyc > >So the basemap declarations are never being reached. Where should I >go from here? Is the other egg you're looking for on sys.path? >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG(a)python.org >http://mail.python.org/mailman/listinfo/distutils-sig

2 2

Re: [Distutils] namespace_packages
by Phillip J. Eby Jan. 20, 2006

Jan. 20, 2006

At 11:44 AM 01/19/2006 -0500, Charlie Moad wrote: >I tried the "." and I keep getting: > >error in basemap setup command: Distribution contains no modules or >packages for namespace package 'matplotlib.toolkits' Great, now we're making progress. :) Can you post your setup.py? It sounds like you don't have 'matplotlib.toolkits' listed in your 'packages' keyword, or perhaps there's something amiss with your package_dir or something.

3 4

python setup.py test not working on Linux
by Gerhard Häring Jan. 19, 2006

Jan. 19, 2006

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I tried adding the test command to my setuptools-enhanced build process. And I got "python setup.py test" to work on Windows, now when I'm on Linux it fails. I *think* that it was in both cases the latest version of setuptools. You should be able to verify it with these sources: svn co http://initd.org/svn/pysqlite/pysqlite/trunk/ pysqlite_trunk -r 202 Oh, you need SQLite version 3.2.2 or later to build this pysqlite project. This is the full output: gerhard@lara:~/src/svn/pysqlite/trunk$ python setup.py test running test running egg_info writing requirements to pysqlite.egg-info/requires.txt writing pysqlite.egg-info/PKG-INFO writing top-level names to pysqlite.egg-info/top_level.txt warning: manifest_maker: standard file not found: should have one of README, README.txt reading manifest template 'MANIFEST.in' writing manifest file 'pysqlite.egg-info/SOURCES.txt' running build_ext copying build/lib.linux-i686-2.4/pysqlite2/_sqlite.so -> lib Traceback (most recent call last): File "setup.py", line 151, in ? main() File "setup.py", line 137, in main classifiers = [ File "/usr/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/usr/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/test.py", line 59, in run self.run_tests() File "/usr/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/test.py", line 73, in run_tests unittest.main(None, None, [unittest.__file__]+self.test_args) File "/usr/lib/python2.4/unittest.py", line 758, in __init__ self.parseArgs(argv) File "/usr/lib/python2.4/unittest.py", line 785, in parseArgs self.createTests() File "/usr/lib/python2.4/unittest.py", line 791, in createTests self.module) File "/usr/lib/python2.4/unittest.py", line 556, in loadTestsFromNames suites = [self.loadTestsFromName(name, module) for name in names] File "/usr/lib/python2.4/unittest.py", line 532, in loadTestsFromName parent, obj = obj, getattr(obj, part) AttributeError: 'module' object has no attribute 'test' - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFD0A/HdIO4ozGCH14RAu1YAJkBQmM/QyuDvCbJEhQ71ggBsy3Y1QCghUee K2b9G/o1QIKqKLWUezRezQw= =RuvI -----END PGP SIGNATURE-----

2 2

OS X problem with pkg_resources and the twisted event loop
by Michael Twomey Jan. 19, 2006

Jan. 19, 2006

Hi, I've come across a problem with an easy_install managed package (matplotlib) on my OS X box. When I try to import it for the first time inside of a twisted application I get this nasty traceback: Traceback (most recent call last): File "/Users/mtt/Desktop/twisted_macosx_vers.py", line 12, in ? reactor.run() File "/usr/local/lib/python2.4/site-packages/twisted/internet/posixbase.py", line 206, in run self.mainLoop() File "/usr/local/lib/python2.4/site-packages/twisted/internet/posixbase.py", line 214, in mainLoop self.runUntilCurrent() --- <exception caught here> --- File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py", line 541, in runUntilCurrent call.func(*call.args, **call.kw) File "/Users/mtt/Desktop/twisted_macosx_vers.py", line 4, in get_platform import pkg_resources File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/pkg_resources.py", line 534, in ? class Environment(object): File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/pkg_resources.py", line 537, in Environment def __init__(self,search_path=None,platform=get_platform(),python=PY_MAJOR): File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/pkg_resources.py", line 147, in get_platform version = _macosx_vers() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/pkg_resources.py", line 126, in _macosx_vers info = os.popen('/usr/bin/sw_vers').read().splitlines() exceptions.IOError: [Errno 4] Interrupted system call This seems to be caused by twisted's signal handling, which breaks os.popen (and subprocess), see this bug: http://twistedmatrix.com/bugs/issue733 I've attached a simple test script which triggers the problem (most of the time for me, you have to run it a few times as it doesn't always crash). I've got a work around though: If I import pkg_resources before starting my twisted app it seems to fix the problem. (I presume that get_platform's output is being kept somewhere, Environment's __init__ args I think.) cheers, Michael

2 1

making eggs and debs live together peacefully
by Frédéric Gobry Jan. 19, 2006

Jan. 19, 2006

Hi, I've started using setuptools as a developer quite recently, and appreciate the extra features. I've a question though, regarding the way dependencies are resolved when installing such a package: I've written a package that uses elementtree, and would have found natural to add it in install_requires. However, I've already installed it as a debian package, and in the current state setuptools tries to reinstall over it. So, should I add some extra logic to the setup.py script (ie, try the dependencies myself) or is there a way to tell it "don't update the dependence if you can already import the module" in a convenient way? Frédéric -- Frédéric Gobry Infoscience SISB / EPFL Tel: +41216932288 http://people.epfl.ch/frederic.gobry

5 6

Re: [Distutils] namespace_packages
by Phillip J. Eby Jan. 19, 2006

Jan. 19, 2006

At 01:35 PM 01/18/2006 -0500, Charlie Moad wrote: >Following the instruction for setuptools, I am trying to make >matplotlib and basemap (a mpl toolkit) share the namespace >"matplotlib/toolkits". I can build and install the eggs no problems. >"matplotlib/toolkits" is in both's "EGG-INFO/namespace_packages.txt" It should be matplotlib.toolkits, not matplotlib/toolkits. >files. I have added __init__.py files with >"__import__('pkg_resources').declare_namespace(__name__)" in both the >mpl and basemap matplotlib/toolkits folders. Basically I have done a >ton of playing around and "from matplotlib.toolkits import basemap" >will not work. It is always limited to the scope of the matplotlib >egg which it is hitting first. Did you run "setup.py develop" or "setup.py install" in both projects? > Are the docs not reflective of the >0.6a9 release? Any help is greatly appreciated. AFAIK, the docs are correct. If you can point me to the source code for your packages I can take a look at them.

2 1

namespace_packages
by Charlie Moad Jan. 18, 2006

Jan. 18, 2006

Following the instruction for setuptools, I am trying to make matplotlib and basemap (a mpl toolkit) share the namespace "matplotlib/toolkits". I can build and install the eggs no problems. "matplotlib/toolkits" is in both's "EGG-INFO/namespace_packages.txt" files. I have added __init__.py files with "__import__('pkg_resources').declare_namespace(__name__)" in both the mpl and basemap matplotlib/toolkits folders. Basically I have done a ton of playing around and "from matplotlib.toolkits import basemap" will not work. It is always limited to the scope of the matplotlib egg which it is hitting first. Are the docs not reflective of the 0.6a9 release? Any help is greatly appreciated. Thanks, Charlie

3 2

bundling packages with setuptools?
by Christopher Fonnesbeck Jan. 16, 2006

Jan. 16, 2006

I have a Python module for doing MCMC simulation that requires a number of prerequisite packages to run, all of which are open source (so I can build and distribute the packages that are needed). I am wondering if there is a way to use the power of python eggs to be able to have a single installer that installs or updates all the required packages. Is there a way of having the egg installer for my package go ahead and fetch the eggs of the prerequisite packages and install them? Thanks, C. -- Christopher J. Fonnesbeck Population Ecologist, Marine Mammal Section Fish & Wildlife Research Institute (FWC) St. Petersburg, FL Adjunct Assistant Professor Warnell School of Forest Resources University of Georgia Athens, GA T: 727.235.5570 E: chris at trichech.us

2 1

2 platforms down, only Mac to go...
by Phillip J. Eby Jan. 16, 2006

Jan. 16, 2006

I've got shared libraries working on Windows and Linux -- without any LD_LIBRARY_PATH nonsense, or any .so file rewriting. I have not had a chance to try this on OS X, however. If there's anybody out there who has experience wrestling with the Mac OS linker (hi, Bob! <wink>) has a chance to try it out and maybe tweak it a little bit, I'd appreciate it. The code is in the subversion trunk now, and the source includes a tests/shlib_test subdirectory that you can change to and run "setup.py test" to build/link/test, e.g.: cd tests/shlib_test PYTHONPATH=../.. python setup.py test This just verifies that the basic linking works, i.e., whether my guesses for the linker options in build_ext.py were correct. The second test is to verify that it works from a different directory: cd .. PYTHONPATH=..:shlib_test python -c "import hello" If this works without crashing, then the dynamic linking works even from a different directory. Based on my limited guesswork, the final test is the one most likely to fail on OS X (assuming the link step succeeded, and I'm not sure it will!). Rename the test directory temporarily: mv shlib_test something_else PYTHONPATH=..:something_else python -c "import hello" mv something_else shlib_test If this works without a bit of hacking, I'll be rather surprised, but *very* pleased. If all three tests work without any changes to the source, it's probably a bug. ;) (Specifically, it probably means setuptools fell back to static linking, and thus isn't having any dynamic link problems.) The basic idea of what I'm doing here is that I try to set the runtime library search path of each extension (that depends on a shared library in the project), to include the current directory *at runtime*, then make a Python wrapper for the extension that temporarily changes the current directory while doing the import. The tricky bit -- if I understand correctly -- is that OS X has no concept of a runtime library search path, so what this probably *should* be doing is specifying something like "-dylib_file libhellolib.dylib:./libhellolib.dylib" in the link options for the 'hello' extension. What I'm not clear on is whether this is actually allowed by the linker, or if it even does what I think it does in the first place. Even if it's allowed and does the right thing, I have a sneaky suspicion that perhaps the path will be converted to an absolute path at link time, rather than leaving the './' in place, given the general reputation for deviousness possessed by the Mac OS linker. :) On the other hand, perhaps it's possible to fudge the paths in the dylib once it's built, such that the './' is in the file? Any thoughts or suggestions would be welcome. In a couple of weeks I'll have a chance to play around with this on a Mac myself, but since I'm starting from scratch (zero Mac experience whatsoever), I'm hoping maybe someone more experienced could provide some degree of guidance by then. Thanks.

3 6

Re: [Distutils] bundling packages with setuptools?
by Kevin Dangoor Jan. 16, 2006

Jan. 16, 2006

(readding distutils-sig which I accidentally dropped) On 1/16/06, Christopher Fonnesbeck <chris(a)trichech.us> wrote: > On Jan 16, 2006, at 11:54 AM, Kevin Dangoor wrote: > > > Yes. This is something setuptools does quite admirably. This is in the > > setup.py for TurboGears: > > > > install_requires = ["TurboKid >= 0.9.0", > > "CherryPy > 2.1.1", > > "SQLObject >= 0.7.1dev_r1457", "simplejson >= 1.1", > > "elementtree >= 1.2.6", "PasteScript >= 0.4.1", > > "cElementTree >= 1.0.2", "FormEncode >= 0.4", > > "setuptools >= 0.6a8", > > "RuleDispatch"], > > > > Running easy_install on the TurboGears package will go and fetch all > > of those pre-requisites. > > > > So, easy_install is the installer, and your egg includes the metadata > > about which packages it needs. > > This is what I was hoping. What about URLs? The packages dont have to > be in the cheese shop, do they? Nope. TurboGears uses some packages that aren't in the Cheeseshop. You can use the -f/--find-links option to tell easy_install where to look. You can see the TurboGears installation instructions to see the full process: http://www.turbogears.org/download/index.html Generally, pretty easy. If you have all of the eggs available on your local machine, you can also have setuptools run with those eggs. Kevin

2 1

setuptools in 2.5?
by Jim Fulton Jan. 16, 2006

Jan. 16, 2006

Has a decision been made whether to include setuptools in 2.5? If it is going to be included, then I'll update doctest to use pkg_resources to find doctest text files. If it isn't going to be included then I'll have a quandry. :) I guess that in that case, I'd have to use some conditional logic. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

3 2

IRC?
by Jim Fulton Jan. 15, 2006

Jan. 15, 2006

Is there an IRC channel that people knowledgable about setuptools hang out on? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

2 1

eggs in Python path
by Jim Fulton Jan. 15, 2006

Jan. 15, 2006

The eggs quick guide http://peak.telecommunity.com/DevCenter/PythonEggs#using-eggs says that eggs can be installed by just putting them on sys.path. This doesn't seem to be enough though. A .pth file seem to also be necessary. Am I missing something? Or is the documentation incorrect? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

3 3

License for virtual-python.py?
by Jim Fulton Jan. 15, 2006

Jan. 15, 2006

virtual-python.py looks very useful. It lacks a license/copyright statement though. Can we get this? Are there plans to include this script in 2.5? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

2 1

Dynamic linking on Linux, Mac OS, et al
by Phillip J. Eby Jan. 14, 2006

Jan. 14, 2006

Hi folks. I just implemented preliminary shared library build support for setuptools, but I ran into a bit of a snag. I was operating under the assumption that you could simply put shared libraries in the directory alongside the extensions that use them, but in practice it turns out that this only works on Windows. On Linux I found that you could only put shared libraries in a directory on LD_LIBRARY_PATH, and I suspect a similar issue will be found under OS X and other Unixes. This seems to be a bit of a problem, in that it appears to mean there's no sane way to include shared libraries in an egg on any platform other than Windows. It's common practice for Python extensions built for Windows to include shared libraries to avoid having to install them in the Windows system directories, but for Unix-like OSes this isn't really the normal practice anyway. But, in the case where a developer wants to include a shared library as part of the distribution, this remains somewhat problematic. One can of course install the library to a platform-appropriate location ($prefix/lib on most platforms), but to do it from an egg is harder. You can't simply unzip the egg any more, you would have to also move the libraries to a standard installation location, breaking any hope of multi-version support. (And this would have to be able to happen on the fly, when shared libraries and extensions are extracted to the cache directory, if running zipped.) I don't suppose you can modify LD_LIBRARY_PATH on the fly? Any other options? Should I just give up on supporting shared libraries on anything but Windows, perhaps silently converting the shared libraries to static ones?

5 8

setuptools: branch versions
by Ian Bicking Jan. 13, 2006

Jan. 13, 2006

One thing that would be nice, but seems rather substantial, would be a way to tag branches. Right now 0.1dev and 0.1wsgi are ordered in some way, and for branches I'd actually like for them not to be ordered -- you really want one tag or the other, and they aren't interchangeable in any way. Which maybe is possible now. Though the version specifiers make it hard too, because how do you say you want 0.1wsgi-*? Ultimately I'm looking for a "tag_svn_branch = true" setting, I guess, but the version sorting makes that less useful. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

2 1

making easy_install install extras
by Andrew Straw Jan. 12, 2006

Jan. 12, 2006

Hi, Is it possible to convince easy_install to grab extras specified in the "extras_require" arguments to setup()? I know this probably isn't something that should be on by default (that's why they're "extras"), but it would be nice to do something like: easy_install some_package --also-install-all-available-extras If it's not currently possible, I'd like to submit this as a feature request. Cheers! Andrew

2 4

man pages (& gzip & bdist_rpm on SuSE, oh my!)
by Charles Duffy Jan. 12, 2006

Jan. 12, 2006

I'm trying to use bdist_rpm to build RPMs for a number of 3rd-party packages on SuSE SLES9. Among the issues I've run into: SuSE has a different %mandir from Red Hat -- man pages go under /usr/share/man rather than /usr/man. Upstream setup.py specifies data_files=[('man/man1', ['bzr.1'])]. I can change this to reference 'share/man/man1', which lets me build my package -- but it'd be nice to have something I can distribute upstream. Do I need to actually look at the filesystem and determine whether /usr/share/man exists and pass in the appropriate path, or is there a better approach? In /usr/lib/rpm/suse-macros, an __os_install_post macro is specified which calls %{suse_check} and then /usr/lib/rpm/brp-compress. Either of these will compress man pages (among other files), should it find them. This leads to the issue described in http://mail.python.org/pipermail/distutils-sig/2001-December/002674.html. Other than temporarily modifying /usr/lib/rpm/suse-macros, are there any ready workarounds? Thanks!

1 0

Re: [Distutils] Dynamic linking on Linux, Mac OS, et al
by Phillip J. Eby Jan. 11, 2006

Jan. 11, 2006

At 02:11 PM 1/6/2006 +0800, Jeff Pitman wrote: >Unless you do something really magical, you're going to find a lot of >resistance to this in most Linux distro camps. Gentoo will want to >recompile via emerge. Fedora/CentOS will want to grab from yum. Debian >will want to grab libs via apt. The issue for the PyICU/PyLucene etc. use cases is that there are a whole bunch of SWIG wrappers for different extensions, but with a large common library for type mapping etc. The authors want a shared library for *their own code* that is used by multiple extensions. This is not separately distributed code. This is not the same thing as the Windows use case for building a shared library, although the ability to do it on Windows satisfies this use case too.

4 4

setuptools: confusion with non-egg installed packages
by Ian Bicking Jan. 10, 2006

Jan. 10, 2006

I had a confusing installation experience today. Trac was installed in /usr/local/lib/python2.4/site-packages/trac, and I checked out a version that I had hacked to use setuptools, and run "python setup.py develop" on. This added my checkout to the path, but after site-packages, so it wouldn't be loaded, even if I did require('trac'). Now, if I did setup.py develop -m it would actually work, because setuptools would fix up the path correctly. I don't know the best resolution here. Maybe develop should check for conflicting installed packages? -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org

2 1

Re: [Distutils] Dynamic linking on Linux, Mac OS, et al
by Phillip J. Eby Jan. 10, 2006

Jan. 10, 2006

At 12:59 PM 1/7/2006 +0800, Jeff Pitman wrote: > If you need .so that's built and exported by the same packaged egg, you > may just consider installing it in a more benign area such as /usr/local/lib. As I mentioned before, this breaks multi-version support, although I suppose one could perhaps force the egg version to always be included in the filename... that would lead to some interesting problems as far as keeping the build area clean, though! You'd have to run a "clean --all" whenever the version number changed, and the extensions using the library would have to refer to the exact version. Unfortunately, this would *still* require users to mess with LD_LIBRARY_PATH in order to use private library versions, and so isn't particularly useful. Some variant of the "patch the extensions' runtime path at installation/cache extraction time" approach would still be necessary. There is actually a hook in pkg_resources whose purpose was to support doing this kind of thing with any code resources extracted to the egg cache, so it's probably doable, and I could probably also make the code usable by easy_install during egg unpacking. It pretty much means the "unzip the egg manually" concept is a dead duck when libraries are involved, though. You could only safely unpack an egg using a manually-created ResourceManager with an appropriate extraction path base. On Mac OS there's an option to automatically pad the rpath area, but on other OSes I'd need to do the padding myself by supplying an extra-long rpath entry that could then be overwritten during the extraction process. What bothers me about all of this is that moving an installed egg then *breaks* it, because the rpath is now hard-wired. Sigh. Ironically enough, an egg containing a shared library would then need to be labeled as "not unzip safe", i.e., it's really only safe to use if it remains zipped. A new category of egg, to be sure. :) Interestingly, this is one of the few times where I have to say that Windows is actually better-designed than Unix. Sure, the same design choice enables all other sorts of DLL hell, but on this very narrow specific issue Windows kicks ass. :) Anyway, I really wish there were a way to hook into the dynamic link process that didn't require one to manage the entire linkage one's self. The dl.open() function lets you get access to symbols, but that doesn't seem sufficient. :(

4 3

New features in setuptools SVN
by Phillip J. Eby Jan. 9, 2006

Jan. 9, 2006

I just checked in a couple of new features for EasyInstall that some of you will be interested in. First, '.' is now allowed in project names, and '_' and '-' in names should no longer confuse EasyInstall when it searches PyPI. It will use whatever characters you originally entered. The bad news is that this could break any existing dependencies to projects with a '.' in their names, because the '.' was being converted to '-' before, and isn't now. The good news is that there are only 5 PyPI projects with a '.' in the name (rsync.py, spider.py, _sre.py, zope.interface, and zope.testbrowser), and of them only '_sre.py' had a prayer of working with EasyInstall before. Now, 'spider.py' also works due to the next new feature, which is that installation of bare '.py' files (no accompanying setup.py) is now supported. If a PyPI "Download URL" points to a file with a .py extension, EasyInstall will automatically create a proper setup.py alongside it. It can also do this for any link to a .py file that includes an '#egg=name-version' fragment identifier, so that you can use a link like this: http://bellsouthpwp.net/m/e/mefjr75/python/PySourceColor.py#egg=PySourceCol… to point to somebody's .py file and assign it a project name and version. Note that the '#egg' tag must be *unambiguous*, which means you cannot have a '-' in the project name or version. You must escape them as '_' characters, using the '-' only to delimit the name from the version. Anyway, if you do this on *your* project's PyPI page to link to your project's unpackaged dependencies, you can fill in for the dependency's lack of a PyPI page or setup script, while still using the normal setuptools way of specifying project dependencies. Please let me know if you have any questions or problems using these new features. Thanks!

1 0

Dynamic linking - another idea
by Phillip J. Eby Jan. 7, 2006

Jan. 7, 2006

Okay, so here's a crazy idea to make dynamic libraries in the same package work. Suppose the linker includes an rpath of '.' (aka os.curdir), and renames each extension module that's using a library such that it's not directly importable by Python. Setuptools already generates a stub .py file loader for extension modules anyway during bdist_egg, so that zipped extensions can be extracted. So, this stub could just change directory to the directory of the extension before doing the import, with setdlopenflags(RTLD_NOW). If I understand correctly, this should force all symbols in the extension to be resolved immediately, so the '.' rpath would let the extension load the nearby library. After this was done, the stub code would restore the original current directory and dlopen flags. Since extensions built this way would not be directly importable, it would ensure that the stub is always used. And since they do not require a hardcoded real path, they would be relocatable and manually extractable from the zipfile. Does any see any holes in this, aside from the sheer craziness of doing it? :) That is, are there any platforms where a '.' rpath is forbidden? Where RTLD_NOW won't let it resolve all the symbols? Where the '.' rpath is interpreted as the executable's startup directory rather than as the load-time current directory?

1 0

Re: [Distutils] Dynamic linking on Linux, Mac OS, et al
by Phillip J. Eby Jan. 6, 2006

Jan. 6, 2006

At 12:29 PM 1/6/2006 +0800, Jeff Pitman wrote: >Are these just extensions that are imported? Because if they are, then you >don't need to mess with LD_LIBRARY_PATH and you can put them anywhere on >Linux as long as PYTHONPATH is correct. -- -jeff Nope. They're shared libraries being built by the distribution itself, as in the case of PyICU, and certain Windows C extensions that need to include the target library (e.g. sqlite) because it's not a "system" library there. It could perhaps be questioned whether it's a good idea to include any other kind of shared library in an extension project, and in fact the common practice to link between Python extensions is to use a Python CObject to wrap an API function pointer table. This neatly bypasses all of the issues... *if* you control the library in question. So far I've done a few experiments with trying to hack LD_LIBRARY_PATH at runtime, use dl.open() with RTLD_GLOBAL, etc. to force finding the right library, but so far it seems like there's no way to get the extensions to consider the opened library to be "the same" as the one they're looking for. (Although it may just be that I'm missing some trick on the build side of the equation.)

3 7

Automatically going for a pre-req dev
by Ben Bangert Jan. 6, 2006

Jan. 6, 2006

When installing a package that requires the dev version of another package, it fails to to and download a dev version of the package. This appears to be because it either searches the filesystem for the prereq version, or asks CheeseShop. Given that the prereq indicates it wants a dev version, it would seem reasonable to search CheeseShop with ==dev equivalent active. As an example, if you install: easy_install -U hellahella==dev It installs fine from svn, then goes back searching for Pylons, and gets: Processing dependencies for hellahella==0.1dev-r576 Searching for Pylons>=0.1dev-r289 Reading http://www.python.org/pypi/Pylons/ Reading http://pylons.groovie.org/ No local packages or download links found for Pylons>=0.1dev-r289 error: Could not find distribution for Requirement.parse ('Pylons>=0.1dev-r289') Yet, if you run: easy_install -U Pylons==dev It has no problem fetching the latest Pylons and the Paste prereqs (which would have the same problem if I hadn't linked to them explicitly on the Pylons page). Since you already indicate you want ==dev, shouldn't it keep that during the prereq if necessary? - Ben

3 5

[setuptools] When .py files are "data", strange things happen
by David Handy Jan. 5, 2006

Jan. 5, 2006

I want setuptools to install a bunch of sample scripts in a package-relative directory. Each end user will later run a script that copies these to a directory relative to his or her home directory. However, strange things happen to data files that have a .py extension. 1) Saying "include_package_data = True" was not enough to get them included in the egg. The .txt files were included, but the .py files were magically excluded. 2) I overcame the above problem with the following setup parameter: package_data = { # Include all .py files in the data directories, they # are excluded by default. 'cpif.data': ['lander/*.py', 'samples/*.py'], } But now I get a bogus .pyc file generated for each of these sample scripts. These bogus .pyc files get included in the egg. 3) I tried to overcome problem #2 above with the following setup parameter: exclude_package_data = { # The example programs should not be shipped with .pyc files # XXX This is not working, the .pyc files are being created and # included anyway. 'cpif.data': ['lander/*.pyc', 'samples/*.pyc'], } But this didn't work. Putting the following lines in my MANIFEST.in file also didn't work: exclude cpif/data/lander/*.pyc exclude cpif/data/samples/*.pyc I suppose I could turn off .pyc file generation, but that's not what I want. I just want to treat certain .py files as data files. The "data" .py files are in a subdirectory of a python package, but the directory in which they directly reside does not contain an __init__.py file, so they are not directly importable. So I don't think .pyc files should be generated for them in any case. Thanks in advance for ideas/sympathy/fixes, etc. -- David Handy Computer Programming is Fun! Beginning Computer Programming with Python http://www.handysoftware.com/cpif/

3 3

Re: [Distutils] [setuptools] When .py files are "data", strange things happen
by Phillip J. Eby Jan. 5, 2006

Jan. 5, 2006

At 01:48 PM 1/5/2006 -0800, Ben Bangert wrote: >On Jan 5, 2006, at 9:07 AM, Phillip J. Eby wrote: > >>If I understand your use case correctly, I don't see how that hurts >>anything. > >It hurts a ton if you want to use Paster templates, because you'll >have files ending in .py that aren't meant to be processed by paster, >and _tmpl, all of which are inside non-package data directories, thus >won't get included any other way. You lost me. How does having the .py included and compiled to .pyc hurt anything? .pyc files include a date/time stamp of the .py they were built from, so if you accidentally copy them somewhere and change the corresponding .py file, the .pyc will be ignored and overwritten in the import case, and ignored altogether in an execfile or script-running case. In no scenario that I understand, does the presence of a .pyc file hurt anything. Could you please explain what problem you think it will cause?

1 0

setuptools 0.6a9 is released
by Phillip J. Eby Jan. 4, 2006

Jan. 4, 2006

This version contains mostly bug fixes and robustness improvements in a variety of areas, ranging from improved web spidering to better handling of spaces in the Windows directory names for generated .exe files. The biggest new feature included, however, is system packager support including improved bdist_* command handling, and support for installing projects in a "single version, externally managed" format. You can now also build RPMs and Windows installers of setuptools-baseds project without any special steps, even if the project uses egg metadata. The other new features are for developers of setuptools-based projects, and mostly have to do with improved handling of source distribution manifest generation and specifying package data to be included in a project. You can now simply specify "include_package_data=True" and any file under revision control or listed in your source manifest (or MANIFEST.in) will be installed as part of the package. There is also an 'exclude_package_data' option that can be used to trim back the list, and the 'package_data' option can still be used to add files that aren't in revision control or the source manifest. Finally, there have been many documentation updates, including an overhaul of the installation instructions and a new "What your users should know" section of the setuptools manual for project developers. The next release of setuptools will be 0.6a10, for which I plan to add shared library building support (it's needed for an OSAF project, PyICU). There are also some minor features (like dependency_links) that didn't make it into 0.6a9 but which aren't big enough to need waiting till 0.7. Special thanks to Kevin Dangoor, Ian Bicking, and everyone else who reported bugs or helped track them down. Thanks to your participation, 0.6a9 is in much better shape than 0.6a8, and I look forward to a great 0.6a10!

2 4

[setuptools] setuptools and fat binaries
by Ronald Oussoren Jan. 4, 2006

Jan. 4, 2006

Hi, Apple supports fat binaries on Mac OS X (they call them universal binaries), that is binaries that contain executable code for multiple architectures. In released version of the os this can be used to build binaries and libraries that support both PPC and PPC64 architectures, which isn't used very much, and in the future this will be the mechanism to support both the current PPC based systems and the to-be-introduced intel systems with a single binary. I'm currently playing around with Python on an Intel developer system (my goal is a patch that will make it easy to build a fat/universal build of Python on Mac OS X), and I wondering how setuptools (and specifically eggs) can support universal binaries. The reason I ask is because of eggs that contain extensions, whose name contains the architecture, such as aem-0.10.0-py2.4-macosx-10.4-ppc.egg. Just using two eggs, one for each architecture, would work but seems wasteful and would make it harder to create self fat application bundles. Ronald

2 4

Re: [Distutils] Setuptools' goal in rpm/deb-based systems
by Phillip J. Eby Jan. 1, 2006

Jan. 1, 2006

At 11:05 PM 12/29/2005 +0800, Jeff Pitman wrote: >I see that --single-version-externally-managed has been added as an >option so that .deb/.rpm packaging can take advantage of that relying >upon apt/yum/smart to take care of business. Hopefully, this will be >pushed out soon. FYI, unless there's some policy preventing it, you can use the current SVN snapshot as a basis for a Debian package of setuptools. If built from a source checkout, it will correctly include its Subversion revision number as part of its name, e.g. "0.6a9dev-r41780". In setuptools' version numbering, this is an *older* release than "0.6a9", so when the final 0.6a9 comes out it will replace the "dev" versions. I realize that this isn't necessarily the same thing as how Debian package versions work, but I assume it must have some facility for pre-release tags. If all else fails, you could treat it as "0.6a8-r41780" instead, which might be a more compatible numbering approach. At this point, the only thing left on my to-do list for 0.6a9 is more documentation for EasyInstall, so don't let that stop you from packaging it. The only reason to wait would be to see if anybody reports bugs with the current version, which probably won't happen until *after* 0.6a9 is released. :)

2 1