Mailman 3 February 2006 - Distutils-SIG

pkg_resources and Mac OS X universal builds
by Bob Ippolito 07 Feb '06

07 Feb '06

Modern Mac OS X applications need to support both the i386 and ppc architectures in what's called a universal build (AKA fat in NeXT- ese). Ronald Oussoren and I are very far along in making python24- maint capable of doing this with patches all over the build procedure and a few smaller patches to distutils itself. One such patch basically expands upon the macosx platform string from pkg_resources, with the additional clause that a fat build of Python (one configured to build multi-architecture extensions) will report its architecture as "fat" rather than "i386" or "ppc". In order for setuptools' pkg_resources to inherit this behavior, I need the attached patch to be accepted. Basically it just skips over the custom behavior if get_platform already returns a sanitized value. -bob

2 6

Supporting arch specific eggs with a fat Python and setuptools
by Bob Ippolito 07 Feb '06

07 Feb '06

With the recent introduction of i386 architecture Macs, it's becoming a necessity to support two architectures for the platform: PPC and i386. Fortunately this is done somewhat easily using Apple's GCC 4 compiler and their linker toolchain. Ronald and I have already basically made all of the necessary changes to Python (in a branch on another svn server) and distutils in order to make Python compile with multiple architectures, and to make distutils also do the right thing. This solution works great (in theory and limited testing) for everyone except Mac OS X 10.3 users. We have gone out of our way to ensure that this build of Python is compatible with Mac OS X 10.3.9 so that those users aren't totally left in the dark, but there is a problem: the GCC 4 toolchain required to build extensions is not available to Mac OS X 10.3 users. They simply can not build binaries compatible with i386.. and if they could, they couldn't link it together as a universal binary. Bummer. The workaround for this issue is to have a way to swap out the Makefile with a PPC-only Makefile for those users. The problem with that is setuptools. Currently setuptools doesn't include a lot of intelligence for choosing eggs given multiple choices. Here's a scenario of various Python builds and the naming of eggs they produce: Mac OS X 10.4 with current means: pysqlite-2.0.5-py2.4-macosx-10.4-ppc.egg Mac OS X 10.3 with current means, or with the universal build and a swapped Makefile: pysqlite-2.0.5-py2.4-macosx-10.3-ppc.egg Mac OS X 10.4 with the universal build: pysqlite-2.0.5-py2.4-macosx-10.3-fat.egg The decision of which egg to choose is subjective, but the greatest benefit overall is achieved by choosing the maximally compatible version: pysqlite-2.0.5-py2.4-macosx-10.3-fat.egg For users of Mac OS X, here is the egg-sorting algorithm I'm proposing. This sort must be used only with eggs that are compatible with the current architecture -- which means either "fat" or _macosx_arch(os.uname()[4].replace(' ', '_')). Also allowing _macosx_arch and universally allowing fat are going to require another patch to setuptools (that I have not yet written, because it's not the right thing to do until egg-sorting is right). # a and b must only differ in platform specifics! def cmp_platform_eggs(a, b): if a.arch == "fat" and b.arch == "fat": pass elif a.arch == "fat": return 1 elif b.arch == "fat": return -1 # lower version sorts higher return -cmp(a.macosx_version, b.macosx_version) This sort means that fat will always be preferred, and lower requirements sort higher. This makes it easier for people who wish to redistribute self-contained applications to other users (a la py2app, py2exe, cx_Freeze), which is a very important use case for this platform. -bob

2 1

Swig support
by Tessa Lau 06 Feb '06

06 Feb '06

Hi, I'm using distutils with python 2.4.2 on Linux and trying to distribute a complex package including an extension module. I'm using SWIG to wrap the extension module. If I invoke swig normally from the command line with "swig -python notes.i", then swig generates a "notes.py" wrapper as well as a "notes_wrap.c" source code file. It expects this source file to be compiled into a module named "_notesmodule.so" (on Linux). I tried recreating the same functionality with distutils, using the following setup.py code: ext_modules = [Extension('mypkg.notes', ['wrapper/notes.i'], include_dirs=['include'], swig_opts=['-Iinclude'])] What happens is that distutils converts "notes.i" into "notes.so" (which cannot be imported as a module) and fails to install the generated "notes.py" wrapper script, which is also required for the module to work. The "notes.so" built by distutils cannot be imported by python because the swig-generated C source code declares the module name to be "_notes", and this doesn't correspond to the name of the created shared library. This is similar to the shadow class behavior found in earlier releases of swig, but now it's the default behavior. Is this supposed to work? Am I doing something wrong, or is SWIG support just broken in this release? Thanks, --Tessa

1 0

Easy install, making "Traditional" PYTHONPATH-based Installation work better
by Jim Fulton 05 Feb '06

05 Feb '06

I'be been learning about setuptools and eggs and am very pleased. There is one area where I'd like to see a small tweak. In: http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-loc… four options are presented for installing outside of Python installation. Naturally, I prefer the the last option, which the documentation recommends against. :) Here's why. An important use case to me is the ability for a developer to work in a development sandbox without affecting anything outside the sandbox. The solution needs to be cross platform. The "Administrator Installation" option doesn't work for me because it requires modifying the Python setup. Neither the "Mac OS X "User" Installation" nor the 'Creating a "Virtual" Python' options work for me because they are not cross platform. I also find the "Virtual Python" approach a bit heavy handed. In playing a bit, I find the "Traditional" PYTHONPATH-based Installation works pretty well except for the problem of having to set the path manually when invoking scripts. This problem could be addressed by having the generated scripts include the custom path settings. I recommend adding two new options to the easy_install section of the configuration file: script_path A list of paths, separated by commas to be added to sys.path by generated scripts. script_pth_files A list of pth files, separated by commas to be added be processed by script files. Generated scripts would include code to manipulate sys.path based on this information. So, for example, in a workspace with a 'lib' directory containing eggs, I'd have: [easy_install] site_dirs = '/here/lib' script_path = '/here/lib' script_pth_files = '/here/lib/easy-install.pth' The script_pth_files option is needed, I think, to avoid hardcoding the setuptools egg path in the generated scripts. Note that it could be argued that the site_dirs option should be enough and that generated scripts should simply reflect this option. Thoughts? For now, I can create a wrapper around easy_install that does custom script generation, which could serve as a prototype for the proposed change. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

2 3

Re: [Distutils] Twisted and easy_install
by Phillip J. Eby 05 Feb '06

05 Feb '06

At 07:05 AM 2/5/2006 -0800, Jay Parlar wrote: >I'm having trouble doing an install of the newest Twisted from the >Cheeseshop, using easy_install. I took a quick look through the >archives of distutils-sig, and couldn't find anything. I'm running on >OS X 10.3.9, latest version of setuptools (did an 'easy_install -U >setuptools' right before I tried installing Twisted). I've pasted the >result below. Any thoughts? [...] >AttributeError: 'bool' object has no attribute 'name' Twisted does some interesting mangling of the build_ext process in order to support optional extensions, that are only determined at build time. I've been meaning to add a facility to do this in setuptools, but it's not done yet, and in any case Twisted would have to *use* that facility in order for it to work. I'm afraid Twisted isn't supportable by easy_install at the present time. When I do add support for optional extensions, I plan to look carefully at what Twisted is doing in order to make sure that the facility I add will be able to meet their needs.

3 2

Twisted and easy_install
by Jay Parlar 05 Feb '06

05 Feb '06

I'm having trouble doing an install of the newest Twisted from the Cheeseshop, using easy_install. I took a quick look through the archives of distutils-sig, and couldn't find anything. I'm running on OS X 10.3.9, latest version of setuptools (did an 'easy_install -U setuptools' right before I tried installing Twisted). I've pasted the result below. Any thoughts? Searching for twisted Reading http://www.python.org/pypi/twisted/ Couldn't find index page for 'twisted' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://www.python.org/pypi/ Reading http://www.python.org/pypi/Twisted/2.1.0 Reading http://www.twistedmatrix.com Reading http://twistedmatrix.com/projects/core/ Best match: Twisted 2.1.0 Downloading http://tmrc.mit.edu/mirror/twisted/Twisted/2.1/Twisted-2.1.0.tar.bz2 Processing Twisted-2.1.0.tar.bz2 Running Twisted-2.1.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ssfbd7/Twisted-2.1.0/egg-dist-tmp--DvGj- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.4/bin/easy_install", line 7, in ? sys.exit( File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 1138, in main setup(script_args = ['-q','easy_install', '-v']+argv, **kw) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 230, in run self.easy_install(spec, not self.no_deps) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 317, in easy_install return self.install_item(spec, download, tmpdir, deps) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 338, in install_item dists = self.install_eggs(spec, download, tmpdir) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 532, in install_eggs return self.build_and_install(setup_script, setup_base) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 807, in build_and_install self.run_setup(setup_script, setup_base, args) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/easy_install.py", line 796, in run_setup run_setup(setup_script, args) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/sandbox.py", line 26, in run_setup DirectorySandbox(setup_dir).run( File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/sandbox.py", line 63, in run return func() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/sandbox.py", line 29, in <lambda> {'__file__':setup_script, '__name__':'__main__'} File "setup.py", line 113, in ? File "./twisted/python/dist.py", line 69, in setup File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/bdist_egg.py", line 167, in run self.run_command("egg_info") File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/egg_info.py", line 129, in run writer(self, ep.name, os.path.join(self.egg_info,ep.name)) File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/command/egg_info.py", line 321, in write_toplevel_names pkgs = dict.fromkeys( File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/setuptools-0.6a9-py2.4.egg/setuptools/dist.py", line 630, in iter_distribution_names yield ext.name AttributeError: 'bool' object has no attribute 'name' Jay P.

1 0

Your message to freebsd-isp awaits moderator approval
by owner-freebsd-isp＠freebsd.org 01 Feb '06

01 Feb '06

Your mail to 'freebsd-isp' with the subject Spam Is being held until the list moderator can review it for approval. The reason it is being held: SpamAssassin identified this message as possible spam Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: http://lists.freebsd.org/mailman/confirm/freebsd-isp/bb6585809095b7ce2f6ebb… PLEASE NOTE! If you would like to post freely to the list, please subscribe first. If you post from multiple addresses, you can subscribe each address and go into the options page and select 'no mail' for all but one address. This will allow you to post without delay in the future. Sorry for the hassle, but certain immature people made this necessary.

1 0