Mailman 3 May 2009 - Distutils-SIG

PEP 376 up-to-date
by Tarek Ziadé 08 Jun '09

08 Jun '09

Hello, I have updated PEP 376, mainly with Phillip's feedback, and changed the API section so it looks like the current prototype. - PEP : http://svn.python.org/view/peps/trunk/pep-0376.txt?view=markup - prototype code : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py I'll wait for a new round of feedbacks, Thanks ! Tarek -- Tarek Ziadé | http://ziade.org

4 8

Mutual recursion between setuptools and distutils with Python 2.7
by Antoine Pitrou 01 Jun '09

01 Jun '09

Hello, I'm not sure where to post this (distutils or setuptools bug?), so I'm trying here first. Using the CPython trunk, there's an infinite recursion due to a mutual recursion between setuptools and distutils. Here is a snippet of a traceback (trimmed for obvious reasons): $ usr/bin/easy_install -Z -i http://www.turbogears.org/2.0/downloads/current/index tg.devtools Searching for tg.devtools Reading http://www.turbogears.org/2.0/downloads/current/index/tg.devtools/ Best match: tg.devtools 2.0 Downloading http://www.turbogears.org/2.0/downloads/current/tg.devtools-2.0.tar.gz Processing tg.devtools-2.0.tar.gz Running tg.devtools-2.0/setup.py -q bdist_egg --dist-dir /home/antoine/tmp/easy_install-AkgVEJ/tg.devtools-2.0/egg-dist-tmp-DQjVL4 Traceback (most recent call last): File "usr/bin/easy_install", line 8, in <module> load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')() File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1671, in main File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1675, in <lambda> File "/home/antoine/cpython/__svn__/Lib/distutils/core.py", line 149, in setup dist.run_commands() File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 929, in run_commands self.run_command(cmd) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 211, in run File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 446, in easy_install File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 476, in install_item File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 655, in install_eggs File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 930, in build_and_install File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 919, in run_setup File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 27, in run_setup File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 63, in run File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 29, in <lambda> File "setup.py", line 53, in <module> return None File "/home/antoine/cpython/__svn__/Lib/distutils/core.py", line 149, in setup dist.run_commands() File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 929, in run_commands self.run_command(cmd) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() File "build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py", line 167, in run File "/home/antoine/cpython/__svn__/Lib/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 177, in run File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 252, in find_sources File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 306, in run File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 330, in add_defaults File "/home/antoine/cpython/__svn__/Lib/distutils/command/sdist.py", line 288, in add_defaults for pkg, src_dir, build_dir, filenames in build_py.data_files: File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 39, in __getattr__ File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 44, in _get_data_files File "build/bdist.linux-x86_64/egg/setuptools/command/build_py.py", line 92, in analyze_manifest File "/home/antoine/cpython/__svn__/Lib/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/antoine/cpython/__svn__/Lib/distutils/dist.py", line 948, in run_command cmd_obj.run() [...] File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 177, in run File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 252, in find_sources File "build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py", line 305, in run File "/home/antoine/cpython/__svn__/Lib/distutils/filelist.py", line 45, in findall self.allfiles = findall(dir) File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 71, in findall File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 294, in walk for x in walk(path, topdown, onerror, followlinks): File "/home/antoine/cpython/__svn__/Lib/os.py", line 284, in walk if isdir(join(top, name)): File "/home/antoine/cpython/__svn__/Lib/genericpath.py", line 41, in isdir st = os.stat(s) File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 87, in wrap RuntimeError: maximum recursion depth exceeded

2 5

Python svn builds
by A. Cavallo 01 Jun '09

01 Jun '09

Hi all, I've just published few python rpms for the trunk source. You can download and install the packages from (either by hand or using yum/yast): http://download.opensuse.org/repositories/home:/cavallo71:/opt-python- interpreters/ The interpreters will install under so it will leave the system python untouched: /opt/opt-python-svnXXXXX-2.7a0 XXXXX is the svn revision. Once installed you can use them just sourcing the file: $> . /opt/opt-python-svnXXXXX-2.7a0/opt-python-svnXXXXX-env.sh $> python Python 2.7a0 (trunk, May 24 2009, 21:18:39) [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2 Type "help", "copyright", "credits" or "license" for more information. There is a web page containing the build results (including a small simple smoke test): http://pyvm.sourceforge.net/ Module developer can use these build to test against the more up to date python source and anticipating bugs. Developer can use it to run regression/integration tests. I hope that will help, Regards, Antonio

2 2

Re: [Distutils] PEP 376 up-to-date
by P.J. Eby 31 May '09

31 May '09

At 12:29 PM 5/25/2009 +0200, Tarek Ziadé wrote: >Hello, > >I have updated PEP 376, mainly with Phillip's feedback, and changed >the API section so it looks like the current prototype. > >- PEP : http://svn.python.org/view/peps/trunk/pep-0376.txt?view=markup >- prototype code : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py > >I'll wait for a new round of feedbacks, Thanks ! * There's no reason for anything shown in the module to be private; is_egg_info() and egg_info_dirs() would be useful API functions, and the additions to DistributionMetadata might as well go in distutils * Better yet, move DistributionMetadata to pkgutil and have distutils.dist import it; that way you won't get tons of distutils.* imports any time you import pkgutil. * get_egg_infos() should take a pathlist argument, which if None can default to sys.path. * Project name normalization and case insensitive comparison is still not implemented * File path normalization (absolutizing, case-normalization, de-cross-platforming, etc.) is not implemented * There is extensive coupling both to sys.path and to the global cache; in particular, the owns() operation properly belongs to an object representing a *directory*, rather than an object representing an individual project. Note too that such "directory" objects would then also be an appropriate cache target, and a suitable home for egg_info_dirs() and get_file_users() operations. This also simplifies get_egg_infos, since it would simply either retrieves directories from the cache or creates them and optionally caches them, without also meddling in the details of cache contents. Also, an application that wishes to do so can simply create directory objects of its own and manage their lifecycle accordingly. Personally, I would also make an object for a collection of directories, but you could possibly get away without it. (At this point, I have only briefly skimmed the updated PEP; I'll take a closer look at it once the API/implementation gets more settled.)

2 2

[issue73] setup.py --name prints warning messages to stdout, rather than stderr
by Sridhar Ratnakumar 31 May '09

31 May '09

New submission from Sridhar Ratnakumar <sridharr(a)activestate.com>: Warning messages must be printed to stderr. This allows for a programattic way to find setup.py metadata. > pwd ../dap.plugins.sql-0.2.1 > apy setup.py --name 2> /dev/null 'dap.plugins' is declared as a package namespace, but 'dap' is not: please correct this in setup.py dap.plugins.sql > echo $? 0 > ---------- messages: 295 nosy: srid priority: bug status: unread title: setup.py --name prints warning messages to stdout, rather than stderr _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue73> _______________________________________________

3 2

[issue74] pkg_resources.ImpWrapper documented but no longer exists
by Nicholas Riley 31 May '09

31 May '09

New submission from Nicholas Riley <com-pythonbugs(a)sabi.net>: pkg_resources.ImpWrapper was (and is) mentioned in the documentation (pkg_resources.txt), and as a result has been used by modules such as modulegraph (http://svn.pythonmac.org/modulegraph/modulegraph/trunk/modulegraph/modulegr… ). However, recent versions of pkg_resources remove this definition. Given this class appears in the documentation and still exists, just not with that name, it would be useful if backwards compatibility could be retained. ---------- messages: 296 nosy: nriley priority: bug status: unread title: pkg_resources.ImpWrapper documented but no longer exists _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue74> _______________________________________________

1 0

Patch for clean command
by James 29 May '09

29 May '09

Hi, I'm certaintly new to distutils and setuptools, however I figured I'd send in this patch, and either it will get merged because it's a great idea or someone will perhaps tell me why this doesn't exist already. (or maybe it does and i can't find it) In any case, it adds the pyc option to the clean command so that the .pyc can be removed on request. Personally i'll have a [clean] pyc=1 option in my setup.cfg, but that's for my convenience. cheers, _J [attached: patch and modified file]

2 2

eggs and Python Unicode variants (UCS2, UCS4)
by Martijn Faassen 29 May '09

29 May '09

Hi there, In the lxml project (http://codespeak.net/lxml), we've just noticed the following problem with lxml eggs: you can easy_install an egg that won't work for your Python. This is because Python can be compiled with either 2 or 4 bytes unicode as its internal representation. Any egg that contains compiled C code that uses unicode such as lxml will run into trouble: if it's compiled with a 4 bytes unicode Python, it won't work on a 2 bytes unicode Python, and vice versa. This problem is fairly common in Linux. Many distributions such as Ubuntu and Fedora compile their python with 4 bytes unicode internal representation. If you compile a Python interpreter by hand it defaults to 2 bytes unicode, however. Hand-building a Python interpreter is done fairly commonly by Linux sysadmins for various reasons. It would therefore be very nice if it became possible to make eggs for the different unicode compilation options of Python. This configuration dimension is a real world issue for any binary Python module that does anything with unicode text.. In an earlier mail to this list: http://mail.python.org/pipermail/distutils-sig/2005-October/005222.html M.-A. Lemburg and Phillip Eby had the following discussion: [MAL] >>Please make sure that your eggs catch all possible >>Python binary build dimensions: >> >>* Python version >>* Python Unicode variant (UCS2, UCS4) >>* OS name >>* OS version >>* Platform architecture (e.g. 32-bit vs. 64-bit) [PJE] >As far as I know, all of this except the Unicode variant is captured in >distutils' get_platform(). And if it's not, it should be, since it >affects any other kind of bdist mechanism. I'm not sure whether this means this needs to be escalated from setuptools to the Python interpreter level itself. With this mail, I've done the job escalating this lxml problem to what appears to be the right place, though. :) Thanks, Martijn

5 5

Buildout: caching of base configuration ('extends')
by Thomas Lotze 28 May '09

28 May '09

Hi, I've been implementing a download API for buildout and have reached the point where I'd like to try using it for downloading the base config files referenced from a buildout config by the 'extends' option. This would have the benefit of allowing to cache those config files and being able to run a buildout using that feature while being offline. However, I'd like to discuss two points about doing this: - I think buildout should not unconditionally use a cached copy of a base configuration file. While it is sensible in other use cases to never access the network if a file is found in the cache, base configs can be expected to change and should IMHO be taken from the cache only when offline mode is active or an attempt at downloading failed. - The download cache to be used is configured as one of the options that are read from either ~/.buildout/default.cfg or buildout.cfg and their respective bases. In order to keep things simple, I'd suggest to use a download cache only if specified directly inside any of these two files, and ignore download-cache options in any downloaded base configs. This would be an exception to how options read from config files are combined, but one I deem worthwhile. I'd allow for configuring and using different download caches in those two files, though. That way, bases of the user's default config can be cached centrally while allowing to keep project-related stuff in a different place. Opinions? -- Thomas

3 7

[Buildout] Include-dirs
by Svenn Helge Grindhaug 28 May '09

28 May '09

Hi, I'm trying to do something similar to the custom egg building in the tutorial, http://www.buildout.org/docs/tutorial.html?highlight=location#custom-egg- building I have the following files: setup.py ********* from setuptools import setup, find_packages setup( name='myproject', version='3.3.0', packages=find_packages('src'), package_dir = {'': 'src'}, include_package_data = True, install_requires=['setuptools', 'numpy', 'ScientificPython'], zip_safe = False, ) buildout.cfg ************* [buildout] parts = numpy ScientificPython mypy [numpy] recipe = zc.recipe.egg find-links = http://sunet.dl.sourceforge.net/sourceforge/numpy/numpy-1.3.0.tar.gz eggs= numpy [ScientificPython] recipe = zc.recipe.egg:custom egg=ScientificPython find-links = http://sourcesup.cru.fr/frs/download.php/2372/ScientificPython-2.9.0.tar.gz include-dirs = ${numpy:location}/numpy/core/include [mypy] recipe = zc.recipe.egg:script eggs = numpy ScientificPython interpreter = python ------------ The ScientificPython package need some header files located in the numpy package/egg. I get an error when using ${numpy:location} Error: Referenced option does not exist: numpy location When I use the absolute path to the include dir it works. What am I doing wrong? Best regards, Svenn.

1 0