Mailman 3 April 2011 - Distutils-SIG

Packaging and the dependency on distribute/setuptools
by Kelsey Hightower April 10, 2011

April 10, 2011

Re-posting for visibility: http://groups.google.com/group/the-fellowship-of-the-packaging/browse_threa… While working on the new pysetup commands I ran into some issues when trying to uninstall projects based on distribute/setuptools. The current install method used in packaging is to "shell" out and call: # projects based on distutils python setup.py install --record=/path/to/record_file # projects based on setuptools python setup.py install --record=/path/to/record_file --single-version-externally-managed" This works and gets the project installed. However, this does not give us PEP 376 compatibility and adds a dependency on distribute/setuptools. Another side effect from the methods above, the RECORD file does not end up in the right place. I am sure we can fix this, but do we really want to? To get everything working it seems we will have to do a custom distribute/setuptools install similar to how pip does things. https://github.com/pypa/pip/blob/master/pip/req.py#L540 Note, we do not have the above issue with packaging (setup.cfg) based projects. At this point it seems we need to pick one of three options: 1. Only support packaging based projects (setup.cfg) 2. Only support packaging and distutils based projects, punt on projects that require distribute/setuptools. 3. Support all three, packaging, distutils, and distribute/setuptool based projects. Pros and Cons ============= Option 1 -------- Only support packaging based projects(setup.cfg). Pros * enforce the new standard, setup.cfg and static metadata. * clear project scope; one way forward * no dependencies on 3rd party modules (distribute) * chance to make a clean break; distutils is still in the stdlib and pip supports Python 3 Cons * unable to install most projects day one; will have to defer to 3rd party tools such as pip and distribute Option 2 -------- Only support installing packaging and distutils based projects and punt on projects that require distribute/setuptools. Pros * can install more projects than option 1. * no dependencies on 3rd party modules (distribute) * promote the new standard with a transition for distutils based projects Cons * still supports setup.py * causes confusion; can install some projects with a setup.py and not others * may prolong the move to setup.cfg and static metadata * miss our chance for a clean break Option 3 -------- Support all three, packaging, distutils, and distribute/setuptools based projects. Pros * can install more projects than option 1 and option 2 Cons * WILL prolong the move to setup.cfg and static metadata. Most people will continue to use what they know best. * miss our chance for a clean break I am leaning toward Option 1 myself, as option 2 will only cause confusion. Option 3 seems like going backward by requiring distribute and will prolong the move to setup.cfg and static metadata. Going with option 1 will require a lot of effort to get the community on board and start converting to a valid setup.cfg, or providing support for both, a valid setup.cfg and setup.py. With packaging and pysetup being new tools we should keep them clean and focused on the future. This is especially true when good alternatives exist to provide a transition strategy for most projects. Thoughts?

1 0

Packaging and the dependency on distribute/setuptools
by Kelsey Hightower April 10, 2011

April 10, 2011

Re-posting for visibility: http://groups.google.com/group/the-fellowship-of-the-packaging/browse_threa… While working on the new pysetup commands I ran into some issues when trying to uninstall projects based on distribute/setuptools. The current install method used in packaging is to "shell" out and call: # projects based on distutils python setup.py install --record=/path/to/record_file # projects based on setuptools python setup.py install --record=/path/to/record_file --single-version-externally-managed" This works and gets the project installed. However, this does not give us PEP 376 compatibility and adds a dependency on distribute/setuptools. Another side effect from the methods above, the RECORD file does not end up in the right place. I am sure we can fix this, but do we really want to? To get everything working it seems we will have to do a custom distribute/setuptools install similar to how pip does things. https://github.com/pypa/pip/blob/master/pip/req.py#L540 Note, we do not have the above issue with packaging (setup.cfg) based projects. At this point it seems we need to pick one of three options: 1. Only support packaging based projects (setup.cfg) 2. Only support packaging and distutils based projects, punt on projects that require distribute/setuptools. 3. Support all three, packaging, distutils, and distribute/setuptool based projects. Pros and Cons ============= Option 1 -------- Only support packaging based projects(setup.cfg). Pros * enforce the new standard, setup.cfg and static metadata. * clear project scope; one way forward * no dependencies on 3rd party modules (distribute) * chance to make a clean break; distutils is still in the stdlib and pip supports Python 3 Cons * unable to install most projects day one; will have to defer to 3rd party tools such as pip and distribute Option 2 -------- Only support installing packaging and distutils based projects and punt on projects that require distribute/setuptools. Pros * can install more projects than option 1. * no dependencies on 3rd party modules (distribute) * promote the new standard with a transition for distutils based projects Cons * still supports setup.py * causes confusion; can install some projects with a setup.py and not others * may prolong the move to setup.cfg and static metadata * miss our chance for a clean break Option 3 -------- Support all three, packaging, distutils, and distribute/setuptools based projects. Pros * can install more projects than option 1 and option 2 Cons * WILL prolong the move to setup.cfg and static metadata. Most people will continue to use what they know best. * miss our chance for a clean break I am leaning toward Option 1 myself, as option 2 will only cause confusion. Option 3 seems like going backward by requiring distribute and will prolong the move to setup.cfg and static metadata. Going with option 1 will require a lot of effort to get the community on board and start converting to a valid setup.cfg, or providing support for both, a valid setup.cfg and setup.py. With packaging and pysetup being new tools we should keep them clean and focused on the future. This is especially true when good alternatives exist to provide a transition strategy for most projects. Thoughts?

1 0

An internal error occurred due to a bug in either zc.buildout or in a recipe being used
by Herson April 9, 2011

April 9, 2011

Hi, Well guys, I'm having trouble running buildout on Mac OSX leopard. My environment consists of: buildout.cfg: [buildout] parts = python develop = . eggs = teste [python] recipe = zc.recipe.egg interpreter = python eggs = ${buildout:eggs} I run the: $ python bootstrap.py Downloading http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/bin'. Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/parts'. Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/eggs'. Creating directory '/Users/hersonls/Code/Python/zc.Buildout/teste/develop-eggs'. Generated script '/Users/hersonls/Code/Python/zc.Buildout/teste/bin/buildout'. And run the ./bin/buildout $ ./bin/buildout Develop: '/Users/hersonls/Code/Python/zc.Buildout/teste/.' While: Installing. Processing develop directory '/Users/hersonls/Code/Python/zc.Buildout/teste/.'. An internal error occurred due to a bug in either zc.buildout or in a recipe being used: Traceback (most recent call last): File "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py", line 1805, in main File "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py", line 446, in install File "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py", line 686, in _develop File "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py", line 1191, in develop File "/private/var/folders/Jp/Jp9P8zk3E9yRgo+ZL4wnx++++TI/-Tmp-/tmpqdrwgm/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py", line 1124, in _copyeggs AssertionError: [] Initially I had no issues with buildout, this problem appeared suddenly in a new project. Can someone give me a light that might be? === Notes === Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin -- Herson Leite, Desenvolvedor Web www.hersonls.com.br

2 1

Trouble with bdist_rpm and data_files
by Lucas Kannebley Tavares April 8, 2011

April 8, 2011

Hello all, I wrote a setup.py file and managed to get debhelper to create a working deb package out of it. But I am having trouble with creating a binary rpm package. Apparently all data files are not found by rpmbuild. running install_data error: can't copy 'etc/proj.conf': doesn't exist or not a regular file error: Bad exit status from /var/tmp/rpm-tmp.drqkKR (%install) (I changed the file name for proj.conf in the e-mail as to not divoulge any corporate information) But that file does indeed exist. It is at etc/file.conf in the repo, and it should be moved to /etc/proj/cli/proj.conf The BUILDROOT/etc/proj/cli directory is properly created. There are other data files listed, but none of them are copied properly, even if I change their order. Any help on this would be appreciated. Thank you. -- Lucas Kannebley Tavares Infrastructure Team IBM Linux Technology Center

1 1

passing command-line options to an additional python interpreter?
by Mark Visser April 5, 2011

April 5, 2011

I'm using buildout for managing a project that runs on several different python interpreters, some of which are embedded in other applications. As such, I'm at the whim of those packaging python for those applications, and sometimes they don't set things up... well, optimally. The problem I've run into is that one of the exposed non-standard python interpreters (mayapy) is actually a shell script. Thus, you can't use its direct path with a shebang. Normally, I get around this by using #!/usr/bin/env mayapy. For the life of me, I can't figure out a way to pass this to buildout. I've tried this: [mayapy] executable = /usr/bin/env mayapy [mayatests] python = mayapy ... etc But buildout doesn't like the space. Is there a way to specify executable options? I tried this to no avail: [mayapy] executable = /usr/bin/env options = mayapy Can anyone suggest a solution? thanks, -Mark

2 1

window 64bit madness
by Adam GROSZER April 5, 2011

April 5, 2011

Hello, Having problems here installing http://pypi.python.org/packages/2.6/r/reportlab/reportlab-2.5.win-amd64-py2… . Neither setuptools 0.6c11 nor distribute 0.6.14 gets it right. We don't have and don't want to have a compiler on the server, so we need rely on binary packages. - setuptools just fails miserably even with easy_install: http://paste.lisp.org/+2KZD - distribute installs it (tho in a ...-win32.egg folder): http://paste.lisp.org/+2KZE - zc.buildout cannot install it with distribute, because it wants to use the source tarball (and we don't have a compiler) or when I make it not to be able to see the tarball, it does not find the reportlab-2.5.win-amd64-py2.6.exe. I guess it does not get the platform right. Any hints? -- Best regards, Adam GROSZER -- Quote of the day: Age before beauty and pearls before swine. -- Dorothy Parker

5 8

Stop using setuptools or distribute in buildout (was Re: Fwd: Re: window 64bit madness)
by Jim Fulton April 4, 2011

April 4, 2011

2011/3/19 Adam GROSZER <agroszer(a)gmail.com>: > Hello, > > As Jim wants to dump setuptools support from buildout, ... I don't want to, and won't until it gets too painful to support both. I just suspect that time is drawing near. I can also foresee a time at which buildout stops using either. That might be the best path for buildout....wow, the more I think of that, the more appealing it seems. Replacing use of setuptools/distribute could be a really nice project for a certain upcoming German buildout sprint. :) Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton

3 3

setuptools download error?
by Reinout van Rees April 4, 2011

April 4, 2011

Hi, My setuptools-using colleagues have problems with downloading setuptools. Initially I suspected the 0.6c12-r12345something releases, that old release numbers were disappearing. But that's apparently not the case, even though having such version numbers downloaded by a standard buildout bootstrap worries me a bit. I tried using "wget" to download one such release and it also failed. It would download 95% of the file and then stall... Pinning setuptools to 06c11 (the latest stable) worked like a charm: that one downloaded just fine (or was already available in the cache, probably). Is there something wrong with the peak server? Should I fix something on our end? (I'm using distribute myself, so I don't have this problem). Reinout

2 1

Re: [Distutils] reservations about pythonv
by Carl Meyer April 3, 2011

April 3, 2011

This alternative approach (with a symlink and config file) sounds pretty good to me as well after our discussion at the sprints. The downsides I see: 1. Cross-platform inconsistency. Windows virtualenvs would be isolated from system python binary upgrades, *nix would not (some, like Brandon, consider this isolation a feature, so maybe *nix should allow that option too?) On the whole this doesn't seem like a big issue. 2. There would be no way to set LD_LIBRARY_PATH to include the virtualenv prior to execution of python. To be honest, I don't understand why this is needed, as current virtualenv doesn't do it and I haven't seen it break anything (compiled modules work fine in a venv). But people with more experience seemed to think it was needed at our open space discussion last Saturday. Perhaps someone can clarify what specifically breaks without this? I'll dive into this approach and see what I can achieve. I'd also appreciate feedback from Tarek or others familiar with the new sysconfig module about my changes there. Something along those lines will I think be needed with either approach, but I'm almost wondering if a new "virtual" install scheme would be a better approach? Carl Sent from my Android phone ----- Reply message ----- From: "Vinay Sajip" <vinay_sajip(a)yahoo.co.uk> Date: Wed, Mar 16, 2011 6:13 pm Subject: [Distutils] reservations about pythonv To: <Distutils-Sig(a)Python.Org> Barry Warsaw <barry <at> python.org> writes: > Here at the sprints we talked about several possible options, the details of > which of course will have to be hashed out. There will also be cross platform > considerations. I think on *nix at least, it would be nice if a symlink and > configuration file were enough to trigger the virtualenv behavior. I feel that this approach definitely has potential to be the most useful and practical. There's no reason for a separate executable if Python itself incorporates the virtual environment functionality, which is mostly about setting paths and environment variables. > For example, if I have a local 'python' symlinked to the real executable, with > a pythonv.conf file next to it, the virtual environment would be enabled. The > real Python binary would adjust its behavior in that case to know where the > standard library was, but also use the locally installed packages. Anyway, > that's how I'd *like* it to work on *nix, but it may have to work differently > on Windows, and it may have to work differently if environment variables have > to be set. Even if on Windows you have to copy rather than symlink, that could just be the main Python executable, which is not prohibitively large since the core of Python is in pythonX.Y.dll. Regards, Vinay Sajip _______________________________________________ Distutils-SIG maillist - Distutils-SIG(a)python.org http://mail.python.org/mailman/listinfo/distutils-sig

15 55

[issue125] macosx unittest failure for /tmp
by Kurt Schwehr April 1, 2011

April 1, 2011

New submission from Kurt Schwehr <kurtschwehr(a)yahoo.com>: On the Mac, /tmp is a symbolic link to /private/tmp. We are stuck Apple's decision to do this. Hit this when trying adding code to the Mac OSX Fink distribute package. -kurt (goatbar) FAIL: test_local_index (setuptools.tests.test_easy_install.TestUserInstallTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/sw32/src/fink.build/distribute-py27-0.6.15-1/distribute-0.6.15/setuptools/tests/test_easy_install.py", line 234, in test_local_index self.assertEquals(res.location, new_location) AssertionError: '/private/tmp/tmpCjeemJ' != '/tmp/tmpCjeemJ' ---------------------------------------------------------------------- Ran 71 tests in 10.861s FAILED (failures=1) ---------- messages: 610 nosy: schwehr priority: bug status: unread title: macosx unittest failure for /tmp _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue125> _______________________________________________

1 0