Mailman 3 October 2009 - Distutils-SIG

Re: [Distutils] supporting multiple versions of one package in one environment is insane
by Chris Withers 20 Oct '09

20 Oct '09

[moving to disutils-sig where this belongs] Toshio Kuratomi wrote: >>> where type of problem == sandboxed environment, sure. How do you solve the >>> problem for system packagers? >> What's to stop a system packager either just running the buildout on >> install, or running the buildout at package build time and then just >> dropping the resulting environment wherever they want to install >> applications? Such a package would only be dependent on the right python >> version at runtime... >> > If buildout creates sandboxed environments like virtualenv then everything > here applies: > > https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries That's fine, buildout can use a shared eggs directory (it's how I run it by default, to minimise the amount of downloading that goes on). In that case, there would be no duplication of packages, the "installed app" would just pick from a folder full of eggs that you could put wherever your OS wants them... In short, have multiple versions of a python package on the python path at the same time is insane ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

1 0

Package install failures in 2.6.3 - setuptools vs Distribute
by Ned Deily 20 Oct '09

20 Oct '09

I'm afraid there is going to be a small deluge of very confused users who will end up needing to install Distribute but only when they eventually figure out why some packages with C extensions mysteriously no longer install after they upgrade to python 2.6.3. For example, following the package instructions to use setuptools easy_install, an easy_install lxml fails with 2.6.3 with the very cryptic: File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py", line 85, in get_ext_filename KeyError: 'etree' http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3 -osx-10-6-snow-leopard Appscript is another package that fails similarly. There must be others. I don't know what's to be done about this awkward incompatibility now that 2.6.3 is out the door other than perhaps making sure maintainers of affected packages are aware of what their users are running into and can update their documentation. There is one small thing that may help some users who find their way to http://pypi.python.org/pypi/distribute. OS X, for one, does not ship with wget however it does have curl. Suggest adding another example: $ curl -O http://nightly.ziade.org/distribute_setup.py $ python distribute_setup.py -- Ned Deily, nad(a)acm.org

21 88

[RFC] Recentering the static metadata need : PKG-INFO
by Tarek Ziadé 19 Oct '09

19 Oct '09

Hey, Marc-André Lemburg gave me a key points about the static metadata discussions we have (wether its PEP 390 or around it) He said that the important thing was to have the context-dependant markers in PKG-INFO, and that having in described in setup.cfg or in setup.py by any way is not the important thing. And he is right ! So, PEP 390 proposes a new setup.cfg format, that's fine. And others are thinking about other ways to build metadata, that's great. But what really matters at the end is to provide a new Metadata format where these marker are present, so any consumer can play with them (and Distutils provide an API to play with them : the DistributionMetadata class) So please, have a look at the new section I've added here with the syntax I am proposing : http://www.python.org/dev/peps/pep-0390/#impact-on-pkg-info-generation-and-… That will probably re-center the debate in creating a Metadata 1.2 format that includes these markers. As someone asked, here is the principal uses cases of having static metadata that includes markers like this: - pip, or easy_install, or any client code, will be able to query PyPI, to get the metadata of a package. *and* will be able to have it depending on a particular environment. - it will be possible to generate and provide alongside the source code the PKG-INFO file, like how install_egg_info does at installation time, exactly like how MANIFEST is built. If that sounds like the right path (it does to me), we could start to work on a PEP 314 update, and add in the work what was started last year: - revision of "requires", "obsoletes", etc.. - link to PEP 386 Regards Tarek

6 18

buglet in setuptools 0.6c10
by Barry Warsaw 19 Oct '09

19 Oct '09

Sorry for the lame-ass bug report, but my stack is really deep and I didn't want this one to go unreported. I just tried 0.6c10 to see if it would fix a problem I'm having. How fortunate that it hit Cheeseshop at just the right time ;). Unfortunately, there's a bug in setuptools/command/sdist.py Line 184 references string.join, but 'string' is undefined. -Barry

3 4

Avoid dependency_links / find_links
by Patrice Neff 19 Oct '09

19 Oct '09

Hi, We manage our projects with setuptools. For this we have a local PyPi mirror with just our required packages. This mirror is specified in our setuptools environments using the index_url directive. The basic idea is: we don't want to access the internet when installing packages. Today I had to realized that we haven't been reaching that goal as Pylons contains a dependency_links pointing to pylonshq.com. So the question would be: is there any way to avoid following any dependency_links and/or find_links paths and only use the index_url? I have quickly checked the setuptools source code and couldn't find any way to do this. Would a patch be accepted if I implemented that behavior? Are there other approaches to do what I'm trying to do? Thanks in advance Patrice

2 2

Setuptools 0.6c10 released
by P.J. Eby 19 Oct '09

19 Oct '09

The long-overdue setuptools 0.6c10 update is now available on PyPI, at: http://pypi.python.org/setuptools/ Major updates and fixes include: * Support for SVN 1.6 and Python 2.6 * Fix for the Python 2.6.3 build_ext API change * Support for the most recent Sourceforge download link insanity * Fix for Vista UAC errors running easy_install.exe or other "installer-looking" executables * Fix for errors launching 64-bit Windows Python * Stop crashing on certain types of HTTP error * Stop re-trying URLs that already failed retrieval once * Fixes for various dependency management problems such as looping builds, re-downloading packages already present on sys.path (but not in a registered "site" directory), and randomly preferring local -f packages over local installed packages * Prevent lots of spurious "already imported from another path" warnings (e.g. when pkg_resources is imported late) * Ensure C libraries (as opposed to extensions) are also built when doing bdist_egg Other changes: * Misc. documentation fixes * Improved Jython support * Fewer warnings under Python 2.6+ * Warn when 'packages' uses paths instead of package names (because it causes other problems, like spurious "already imported" warnings) * Stop using /usr/bin/sw_vers on Mac OS (replaced w/'platform' module calls) You can install the updated version using easy_install (or pip!), asking for setuptools==0.6c10. (Note for users of Distribute: Distribute and setuptools use the same package name 'setuptools', and thus cannot both be present on the same sys.path (e.g. in the same virtualenv). If you wish to switch a given environment from Distribute to setuptools or vice versa, you will need to completely uninstall one before installing the other. If you currently have Distribute installed, please follow Distribute's uninstall instructions if you wish to reinstall setuptools.) Please report any problems with setuptools to the setuptools bug tracker at: http://bugs.python.org/setuptools/ For faster response to questions, please use the distutils-sig mailing list, rather than the tracker. Setuptools documentation can be found via links at http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall -- and the pages on the PEAK wiki now load much, much faster than they did a few months ago. (They're static cached pages now, rather than dynamically generated, unless you're actually logged into the wiki.)

2 2

[buildout] Fixed verbose mode when developing eggs
by Jonathan Ballet 19 Oct '09

19 Oct '09

Hello, I had a small headache using Buildout this morning: I had a package which contained a C extension and which was developed in my buildout.cfg file. I wanted to see what was the gcc call used to build this extension, but using as much as -v as I could (I stopped at ten), I didn't see anything more. So, I dig in the code, and there was a typo in easy_install.py, which is fixed in fix-verbose-develop.patch (this needs a test to be updated, BTW). The fix was trivial (replacing "==" with "="), but I think the way args is build is now better. I propose also another patch (uniformize-args.patch), which refactors args construction using a list instead of a tuple, which looks smarter (why a tuple whereas it is updated several times?). This is not mandatory for the fix, feel free to integrate it or not. Cheers, Jonathan

2 1

Brand new, clean server, set up from scratch, buildbot still fails
by Steve Steiner 19 Oct '09

19 Oct '09

http://pastebin.com/m69796bc5 This is running on a bare system, after figuring out which development tools were needed, all the version control systems necessary etc. I have all that captured in my Fabric script so we're all set for being able to duplicate it. I have NOT installed anything into the system Python at all. In fact, other than the tools necessary to pull the products out of VCS, and the compilers and such, I haven't installed (or done) anything to the system at all. That's kind of the point. We want to be able to go from bare metal to running buildbot completely automatically. I don't quite understand what this thing is supposed to be doing but I'm sure it's not doing everything it needs to do to get itself "bootstrapped"; on a fresh Ubuntu install, anyway. BTW, on my OS X development system, running the system Python 2.6.1, it doesn't have any trouble at all except for this little gem early in the process. /usr/bin/python "-c" "from setuptools.command.easy_install import main; main()" "-mUNxd" "/Users/ssteiner/Dropbox/work/distutils-builbot/ eggs/tmptMBdZk" "-q" "/Users/ssteiner/Dropbox/src/twisted_trunk" path=/Users/ssteiner/Dropbox/work/distutils-builbot/eggs/ setuptools-0.6c9-py2.6.egg /Library/Python/2.6/site-packages/Pyrex-0.9.8.5-py2.6.egg/Pyrex/ Compiler/Errors.py:17: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 self.message = message unrecognized .svn/entries format in conftest.c:1:23: error: sys/epoll.h: No such file or directory conftest.c:1:23: error: sys/epoll.h: No such file or directory conftest.c:1:23: error: sys/epoll.h: No such file or directory lipo: can't open input file: /var/folders/28/28Xggq0cGXqLpT1Uz5Gk4E++ +TI/-Tmp-//ccOmCair.out (No such file or directory) Got Twisted 8.2.0-r27376. Picked: Twisted = 8.2.0-r27376 Getting required 'zope.interface' required by collective.buildbot 0.3.5. required by Twisted 8.2.0-r27376. ... and then it carries on. However, when trying to run it with /bin/test, something tries to refer to "trunk-checkout" when the python trunk was actually checked out into python-trunk. (~/work/distutils-builbot)# bin/test Traceback (most recent call last): File "bin/test", line 12, in <module> os.chdir(python_trunk) OSError: [Errno 2] No such file or directory: '/Users/ssteiner/work/ distutils-builbot/parts/trunk-checkout/python' All in all, I really can't get it to build or run anywhere at this point... S

3 9

Checking mailing list for Distribute
by Tarek Ziadé 18 Oct '09

18 Oct '09

Hey, if you want to follow the commits diffs and the issues for Distribute, there's now a distinct mailing list you can follow : http://groups.google.fr/group/distribute-checkins Regards Tarek

1 0

Buildbot throwing a gear after default setup?
by ssteinerX＠gmail.com 18 Oct '09

18 Oct '09

I've just downloaded and configured the Distribute buildbot and it appears that a dependency is missing somewhere. I get this (just the errors excerpted). Just wanted to capture this before I figure out what needs to be done... test_make_tarball (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'requires zlib' test_make_zipfile (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' test_tarfile_root_owner (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' test_tarfile_vs_tar (distutils.tests.test_archive_util.ArchiveUtilTestCase) ... skipped 'Requires zlib' ====================================================================== ERROR: test_make_archive_owner_group (distutils.tests.test_archive_util.ArchiveUtilTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/tests/test_archive_util.py", line 224, in test_make_archive_owner_group group=group) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/archive_util.py", line 236, in make_archive filename = func(base_name, base_dir, **kwargs) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/distutils/archive_util.py", line 163, in make_zipfile compression=zipfile.ZIP_DEFLATED) File "/home/ssteiner/distutils-builbot/parts/trunk-checkout/python/ Lib/zipfile.py", line 675, in __init__ "Compression requires the (missing) zlib module" RuntimeError: Compression requires the (missing) zlib module ---------------------------------------------------------------------- Ran 158 tests in 1.242s FAILED (errors=1, skipped=11) test test_distutils crashed -- <type 'exceptions.OSError'>: [Errno 2] No such file or directory Traceback (most recent call last): File "Lib/test/regrtest.py", line 767, in runtest_inner File "Lib/test/regrtest.py", line 723, in __exit__ File "Lib/test/regrtest.py", line 686, in get_cwd OSError: [Errno 2] No such file or directory 1 test failed: test_distutils

5 27