Mailman 3 December 2012 - Distutils-SIG

Override default build dir from setup.py
by anatoly techtonik 19 Dec '12

19 Dec '12

Hi, I disliked that setup.py creates its own temp directories without any private prefix, so I wanted to override this option in setup.py: $ setup.py build -h ... Options for 'build' command: --build-base (-b) base directory for build library And the only way I could find to do this is through setup.cfg: http://docs.python.org/2/distutils/configfile.html Is it really the case or I overlooked some essential documentation part? Please, CC. -- anatoly t.

2 2

buildout in offline mode - install receipts also from cache?
by Anton Koval' 17 Dec '12

17 Dec '12

hello, in our project we've decided to completely move buildout process to offline mode. according to documentation (http://pypi.python.org/pypi/zc.buildout/2.0.0a5) I made updates in our main buildout configuration file, like: download-cache = ... install-from-cache = true offline = true But when buildout tries to install any part with recipe specified, it is (obviously) failing, e.g.: Error: Couldn't find a distribution for 'iw.recipe.cmd'. So, my question is: are there ways to fetch recipes also from some cache (or whatever) on local disk? Thank you.

1 0

Is is worth disentangling distutils?
by Antonio Cavallo 16 Dec '12

16 Dec '12

Hi, I wonder if is it worth/if there is any interest in trying to "clean" up distutils: nothing in terms to add new features, just a *major* cleanup retaining the exact same interface. I'm not planning anything like *adding features* or rewriting rpm/rpmbuild here, simply cleaning up that un-holy code mess. Yes it served well, don't get me wrong, and I think it did work much better than anything it was meant to replace it. I'm not into the py3 at all so I wonder how possibly it could fit/collide into the big plan. Or I'll be wasting my time? Thanks

5 8

zc.buildout 2.0.0a4 released
by Jim Fulton 15 Dec '12

15 Dec '12

zc.buildout 2.0.0a4 released zc.buildout 2.0.0a4 (and zc.recipe.egg 2.0.0a3) has been released. This release is based on a new buildout2 effort that was started a few months ago and drops some complexity from buildout 1, including: - Attempts to be isolated from site-packages - Support for multiple Python's - Support for zipped eggs. This release support Python 2.6, 2.7, 3.2 and 3.3. To experiment with it, get the bootstrap script: curl https://raw.github.com/buildout/buildout/master/bootstrap/bootstrap.py > bootstrap.py Since this is still an alpha release, you'll need to supply the -y option to bootstrap.py:: python bootstrap.py -t Expect fairly rapid iteration on this in the coming weeks. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm

7 16

buildout and scikit-learn dependency issues
by Eddie Bell 14 Dec '12

14 Dec '12

Hi, We are having some issues with scikit-learn in our buildout process. It appears that scikit-learn imports numpy in its setup.py. We have numpy as a dependency but the scikit-learn installation only succeeds if numpy has been installed externally (i.e. pip install numpy). Is there a way to solve this issue within buildout? thanks - e p.s. I am very new to buildout!

1 0

easy_install
by Yoshio Arai 12 Dec '12

12 Dec '12

Hello I installed Python 2.7 and installed easy_install. It did work. But, to install Brian which is neuroscience package, Brian did not come with Scipy, because Scipy works with Python 2.6. So I installed Python 2.6 and tried to install easy_install 2.6 but, can not install easy_install for 2.6, I believe already 2.7 version easy_install is occupying a spot. Please let me know how to replace 2.7 easy_instal to 2.6. Yoshio Arai

2 2

separate distribution of arbitrary submodules?
by Václav Šmilauer 08 Dec '12

08 Dec '12

Hello, I am using setuptools to package module A. I would like to separately package also modules A.* such that those submodules are discoverable (using pkgutil.iter_modules preferrably) from within A, provided their names are unknown to A. Can this be done with some clever __path__ manipulation perhaps? I tried installing A.B with its own setup.py, with setup(...,packages=['A.B']), it is installed in A.B-....egg, but not found when I want to import A.B (let alone with iter_modules). Background of this question: a common code A is distributed to customers, which in addition use their specific modules (A.A, A.B, A.C, ...); the code in A should, however, show them which modules are available - for user-friendliness. Cheers, Vaclav

2 1

Re: [Distutils] [Python-Dev] Accept just PEP-0426
by Daniel Holth 04 Dec '12

04 Dec '12

On Tue, Nov 20, 2012 at 11:01 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Wed, Nov 21, 2012 at 1:20 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > >> On Wed, Nov 21, 2012 at 1:10 PM, PJ Eby <pje(a)telecommunity.com> wrote: >> >>> Conversely, if you have already installed a package that says it >>> "Obsoletes" another package, this does *not* tell you that the obsolete >>> package shouldn't still be installed! A replacement project doesn't >>> necessarily share the same API, and may exist in a different package >>> namespace altogether. >>> >> >> Then that's a bug in the metadata of the project misusing "Obsoletes", >> and should be reported as such. If the new package is not a drop-in >> replacement, then it has no business claiming to obsolete the other package. >> >> I think one of the big reasons this kind of use is rare in the Python >> community is that project name changes are almost always accompanied by >> *package* name changes, and as soon as you change the package name, you're >> changing the public API, and thus it is no longer appropriate to use >> Provides or Obsoletes, as the renamed project is no longer a drop-in >> replacement for the original. >> > > I realised that my comments above are more about the appropriate use of > "Provides", rather than "Obsoletes". For a practically useful "Obsoletes", > I think I'm inclined to agree with you, as "Obsoleted-By" provides a way > for a maintainer to explicitly declare that a project is no longer > receiving updates, and users should migrate to the replacement project if > they want to continue to receive fixes and improvements. The current > version of "Obsoletes" is, as Daniel describes, really only useful as > documentation. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia > A few more changes to try to address some of the confusion about Requires-Dist: without re-designing the entire requirements system. PEP-426 was written only to add extras support to the format. The other changes, re-writing much of the PEP, have been an unfortunate side-effect. The file format's keys are case-insensitive. The version number should be in PEP 386 form. There are too many non-PEP-386 versions now and in the future to make it a must. Distribution (requirement) names are noted as being distinct from ``import x`` module names. Parenthetical explanation has balanced parens. "bundled" has been struck from the PEP. diff -r 55c706023fa2 -r 026aebf2265d pep-0426.txt --- a/pep-0426.txt Sun Nov 18 19:55:10 2012 +0200 +++ b/pep-0426.txt Mon Dec 03 20:36:13 2012 -0500 @@ -34,9 +34,9 @@ The syntax defined in this PEP is for use with Python distribution metadata files. The file format is a simple UTF-8 encoded Key: value -format with no maximum line length, followed by a blank line and an -arbitrary payload. The keys are case-insensitive. It is parseable by -the ``email`` module with an appropriate ``email.policy.Policy()``. +format with case-insensitive keys and no maximum line length, followed by +a blank line and an arbitrary payload. It is parseable by the ``email`` +module with an appropriate ``email.policy.Policy()``. When ``metadata`` is a Unicode string, ```email.parser.Parser().parsestr(metadata)`` is a serviceable parser. @@ -94,7 +94,7 @@ ::::::: A string containing the distribution's version number. This -field must be in the format specified in PEP 386. +field should be in the format specified in PEP 386. Example:: @@ -283,12 +283,13 @@ Each entry contains a string naming some other distutils project required by this distribution. -The format of a requirement string is identical to that of a -distutils project name (e.g., as found in the ``Name:`` field. -optionally followed by a version declaration within parentheses. +The format of a requirement string is identical to that of a distribution +name (e.g., as found in the ``Name:`` field) optionally followed by a +version declaration within parentheses. -The distutils project names should correspond to names as found -on the `Python Package Index`_. +The distribution names should correspond to names as found on the `Python +Package Index`_; often the same as, but distinct from, the module names +as accessed with ``import x``. Version declarations must follow the rules described in `Version Specifiers`_ @@ -305,7 +306,8 @@ Like Requires-Dist, but names dependencies needed while the distributions's distutils / packaging `setup.py` / `setup.cfg` is run. -Commonly used to generate a manifest from version control. +Commonly used to bring in extra compiler support or a package needed +to generate a manifest from version control. Examples:: @@ -318,17 +320,19 @@ Provides-Dist (multiple use) :::::::::::::::::::::::::::: -Each entry contains a string naming a Distutils project which -is contained within this distribution. This field *must* include -the project identified in the ``Name`` field, followed by the -version : Name (Version). +Each entry contains a string naming a requirement that is satisfied by +installing this distribution. This field *must* include the project +identified in the ``Name`` field, optionally followed by the version: +Name (Version). A distribution may provide additional names, e.g. to indicate that -multiple projects have been bundled together. For instance, source -distributions of the ``ZODB`` project have historically included -the ``transaction`` project, which is now available as a separate -distribution. Installing such a source distribution satisfies -requirements for both ``ZODB`` and ``transaction``. +multiple projects have been merged into and replaced by a single +distribution or to indicate that this project is a substitute for another. +For instance distribute (a fork of setuptools) could ``Provides-Dist`` +setuptools to prevent the conflicting package from being downloaded and +installed when distribute is already installed. A distribution that has +been merged with another might ``Provides-Dist`` the obsolete name(s) +to satisfy any projects that require the obsolete distribution's name. A distribution may also provide a "virtual" project name, which does not correspond to any separately-distributed project: such a name @@ -359,10 +363,9 @@ Version declarations can be supplied. Version numbers must be in the format specified in `Version Specifiers`_. -The most common use of this field will be in case a project name -changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0. -When you install Torqued Python, the Gorgon distribution should be -removed. +The most common use of this field will be in case a project name changes, +e.g. Gorgon 2.3 gets renamed to Torqued Python 1.0. When you install +Torqued Python, the Gorgon distribution should be removed. Examples::

1 0

Packager/Installer for Jython Deps?
by Alexander Dorsk 03 Dec '12

03 Dec '12

Hi Packaging Folks, I'm looking for some tips on developing a packaging/install system for Jython distributions. Ideally I would like a create a system that would manage not only python dependencies, but also java dependencies. Perhaps a quick example will help to illustrate: = Begin Example A = Say I'm working on a project, and I want to install a package, 'my_jython_h2_stuff' that has these dependencies: - the sqlalchemy python package - the H2 Database .jar In my dream world I would do something like this: $ jython virtualenv.py jython_env; $ source jython_env/bin/activate; $ jython_env/bin/some_jython_package_installer install my_jython_h2_stuff The end result of the above commands would be: - The pure python components of my_jython_h2_stuff are in jython_env/Lib/*/site_packages/my_jython_h2_stuff - sqlalchemy is in jython_env/Lib/*/site_packages/sqalchemy - The H2.jar file is in some place where jython can find it, e.g. jython_env/javalibs/h2.jar Then I could do something like this: $ jython -c "import my_jython_h2_stuff" Furthermore, the 'my_jython_h2_stuff' dependencies could be described in a distutils2-style setup.cfg w/ sections like: --- [requires-dist] sqlalchemy #A. Define single jars, through some uri/vcs scheme [requires-jars] h2.jar http://h2.org/download/h2.jar #B. Define individual jars, via maven-style IDs [requires-jars-maven] org.h2:h2.jdbc.jar:1.14 #C. Define requirements in a pom.xml file # Defines path to .pom file, relative to setup.cfg file. [requires-pom] pom.xml --- At this point, since we're still in my dream world, Jennifer Connelly would show up in a boat with umbrella drinks and a fresh rhubarb pie. But I digress. = End Example A = Hopefully that example helps illustrate what I mean. Currently it's somewhat tricky to manage both .jar and python dependencies for Jython projects, due to Jython's mixed heritage. You might say that Jython is the Obama of programming; a hybrid that grapples with its dual heritage and yet draws upon it for strength. It's complicated sometimes. However, with recent developments such as the distutils2 style of keeping metadata outside setup.py, and tools like sunng87's jip < https://github.com/sunng87/jip >, I think there may be ways to address the difficulty of managing dependencies for jython distributions. One way might be to create an installer, or an extension to an existing installer that handles special jython-specific metadata, and this leads me to some questions: Q1: Is there an existing installer that could handle special cases of metadata? Q2: If 'no' to Q1, would it be better to (A) create a new installer from scratch, or (B) is there a way to extend an existing installer, such as pip? Q3: If (B) for Q2, is there documentation on how how to do this? I didn't see anything the pip documents about extending PIP to handle special metadata. I'd love to hear any pointers. There is possibly great value to be gained if a Jython distribution installer could be developed. For example, it could be possible to easily develop and distribute python apps via self-contained stand-alone .jars . Imagine something like this: 1. create a jython_installer requirements file (just like a pip requirements file) 2. $ jython create_jython_virtualenv -r requirements.txt my_app_virtualenv 3. (do custom app development in the virtual env, and define an application entry point) 4. jython virtualenv_to_jar my_app_virtualenv > my_standalone_app.jar 5. mail my_standalone_app.jar to some windows user. 6. user double-clicks my_standalone_app.jar, and- *it just runs*, no installers or anything. It would also be possible to include jython ports in the same source bundles as the python originals. For example, a numpy port that uses java libs instead of c extensions. The .py source would check for the platform and import jython or cython modules appropriately, and requisite .jars would be described in the setup.cfg file. For normal cython installs via pip or the such, the .jar metadata would be ignored. For jython installs via a jython_installer, the .jars would be fetched. And the naming possibilities for such a packager are endless. "Samuel L. Jackson's 'Snakes-in-a-jar', or perhaps "PeanutCan", like the old Snake-in-a-can prank... < http://www.youtube.com/watch?v=GADUPKy4jos >. Anyway, if anyone has any tips on the above questions, I'm would love to hear 'em. -Alex

1 0

zc.buildout 2.0.0a5 released
by Jim Fulton 02 Dec '12

02 Dec '12

The release is available from github: https://github.com/downloads/buildout/buildout/zc.buildout-2.0.0a5.tar.gz To bootstrap this release, use the bootstrap.py file at: https://github.com/downloads/buildout/buildout/bootstrap.py Changes from the previous release: - Buildout will no-longer install non-final releases if there are final releases that meet requirements. (IOW, the prefer-final buildout option now defaults to true. You can get the old behavior by setting prefer-final to false.) - You can now specify simple constrains in versions sections. Previously, you could only set versions, for example: zc.recipe.egg == 2.0.0a3 But now you can specify that you want versions greater than or less then a given version: zc.recipe.egg == >=2.0.0a3 This was added so that buildout2 can constrain the version of zc.recipe.egg used without requiring a specific version. buildout1 will be updated with this feature too, so that it can prevent itself from using newer versions of zc.recipe.egg and to prevent upgrading itself to buildout2. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm

1 0