Mailman 3 zc.recipe.egg extension supporting "easy_install --editable" and running arbitrary setup.py commands - Distutils-SIG

Sept. 30, 2007

      I wrote some recipes that extend the base zc.recipe.egg recipes.  They
currently live in my own namespace for development.  The recipe code has
good test coverage and documentation.  I'm wondering if there's any
interesting in incorporating them into zc.recipe.egg.  If so, I'll be
happy to integrate it and commit or submit patches.  If not, I'll just
release it as z3c.recipe.egg.

The editable recipe adds support for downloading source distributions
using easy_install.  A svn checkout can be done if the appropriate
find-link is available.  Also, it also allows build-directory to be
specified and as such if build-directory is not in the part directory,
the source distribution will be preserved when buildout is run.  So
among other things, this allow for preserving source distributions in
development.  Unfortunately, this duplicates the effort in ???.  I
should have released sooner.

The setup recipe simply calls arbitrary setup.py commands in specified
source distributions.  I use this, for example, in buildout files that
release multiple projects at once.

Below I've attached the documentation for the two recipes for further
view.  You can see or checkout the source here:

https://svn.plone.org/svn/collective/grouparchy/grouparchy.recipe.egg/trunk/

Ross

;-*-Doctest-*-
======================
Editable Distributions
======================

The grouparchy.recipe.egg.Editable recipe use the easy_install
--editable and --build-directory options to download a source
distribution.

If no build-directory is specified the part directory will be used.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demoneeded
    ...
    ... [demoneeded]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... """ % dict(server=link_server))

    >>> print system(buildout),
    Installing demoneeded.

    >>> ls(sample_buildout, 'parts', 'demoneeded')
    d  demoneeded

If the develop option is true then the distributions will be
installed in develop mode.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demoneeded
    ...
    ... [demoneeded]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... eggs = demoneeded
    ... develop = true
    ... """ % dict(server=link_server))

    >>> print system(buildout),
    Uninstalling demoneeded.
    Installing demoneeded.
    Develop: '/sample-buildout/parts/demoneeded/demoneeded'

    >>> ls(sample_buildout, 'develop-eggs')
    -  demoneeded.egg-link
    -  grouparchy.recipe.egg.egg-link
    -  zc.recipe.egg.egg-link

Multiple distributions can be specified.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demoneeded
    ...
    ... [demoneeded]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... eggs = demoneeded
    ...        extdemo
    ... """ % dict(server=link_server))

    >>> print system(buildout),
    Uninstalling demoneeded.
    Installing demoneeded.

    >>> ls(sample_buildout, 'parts', 'demoneeded')
    d  demoneeded
    d  extdemo

If a new qualifying version is available then the part will be
reinstalled.

    >>> write(
    ...     sample_buildout, 'parts', 'demoneeded', 'demoneeded',
    ...     'setup.py', """
    ... from setuptools import setup
    ... setup(name='demoneeded', py_modules=['eggrecipedemobeeded'],
    ...       zip_safe=True, version='1.3', author='bob',
    ...       url='bob', author_email='bob')
    ... """)

    >>> import os
    >>> sdist(os.path.join(
    ...     sample_buildout, 'parts', 'demoneeded', 'demoneeded'),
    ...     sample_eggs)

    >>> print system(buildout),
    Uninstalling demoneeded.
    Installing demoneeded.

    >>> ls(sample_buildout, 'parts', 'demoneeded')
    d  demoneeded
    d  extdemo

    >>> cat(
    ...     sample_buildout, 'parts', 'demoneeded', 'demoneeded',
    ...     'setup.py')
    <BLANKLINE>
    from setuptools import setup
    setup(name='demoneeded', py_modules=['eggrecipedemobeeded'],
          zip_safe=True, version='1.3', author='bob',
          url='bob', author_email='bob')

If the part is not to be reinstalled, running buildout in offline or
non-newest mode, the recipe preserves the existing installation.

    >>> print system(buildout+' -No'),
    Updating demoneeded.

    >>> ls(sample_buildout, 'parts', 'demoneeded')
    d  demoneeded
    d  extdemo

If the part is to be reinstalled and buildout is run in offline or
non-newest mode, the recipe does nothing.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demoneeded
    ...
    ... [demoneeded]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... eggs = demoneeded
    ... """ % dict(server=link_server))

    >>> print system(buildout+' -N'),
    Uninstalling demoneeded.
    Installing demoneeded.

    >>> ls(sample_buildout, 'parts')

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demoneeded
    ...
    ... [demoneeded]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... eggs = demoneeded
    ...        extdemo
    ... """ % dict(server=link_server))

    >>> print system(buildout+' -o'),
    Uninstalling demoneeded.
    Installing demoneeded.

    >>> ls(sample_buildout, 'parts')

If build-directory is specified, the source distributions will be
extracted there.

    >>> mkdir(sample_buildout, 'src')

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demoneeded
    ...
    ... [demoneeded]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... eggs = demoneeded
    ...        extdemo
    ... build-directory = src
    ... """ % dict(server=link_server))

    >>> print system(buildout),
    Uninstalling demoneeded.
    Installing demoneeded.

    >>> ls(sample_buildout, 'parts')
    >>> ls(sample_buildout, 'src')
    d  demoneeded
    d  extdemo

If the build directory is present, it will not be removed when the
part is installed preserving any changes.

    >>> write(
    ...     sample_buildout, 'src', 'demoneeded', 'setup.py', """
    ... from setuptools import setup
    ... setup(name='demoneeded', py_modules=['eggrecipedemobeeded'],
    ...       zip_safe=True, version='1.4', author='bob',
    ...       url='bob', author_email='bob')
    ... """)

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = demo
    ...
    ... [demo]
    ... recipe = grouparchy.recipe.egg:editable
    ... find-links = %(server)s
    ... eggs = demoneeded
    ...        extdemo
    ... build-directory = src
    ... """ % dict(server=link_server))

    >>> print system(buildout),
    Uninstalling demoneeded.
    Installing demo.

    >>> ls(sample_buildout, 'src')
    d  demoneeded
    d  extdemo

    >>> cat(
    ...     sample_buildout, 'src', 'demoneeded', 'setup.py')
    <BLANKLINE>
    from setuptools import setup
    setup(name='demoneeded', py_modules=['eggrecipedemobeeded'],
          zip_safe=True, version='1.4', author='bob',
          url='bob', author_email='bob')

;-*-Doctest-*-
=====================
Running Setup Scripts
=====================

The grouparchy.recipe.egg.Setup recipe calls arbitrary setup.py
commands on a distribution in a buildout.

The buildout requires only the setup argument which specifies a setup
script or the directory containing a setup.py script.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = foo
    ...
    ... [foo]
    ... recipe = grouparchy.recipe.egg:setup
    ... setup = foo
    ... args = sdist
    ... """)

The seupt script is run with the specified arguments.

    >>> print system(buildout),
    Installing foo.
    Running setup script '/sample-buildout/foo/setup.py'.
    running sdist
    running egg_info
    creating foo.egg-info
    writing foo.egg-info/PKG-INFO
    writing top-level names to foo.egg-info/top_level.txt
    writing dependency_links to foo.egg-info/dependency_links.txt
    writing manifest file 'foo.egg-info/SOURCES.txt'
    writing manifest file 'foo.egg-info/SOURCES.txt'
    creating foo-0.0.0
    creating foo-0.0.0/foo
    creating foo-0.0.0/foo.egg-info
    making hard links in foo-0.0.0...
    hard linking README.txt -> foo-0.0.0
    hard linking setup.py -> foo-0.0.0
    hard linking foo/__init__.py -> foo-0.0.0/foo
    hard linking foo.egg-info/PKG-INFO -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/SOURCES.txt -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/dependency_links.txt -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/top_level.txt -> foo-0.0.0/foo.egg-info
    Writing foo-0.0.0/setup.cfg
    creating dist
    tar -cf dist/foo-0.0.0.tar foo-0.0.0
    gzip -f9 dist/foo-0.0.0.tar
    removing 'foo-0.0.0' (and everything under it)

    >>> ls(sample_buildout, 'foo', 'dist')
    -  foo-0.0.0.tar.gz

    >>> remove(sample_buildout, 'foo', 'dist', 'foo-0.0.0.tar.gz')

The setup script is also run on update.

    >>> print system(buildout),
    Updating foo.
    Running setup script '/sample-buildout/foo/setup.py'.
    running sdist
    running egg_info
    writing foo.egg-info/PKG-INFO
    writing top-level names to foo.egg-info/top_level.txt
    writing dependency_links to foo.egg-info/dependency_links.txt
    writing manifest file 'foo.egg-info/SOURCES.txt'
    creating foo-0.0.0
    creating foo-0.0.0/foo
    creating foo-0.0.0/foo.egg-info
    making hard links in foo-0.0.0...
    hard linking README.txt -> foo-0.0.0
    hard linking setup.py -> foo-0.0.0
    hard linking foo/__init__.py -> foo-0.0.0/foo
    hard linking foo.egg-info/PKG-INFO -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/SOURCES.txt -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/dependency_links.txt -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/top_level.txt -> foo-0.0.0/foo.egg-info
    Writing foo-0.0.0/setup.cfg
    tar -cf dist/foo-0.0.0.tar foo-0.0.0
    gzip -f9 dist/foo-0.0.0.tar
    removing 'foo-0.0.0' (and everything under it)

    >>> ls(sample_buildout, 'foo', 'dist')
    -  foo-0.0.0.tar.gz

    >>> remove(sample_buildout, 'foo', 'dist', 'foo-0.0.0.tar.gz')

Multiple setup directories or scripts can be specified.  Add the
second distribution.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = foo
    ...
    ... [foo]
    ... recipe = grouparchy.recipe.egg:setup
    ... setup =
    ...     foo
    ...     bar
    ... args = sdist
    ... """)

Now the buildout will run the setup script on both.

    >>> print system(buildout),
    Uninstalling foo.
    Installing foo.
    Running setup script '/sample-buildout/foo/setup.py'.
    running sdist
    running egg_info
    writing foo.egg-info/PKG-INFO
    writing top-level names to foo.egg-info/top_level.txt
    writing dependency_links to foo.egg-info/dependency_links.txt
    writing manifest file 'foo.egg-info/SOURCES.txt'
    creating foo-0.0.0
    creating foo-0.0.0/foo
    creating foo-0.0.0/foo.egg-info
    making hard links in foo-0.0.0...
    hard linking README.txt -> foo-0.0.0
    hard linking setup.py -> foo-0.0.0
    hard linking foo/__init__.py -> foo-0.0.0/foo
    hard linking foo.egg-info/PKG-INFO -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/SOURCES.txt -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/dependency_links.txt -> foo-0.0.0/foo.egg-info
    hard linking foo.egg-info/top_level.txt -> foo-0.0.0/foo.egg-info
    Writing foo-0.0.0/setup.cfg
    tar -cf dist/foo-0.0.0.tar foo-0.0.0
    gzip -f9 dist/foo-0.0.0.tar
    removing 'foo-0.0.0' (and everything under it)
    Running setup script '/sample-buildout/bar/setup.py'.
    running sdist
    running egg_info
    creating bar.egg-info
    writing bar.egg-info/PKG-INFO
    writing top-level names to bar.egg-info/top_level.txt
    writing dependency_links to bar.egg-info/dependency_links.txt
    writing manifest file 'bar.egg-info/SOURCES.txt'
    writing manifest file 'bar.egg-info/SOURCES.txt'
    creating bar-0.0.0
    creating bar-0.0.0/bar
    creating bar-0.0.0/bar.egg-info
    making hard links in bar-0.0.0...
    hard linking README.txt -> bar-0.0.0
    hard linking setup.py -> bar-0.0.0
    hard linking bar/__init__.py -> bar-0.0.0/bar
    hard linking bar.egg-info/PKG-INFO -> bar-0.0.0/bar.egg-info
    hard linking bar.egg-info/SOURCES.txt -> bar-0.0.0/bar.egg-info
    hard linking bar.egg-info/dependency_links.txt -> bar-0.0.0/bar.egg-info
    hard linking bar.egg-info/top_level.txt -> bar-0.0.0/bar.egg-info
    Writing bar-0.0.0/setup.cfg
    creating dist
    tar -cf dist/bar-0.0.0.tar bar-0.0.0
    gzip -f9 dist/bar-0.0.0.tar
    removing 'bar-0.0.0' (and everything under it)

    >>> ls(sample_buildout, 'foo', 'dist')
    -  foo-0.0.0.tar.gz

    >>> ls(sample_buildout, 'bar', 'dist')
    -  bar-0.0.0.tar.gz

    >>> remove(sample_buildout, 'foo', 'dist', 'foo-0.0.0.tar.gz')
    >>> remove(sample_buildout, 'bar', 'dist', 'bar-0.0.0.tar.gz')

If the develop option is true then the distributions will be
installed in develop mode.

    >>> write(sample_buildout, 'buildout.cfg',
    ... """
    ... [buildout]
    ... parts = foo
    ...
    ... [foo]
    ... recipe = grouparchy.recipe.egg:setup
    ... setup = foo
    ...         bar
    ... develop = true
    ... """)

    >>> print system(buildout),
    Uninstalling foo.
    Installing foo.
    Develop: '/sample-buildout/foo'
    Develop: '/sample-buildout/bar'

    >>> ls(sample_buildout, 'develop-eggs')
    -  bar.egg-link
    -  foo.egg-link
    -  grouparchy.recipe.egg.egg-link
    -  zc.recipe.egg.egg-link

zc.recipe.egg extension supporting "easy_install --editable" and running arbitrary setup.py commands

Ross Patterson

tags

participants (1)