On Jul 7, 2005, at 1:12 PM, Phillip J. Eby wrote:

At 12:02 PM 7/7/2005 -0400, Ryan Tomayko wrote:

...

I'm happy to announce that an initial 0.1.0 version of Python Buildutils is available.

Looks nice; I could easily see wanting to steal some of those commands for setuptools. :)

Interestingly, there's some overlap between the commands as they exist now.

Yep. I've been meaning to talk to you about that, in fact. Buildutils depends on and utilizes setuptools quite a bit. So there's really no reason not to move buildutils commands and functionality down into setuptools if they seem more relevant there. Ditto with distutils really. We may want to just start a conversation to establish whether splitting commands up as we our now is a even the best way to do it. Do we need distutils, setuptools, and buildutils? I think there's a good and obvious separation but we could probably pound out some general guidelines for where different types of functionality should go.

For example, your 'use' command could use the setuptools "develop" command to install and uninstall development eggs safely on all platforms (note that Windows doesn't do symlinks) and your code is making a false assumption that .egg-info will always be in the package checkout's root. It might be better to have your 'use' command just be a wrapper for running the 'develop' command in each of the named projects, so that those details will work right. (i.e., AFAICT 'use' is short for cd-ing to each project and doing "pbu develop".)

Absolutely. I need to revisit use now that you've added support for develop in setuptools. I didn't even document it because I figured it would be retooled or removed completely. I added it back when we first had our discussion about development environment layout.

Also, there's some limited overlap between the new "upload" in setuptools and your "publish"; the difference being that "upload" only does source and egg uploads to PyPI. (I just implemented a workaround for the absence of egg support in PyPI.)

publish is a bit weird right now and I imagine it will be refactored considerably in the coming month. I wonder if if might be better to use subcommands here? e.g. publish_pypi, publish_scp, etc. and then a top level publish command that determines which subcommands to run based on setup.cfg or other metadata. Ryan Tomayko rtomayko@gmail.com http://naeblis.cx/rtomayko/

At 11:16 AM 7/8/2005 -0400, Ryan Tomayko wrote:

If I want to use the development version of kid from buildutils, I would do:

cd devel/kid python setup.py develop --install-dir=../buildutils --script- dir=../buildutils -m

FYI, --script-dir defaults to --install-dir if you explicitly set --install-dir, so the above could read: python setup.py develop --install-dir=../buildutils -m or for the ultimate in cryptic brevity: python setup.py develop -md../buildutils :)

I may still leave the use command in tact and have it perform these commands on multiple projects the way it does today. For example, if I wanted to use the development versions of kid and pudge from buildutils:

pbu use --projects=kid,pudge

And to no longer use those projects:

pbu use --stop --projects=kid,pudge

I'm not sure if you're saying you're going to keep having 'use' do what it does now to implement this, or whether you mean you're going to have it run "setup.py develop" with the right options in its targets. I'd recommend the latter, since your current implementation doesn't work for projects that put their source code in a subdirectory of the project directory (e.g. a "lib" or "src" subdirectory). If you make it a wrapper for "develop", then any setuptools-using project should be able to use it, without being forced to follow your personal choice of project directory layout. :) If you do this, you should also use the current package's: os.path.abspath(self.get_finalized_command('egg_info').egg_base) as the --install-dir for the develop commands, so that projects with "src" or "lib" dirs can be the target of the use command, as well as their source. (egg_base is the directory where the project's .egg-info will be located; getting it from the 'egg_info' command allows advanced users to override it in setup.cfg if it can't be determined from the distribution package_dir setting(s).) By the way, the whole idea of using develop to link one package to another is quite interesting; I hadn't thought of making one package the target of another like this, but it seems useful and ought to work. Even more interesting is that if you run "develop" on the package that has had other projects develop-linked to it, then the whole tree of dependencies becomes available in the target. However, you should be careful to avoid creating a cycle of develop-links, because it will cause an infinite loop at runtime. pkg_resources does not have any ability to detect such link cycles at the moment, although I could probably add something without too much difficulty.

On Jul 8, 2005, at 3:14 PM, Kevin Dangoor wrote:

As an aside, the thing that I'm trying to debug in A is a "Not a directory" error that I think is a __file__ usage. Ahh, the joys of being among the first to try to exploit this stuff. But I must say that, despite this, eggs are working great!

Yea. I'd like to chime in on that too and just say that after working with setuptools for the past couple of weeks I think it's going to work out nicely. So there's some market validation for you :) Ryan Tomayko rtomayko@gmail.com http://naeblis.cx/rtomayko/

On 7/8/05, Phillip J. Eby <pje@telecommunity.com> wrote:

...

Removing the older egg from the easy-install.pth file fixed this problem... I was able to find the new version that was linked in just fine.

Sounds like it is working for you, then. :)

Seriously, pkg_resources is very conservative right now about versioning; if you install something without the -m flag, then you are telling it that that is the version you want to use, period. Part of the problem is that pkg_resources adds new versions to the *end* of sys.path, which means that if you have an older version already on sys.path when you ask for the newer one, there's no way that pkg_resources can safely change the path to allow the new version to take effect.

Ahh, that makes sense to me now. I discovered that I even had another copy of that package lying around on my path (left over from when I expanded the egg for py2app), which certainly couldn't have been helping things... Kevin

On Jul 7, 2005, at 1:27 PM, Phillip J. Eby wrote:

At 12:02 PM 7/7/2005 -0400, Ryan Tomayko wrote:

...

I'm happy to announce that an initial 0.1.0 version of Python Buildutils is available.

By the way, I notice your extension doc suggests subclassing distutils.cmd.Command rather than setuptools.Command, and your setup.py doesn't use setuptools.setup(). You might want to revise those, because using setuptools.setup() provides important additional features that distutils.core.setup() cannot:

* running "setup.py install" installs the package as an egg+.pth, for easy uninstalls and upgrades

* running "setup.py sdist" with no MANIFEST.in file will automatically include all files under revision control (CVS or svn) in the project tree, not just those found by the distutils default search algorithm that searches only for source files, not docs, headers, or data.

* running "setup.py install" when setuptools is not installed will bootstrap an actual installation of setuptools, whereas your current arrangement will not actually install it, just put it on sys.path during the execution of setup.py

Also, 'depends.txt' is now unneeded; if you use 'from setuptools import setup' then you can pass 'install_requires' and 'extras_require' keywords to setup() and get the same effect. (If you do this, you should remove the depends.txt file from your .egg- info directory, so as not to list them twice; the keyword arguments get written to a different file.)

These are all things I have on my list. I have a hard time keeping pace with your progress on setuptools. :) But yea, my plan is to try to get this straightened out for a quick release tonight. Ryan Tomayko rtomayko@gmail.com http://naeblis.cx/rtomayko/