EasyInstall: installing from svn
Now I'm feeling confused about how installation from svn should work. Or more generally, how deployment and development work. Usually in development, I like to install libraries directly out of svn (no distutil step). Then I can fiddle with them if necessary, commit changes (or create diffs if I don't have commit access), etc. This is all much easier to do when there's no distutil step. When deploying, that's not really what I want to do, even though I might very well be deploying from a repository. What easy_install does now with svn repositories is good in that case. The problem is that the two are conflicting. If I require a package, I'm not going to get the svn checkout, even if the checkout was on the path initially. Since the egg package names don't map to Python packages, you can't even really tell that a package is already available without egg installation. The only way I can think of right now to safely do this would be: try: import wsgiutils except ImportError: require('WSGI-Utils') Ick. So how can I set up a working environment that is close, but not too close, to the deployment environment? Maybe there's a way that, up front, I can specify which packages are made available through alternate methods? E.g., fulfill_requirement('WSGI-Utils', 'checkouts/WSGIUtils') (where adding the second path to sys.path will fulfill the requirement, irrespective of any eggs). Ian
On Jun 3, 2005, at 1:56 PM, Ian Bicking wrote:
Now I'm feeling confused about how installation from svn should work. Or more generally, how deployment and development work.
Usually in development, I like to install libraries directly out of svn (no distutil step). Then I can fiddle with them if necessary, commit changes (or create diffs if I don't have commit access), etc. This is all much easier to do when there's no distutil step.
In development, with svn libraries, I check out the source tree and create a .pth file <http://bob.pythonmac.org/archives/2005/02/06/ using-pth-files-for-python-development/>. This .pth file does one of two things: 1. If the source requires distutils to build stuff (i.e. PyObjC), then I will point the .pth at the distutils build directory, so I can simply "python setup.py build" to update. 2. If the source does not (i.e. Twisted), I point the .pth at the right place in the source tree From an EasyInstall perspective, 2 is hard to figure out because there is no general way (without running distutils, anyway) to figure out where the right place is. You could do it heuristically, but you'd probably be wrong :) -bob
At 03:56 PM 6/3/2005 -0500, Ian Bicking wrote:
Now I'm feeling confused about how installation from svn should work. Or more generally, how deployment and development work.
Usually in development, I like to install libraries directly out of svn (no distutil step). Then I can fiddle with them if necessary, commit changes (or create diffs if I don't have commit access), etc. This is all much easier to do when there's no distutil step.
When deploying, that's not really what I want to do, even though I might very well be deploying from a repository. What easy_install does now with svn repositories is good in that case.
The problem is that the two are conflicting. If I require a package, I'm not going to get the svn checkout, even if the checkout was on the path initially. Since the egg package names don't map to Python packages, you can't even really tell that a package is already available without egg installation. The only way I can think of right now to safely do this would be:
try: import wsgiutils except ImportError: require('WSGI-Utils')
Ick. So how can I set up a working environment that is close, but not too close, to the deployment environment? Maybe there's a way that, up front, I can specify which packages are made available through alternate methods? E.g., fulfill_requirement('WSGI-Utils', 'checkouts/WSGIUtils') (where adding the second path to sys.path will fulfill the requirement, irrespective of any eggs).
Ah, you've encountered the part of the egg docs that I haven't written yet. Basically, the solution is to have a PackageName.egg-info directory in the dependent package. pkg_resources.require() will see the directory and use it for the metadata. It will then know that the directory *contianing* the .egg-info directory is the directory that needs to be (and already is) on sys.path. Here's how you do it: go to the library you want to use for development, and run setup.py bdist_egg. This will create (as a side effect) a PackageName.egg-info subdirectory in the directory that you're going to be putting on sys.path. pkg_resources automatically detects this "development egg" and knows what version is present. This can work even for several packages installed in the same sys.path directory, you just have to have a PackageName.egg-info for each one, and you'll need to move or symlink them from their individual package locations. Basically, you're not the first person to have this problem, or rather, you are the first person to have the problem, you're just not the first person to think of it. :) At PyCon, I was describing the egg system design to Fred Drake, and he pointed out the problem of require()-ing something you're developing in this fashion, so later I came up with the PackageName.egg-info trick as a fix. (Originally, PackageName.egg-info was called EGG-INFO.in or something like that, and it lived in a different place; later I realized that the staging area could be reused to provide metadata for in-development packages, and renamed it to use the distro name so that more than one could live in the same parent directory on sys.path.)
participants (3)
-
Bob Ippolito -
Ian Bicking -
Phillip J. Eby