So, coming back to the idea of a working environment, an isolated and more-or-less self-contained environment for holding installed packages. Sorry if this is a little scattered. I'm just summarizing my thoughts and the open issues I see, in no particular order (because I'm not sure what order to approach these things in). I'm assuming such an environment will be encapsulated in a single directory, looking something like: env/ bin/ lib/python2.4/ src/ conf/ The conf/ directory doesn't really relate to much of this specifically, but in many situations it would be useful. Depending on the situation, other subdirectories may exist. Each of the scripts in bin/ should know what their working environment is. This is slightly tricky, depending on what that means. If it is a totally isolated environment -- no site-packages on sys.path -- then I feel like the script wrappers have to be shell scripts, to invoke Python with -S (which is hard to do portably on the #! line). I don't know the details of doing the same thing on Windows, but I assume it is possible. The actual directory location should be portable -- all paths should be relative, and you should be able to move the directory around. lib/python2.4/ is for packages. I'm almost inclined to say that --single-version-externally-managed makes sense on some level, with a record kept in some standard place (lib/python2.4/install-record.txt?) -- but I'm basically indifferent. I at least don't see a need for multiple simultaneous versions in this setup, and multiple versions do lead to confusion. Version metadata is still nice, of course. src/ is for checkouts, where each package is installed with setup.py develop. These are naturally single-version, which is part of why I like the idea of only using single-version setups. I'm a little unsure of how src/ should be layed out. In practice I want all "my" packages to be installed in src/ as checkouts, either from tags or the trunk (or a branch or whatever). So I'm not sure if I should name the subdirectories after the package, or maybe even the package plus a tag name. One of the things SwitchTower (now "Cappucino", I think) does in Rails land is it makes a dated checkout, then activates that checkout (it does it with a symlink, we'd do it with setup.py develop). It then rolls back by switching to an existing checkout. Of course svn switch + svn up does this in place, and with less checkout trash laying around, even if rollbacks aren't as fast as a result. So, I'm thinking just src/PackageName/ There's an installation issue there -- it would be nice if I could say "these are the packages I want to install as editable" and easy_install would pick those up (maybe detecting based on what package index the package was found in) and install them in src/ as editable. sys.path would contain /usr/lib/python2.4, optionally /usr/lib/python2.4/site-packages, and env/lib/python2.4/, and all the similar directories. Unfortunately figuring out what "similar" directories there are is hard. sys.path on my machine now has 63 entries normally and 12 with python -S. I guess I'd really like to start with 12 and build up, instead of 63 and try to strip them down. Installation as a whole is an open issue. Putting in env/setup.cfg with the setting specific to that working environment works to a degree -- easy_install will pick it up if invoked from there. But that doesn't work with setup.py develop, or setup.py install, or some other scenarios. The system distutils.cfg doesn't really work, because the only expansion it knows how to do is of user directories, so there's little way to pass interesting information in (like a "this is my setup.cfg" environmental variable or something). Maybe with PYTHONPATH to indicate the working environment, and a distutils monkeypatch put into lib/python2.4/distutils/__init__.py? I played around with putting the path setup in sitecustomize, but that runs after site.py, and doesn't run at all if python -S is used, so it seems like it brings in too much before it can remove stuff. Another option is a completely new python interpreter bound to the environment. Basically the virtual-python.py option (http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-pytho...). In this model using env/bin/python indicate the proper environment, and you'd have local installs of *everything* including easy_install. This fixes so many problems without crazy hacks that it strongly appeals to me, especially if we can make it somewhat lighter. I get this /usr/lib/python2.4.zip on my path, that doesn't usually exist (does it ever get created by default); if we could create that somehow on demand and use such a big-bundle-zip, that seems lighter and faster and nicer. If we just put .pyc files in it, and those .pyc files refer back to the actual module source (in /usr/lib/python2.4/), then tracebacks should also still work, right? No actual symlinks either, so it should work on Windows. I'm not entirely sure where I'm going with this, though. Sorry for the length. I've been stewing on this without a lot of progress since PyCon so I thought I'd just throw out my current thoughts. Maybe what I really want to do is hack on virtual-python.py some more and see where that gets me. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
participants (5)
-
Anthony Baxter
-
Ian Bicking
-
Iwan Vosloo
-
Jim Fulton
-
Phillip J. Eby