site enhancements (request for review)
A few weeks ago I put together a patch to site.py for Python 2.5 http://python.org/sf/1174614 that solves three major deficiencies: (1) All site dirs must exist on the filesystem: Since PEP 302 (New Import Hooks) was adopted, this is not necessarily true. sys.meta_path and sys.path_hooks can have valid uses for non- existent paths. Even the standard zipimport hook supports in-zip- file paths (i.e. foo.zip/bar). (2) The directories added to sys.path by .pth files are not scanned for further .pth files. If they were, you could make life much easier on developers and users of multi-user systems. For example, it would be possible for an administrator to drop in a .pth file into the system-wide site-packages to allow users to have their own local site-packages folder. Currently, you could try this, but it wouldn't work because many packages such as PIL, Numeric, and PyObjC take advantage of .pth files during their installation. (3) To support the above use case, .pth files should be allowed to use os.path.expanduser(), so you can toss a tilde in front and do the right thing. Currently, the only way to support (2) is to use an ugly "import" pth hook. So far, it seem that only JvR has reviewed the patch, and recommends apply. I'd like to apply it, but it should probably have a bit more review first. If no negative comments show up for a week or two, I'll assume that people like it or don't care, and apply. -bob
Bob Ippolito wrote:
A few weeks ago I put together a patch to site.py for Python 2.5 http://python.org/sf/1174614 that solves three major deficiencies:
[concerning .pth files]
While we're on the subject of .pth files, what about the idea of scanning the directory containing the main .py file for .pth files? This would make it easier to have collections of Python programs sharing a common set of modules, without having to either install them system-wide or write hairy sys.path-manipulating code or use platform-dependent symlink or PATH hacks. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+
On Apr 26, 2005, at 1:12 AM, Greg Ewing wrote:
Bob Ippolito wrote:
A few weeks ago I put together a patch to site.py for Python 2.5 http://python.org/sf/1174614 that solves three major deficiencies:
[concerning .pth files]
While we're on the subject of .pth files, what about the idea of scanning the directory containing the main .py file for .pth files? This would make it easier to have collections of Python programs sharing a common set of modules, without having to either install them system-wide or write hairy sys.path-manipulating code or use platform-dependent symlink or PATH hacks.
I don't think I'd ever use that, but it doesn't sound like a terrible idea. -bob
While we're on the subject of .pth files, what about the idea of scanning the directory containing the main .py file for .pth files? This would make it easier to have collections of Python programs sharing a common set of modules, without having to either install them system-wide or write hairy sys.path-manipulating code or use platform-dependent symlink or PATH hacks.
I do that all the time without .pth files -- I just put all the common modules in a package and place the package in the directory containing the "main" .py files. I do have use cases where for reasons of separate development cycles (etc.) I have some code (usually experimental or "unofficial" in some way) in a different place that also needs access to the same set of common modules, and there I use explicit sys.path manipulations. I think that even if the proposed feature was available I wouldn't switch to it -- it's too easy to forget about the .pth file and be confused when it points to the wrong place. That's also the reason why I don't use symlinks or $PYTHONPATH for this purpose. EIBTI. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
I do that all the time without .pth files -- I just put all the common modules in a package and place the package in the directory containing the "main" .py files.
That's fine as long as you're willing to put all the main .py files together in one directory, with everything else below it, but sometimes it's not convenient to do that. I had a use for this the other night, involving two applications which each consisted of multiple .py files (belonging only to that application) plus some shared ones. I wanted to have a directory for each application containing all the files private to that application.
it's too easy to forget about the .pth file and be confused when it points to the wrong place.
I don't think I would be confused by that. I would consider the .pth file to be a part of the source code of the application, to be maintained along with it. If I got an ImportError for one of the shared modules, checking the .pth file would be a natural thing to do -- just as would checking the sys.path munging code if it were being done that way. And a .pth file would be much easier to maintain than the hairy-looking code you need to write to munge sys.path in an equivalent way.
That's also the reason why I don't use symlinks or $PYTHONPATH for this purpose.
Another reason for avoiding that is portability. My first attempt at solving the aforementioned problem used symlinks. Trouble is, it also had to work on Windows running under Virtual PC mounting the source directory from the host system as a file share, and it turns out that reading a unix symlink from the Windows end just returns the contents of the link. Aaarrghh! Braindamage! -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+
participants (3)
-
Bob Ippolito -
Greg Ewing -
Guido van Rossum