[Python-Dev] Internal documentation for egg formats now available

[Phillip J. Eby]

At 06:47 PM 4/27/2006 +0200, M.-A. Lemburg wrote:
>Just read that you are hijacking site.py for setuptools'
>"just works" purposes.

"hijacking" isn't the word I'd use; "wrapping" is what it actually 
does.  The standard site.py is executed, there is just some pre- and 
post-processing of sys.path.


>Please be aware that by allowing .pth files in all PYTHONPATH
>directories you are opening up a security hole - anyone with
>write-permission to one of these .pth files could manipulate
>other user's use of Python.

FUD.  Write access to a PYTHONPATH-listed directory already implies 
complete control over the user's Python environment.  This doesn't 
introduce any issues that aren't implicit in the very existence of PYTHONPATH.


>That's the reason why only site-packages .pth files are
>taken into account, since normally only root has write
>access to this directory.

False.  On OS X, Python processes any .pth files that are found in the 
~/Library/Python2.x/site-packages directory.  (Which means, by the way, 
that OS X users don't need most of these hacks; they just point their 
install directory to their personal site-packages, and it already Just 
Works.  Setuptools only introduces PYTHONPATH hacks to make this work on 
*other* platforms.)


>The added startup time for scanning PYTHONPATH for .pth
>files and processing them is also not mentioned in the
>documentation. Every Python invocation would have to pay
>for this - regardless of whether eggs are used or not.

FUD again.  This happens if and only if:

1. You used easy_install to install a package in such a way that the .pth 
file is required (i.e., you don't choose multi-version mode or 
single-version externally-managed mode)

2. You include the affected directory in PYTHONPATH

So the idea that "every Python invocation would have to pay for this" is 
false.  People who care about the performance have plenty of options for 
controlling this.

Is there a nice HOWTO that explains what to do if you care more about 
performance than convenience?  No.  Feel free to contribute one.


>I really wish that we could agree on an installation format
>for package (meta) data that does *not* rely on ZIP files.

There is one already, and it's used if you select single-version 
externally-managed mode explicitly, or if you install using --root.


>All the unnecessary magic that you have in your design would
>just go away - together with most of the issues people on this
>list have with it.

Would you settle for a way to make a one-time ~/.pydistutils.cfg entry that 
will make setuptools act the way you want it to?  That is, a way to make 
setuptools-based packages default to --single-version-externally-managed 
mode for installation on a given machine or machine(s)?  That way, you'll 
never have to wonder whether a package uses setuptools or not, you can just 
"setup.py install" and be happy.