[Python-Dev] Internal documentation for egg formats now available
Phillip J. Eby
pje at telecommunity.com
Fri Apr 28 19:33:51 CEST 2006
At 06:20 PM 4/28/2006 +0200, M.-A. Lemburg wrote:
>A single malicious .pth file on PYTHONPATH could trick
>the user into running Python modules that he isn't really
>Adding directories to PYTHONPATH which are writable by
>others is easy, just add '/tmp' as entry in some .pth
>file. Then put e.g. a modified os.py into /tmp and have
>it execute 'rm -rf /' at import time.
You still haven't explained *how* these malicious .pth files or entries
come into being *without the user's permission*. How does this malicious
user get write access to the .pth file to put /tmp into it? Whatever your
answer is, *that* is the security hole, not the existence of usable .pth files.
So this is still just handwaving and FUD.
> > And if you replace 'syspathtools.use()' in that email, with
> > 'pkg_resources.require()', then it describes *exactly how setuptools
> > works with .egg directories today*.
>Interesting that you used that idea, because back then you didn't
>reply to the email. Looks like I deserve some credit ;-)
In truth, I completely forgot about that post and only rediscovered it
yesterday. When Bob and I were doing the initial design, he said he
thought that the naively-unzipped form of an egg should be usable as-is,
and I agreed that it made sense to do that.
>If you've already implemented this (which I wasn't aware of, since
>when I played with setuptools it kept installing .egg ZIP files),
See the --always-unzip option to easy_install, which has been there since
0.5a9 -- i.e., since July 2005. Package authors can also mark their
packages "zip_safe=False" to force it to be installed unzipped (unless the
user forces it to be zipped using --zip-ok).
>then why don't you make .egg *directories* the standard installation
>scheme, instead of insisting on having .egg ZIP files in
Because zip files add *less* import overhead over the life of a program
(even a relatively short program) than directories do!
(Which, by the way, I believe I've explained to you on the distutils-SIG on
more than one occasion, during some of the previous N times you brought
>I wonder why you don't always create .egg directories.
Performance, plain and simple.
What you *should* be benchmarking when you test performance impact, is
twenty directories at the front of sys.path vs. twenty zipfiles at the
front, both for "python -c pass" and for a program that does a few stdlib
imports. That will show you why EasyInstall never unzips eggs if it can
>* I am against using ZIP as medium for package installation.
Then add this to your distutils.cfg or ~/.pydistutils.cfg and be happy
always_unzip = 1
If, however, you mean that you want *everyone else* to do as you prefer,
forget about it, 'cause it's not going to happen. You aren't setuptools'
target user, so the defaults aren't going to be catering to your prejudices
-- especially not the ones based on ignorance, like this one.
>* I am against using standard distutils commands to mean
> different things depending on which set of packages you
> have installed (in this case the "install" command)
And as I just pointed out, you've failed to show that there is any
practical purpose to this. I've already made the behavior an option, and
it's the default behavior when you use --root. You haven't shown any
actual use case for it being the default behavior, and practicality beats
purity. What's more, you said you don't want the option to make it the
default behavior for you. Which means that this is only about you deciding
what is "right" for *other* people than you to have, so I don't see any
reason to consider that subject any further. If other people want that
behavior, they can certainly speak up for themselves about what *actual*
(not hypothetical) problems it creates for them. (And when they do speak
up, as some system packagers did, I made --root enable the old behavior and
that addressed the *actual* problems the new behavior created for that
>Phillip, you're always side-stepping answering questions
>(and cutting away quoted email text doesn't help either).
Repeating arguments I've already shown to be spurious helps even less. I
don't see much point to chasing red herrings and straw men, either.
>You should be aware that both package authors and users
>are capable of learning and adapting.
Then they can certainly adapt to "install" doing the right thing for most
users and most packages: to install things in such a way that they can be
uninstalled or upgraded cleanly, by default.
>If we tell package authors to put a link to their eggs on
>PyPI along with an MD5 sum, then the users be in a much
>better position and could feel more secure about what they
>download and install.
People who want to do that can easily use:
or put it in their configuration files to make it the default behavior. See:
>Have a nice weekend,
It'd be an even nicer one if people objecting to things would read the
documentation first, to find out whether their objections actually
apply. Your zip vs. directory arguments would've been moot if you'd read
things like this:
Or for that matter, if you'd read the documentation that was linked to in
the post that started this thread, which explains that .egg can be a
zipfile or a directory. Or if you'd read the parts of the EasyInstall
documentation where it explains that OS X already processes .pth files from
If you don't have time to read the documentation, please have the courtesy
to *ask questions* to try to alleviate your concerns, preferably *before*
you launch attacks using nothing but vague premonitions based on ignorance.
More information about the Python-Dev