Donald Stufft <donald <at> stufft.io> writes:
1. That unpacking into the home directory is problematic because users that run services often don’t have home directories. This you waved away by saying that the home directory isn’t a required place, to which Jim responded that unpacking them anywhere else was likely to have access control / security issues and “generally cause pain”.
$HOME is just a sensible default location for caches, but as long as other locations can be used in situations where there's no HOME directory, I don't see how that's "waving away". If a service has no access to writeable disk locations at all, I can see how that might pose a problem, but that hardly seems like a use case which should dictate policies for every other scenario. OTOH "anywhere else was likely to have access control / security issues and 'generally cause pain'" is light on details.
wholly agree with, intact Mitre has assigned CVE’s to applications that did switch their Egg Cache to directories other than HOME.
Are you referring to the situation where users of setuptools set PYTHON_EGG_CACHE to a world-writeable location? If so, the problem would appear to be with the using code, not setuptools itself. AFAICT it was felt that while CVEs could be assigned against individual applications that did this, and setuptools itself was not assigned a CVE. If a HOME-less application uses an alternative cache location which isn't world-writeable, does a security issue arise?
2. Zipped Eggs are more difficult to work with for debugging purposes, which you responded saying that Wheels are a “deployment format” (which I don’t believe is supported by the PEP at all) and Jim responded with his experience in developing against things that got installed as zipped Eggs and the pain that has personally caused him.
3. Zipped imports (In Jim’s experience) are slower than unpacked imports, which you replied (quite literally) "Caveat emptor”. Jim’s response was that this had been tried with Egg and had been found to be more pain than it was worth. In his exact words:
"It's been tried with eggs. This is not new ground. Encouraging people to do this is going to cause pain and resentment.
I think one of the reasons there's so much (IMO mostly irrational) hate for eggs is that people think you can only used zipped eggs, and zipped eggs cause pain and agony.”
He closes up with,
"Importing from zipped eggs has proved itself to be an anti pattern.
Buildout (as of buildout 2) always unzips eggs.”
So you have pip that doesn’t use zipped Eggs, buildout which made sure in buildout 2 (presumably drawing from the lessons of buildout 1) to always unzip eggs, and easy_install which does support zipped eggs (sometimes, if it thinks that particular Egg will support it).
Why do we insist upon repeating the mistakes of the past?
Because there are scenarios where that functionality is desirable, and the downsides (such as greater difficulty in debugging, no caching of byte code, lower performance of imports) are tolerable. Just because something can cause problems in some scenarios is no reason for a wholesale ban. C makes segfaulting easy, but that doesn't mean that C extensions are right out ;-) Python is generally a "consenting adults" kind of system. An important difference from the setuptools/easy_install/zipped eggs experience is that nothing happens automatically - a specific action has to be taken to add wheels to sys.path, whereas in the zipped eggs case, sys.path manipulation happens under the hood. Since you don't know it's happening, failures are bound to seem weird - "it's nothing I've done, I'm sure!" Making something possible != recommending its use wholesale.
Zipped Eggs have a long long history of causing very weird failures
conditions,
you say your Wheel.mount checks for compatibility, does it also check that the library to be installed isn’t using __file__ shenanigans instead of pkgutil.get_data()? easy_install tried to do this with varying degrees of success, I’ve never seen much except pain from zipped Eggs.
Weird failure conditions are only weird because they haven't been investigated to find the exact cause. There's a lot of software around that can't run from zips because of __file__ shenanigans - that doesn't negate the very real usefulness of zipimport for other software. If I want to write software that runs from zip, I could take care to use pkgutil rather than __file__. Presumably, others can do the same. Regards, Vinay Sajip