On Jan 29, 2014, at 10:00 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
-------------------------------------------- On Wed, 29/1/14, Donald Stufft <donald@stufft.io> wrote:
It’s hard to pin down because the failure modes of zipped eggs are nebulous themselves.
For instance take pip. I just recently redid the get-pip.py installer to use a zip file (not a Wheel or Egg) that contained pip and add that zip file straight to sys.path instead of unzipping it. One of the failure modes was that it was suddenly unable to validate TLS certificates. The reason why, was because it bundles it’s own cacerts.pem which it passes to the ssl library to validate the connection. In this case it’s impossible as far as I can tell to use pkgutil.get_data directly because the ssl library does not directly support pkgutil.get_data nor does it support passing the certificates as a string or other in memory.
That’s really the biggest problem with strictly pure python Zipped Eggs.
If that's all, I'm not overly worried as long as there's a mountability flag on wheels. I looked at running pip from a zip a while ago, the cacerts thing was easy to diagnose and fix - a five-or-ten-minute job. (I'm not trying to come across as a show-off.) Here's the code I added as a fix in pip/locations.py (no doubt it could be improved):
def _check_for_zip(): import zipimport global default_cert_path try: if isinstance(__loader__, zipimport.zipimporter): # running from a zip file. Save data to a handy location. data = __loader__.get_data(default_cert_path) destdir = os.path.expanduser('~/.pip') if not os.path.isdir(destdir): os.mkdir(destdir) default_cert_path = os.path.join(destdir, 'cacert.pem') with open(default_cert_path, 'wb') as f: f.write(data) except NameError: # __loader__ not defined, so not in zip pass
_check_for_zip() del _check_for_zip
Of course, pip wasn't originally designed to run from zip, so one would expect to find these kinds of issues. OTOH, if importable zips were more commonplace, then presumably people would think about these kinds of issues more when writing their code. Of course, in the above case, using pkgutil wouldn't have worked, and there are bound to be other similar cases, but I still don't see any real reason to fear the consequences of importable wheels.
Regards,
Vinay Sajip
FWIW I'm not particularly that upset about this feature itself. I think it's a bad idea and I think the way it locks the format into a particular method is a poor trade off and that there is possibly a better form for this feature, but if I had engaged the process argued my side, and then lost I could deal with that. What I'm upset about is that I can find no support that the *PEP* text (not the intentions of the PEP) contains explicit support for this feature (as in, it exists because we want it to work and we think people should use it) and little to no evidence that the fact that this was (apparently) a design goal was properly communicated (which includes multiple statements by various people including Daniel that it wasn't really designed for this, though Daniel is apparently recanting that meaning). According to PEP1 a PEP must "be a clear and complete description of the proposed enhancement". I believe that what was proposed for discussion, if it was indeed intended that Wheels should be generally importable by adding them to sys path, was not clear about the fact that this was part of the proposal (and in fact contains text that reads contrary to that intent), and because the proposal and the resulting discussion was unclear, that people, such as myself, were not given the chance to discuss this particular aspect of the proposal. I feel that as BDFL delegate Nick is abusing the fact that he ultimately accepted the PEP to circumvent the lack of a clear proposal of a particular design goal (albeit likely unwittingly) to include this part of the proposal without discussion. *This* is what I have a problem with. The PEP process exists to gather everyone's input on a proposal and that ability was not afforded to people who do not believe that this particular feature should be supported. This is why I started this thread (which has someone gone off the rails trying to discuss whether or not this is a good feature or not which is immaterial to the actual issue), because I believe that this change should be reverted, and if this is something that Nick, Daniel, Vinay, or anyone else feels is important then it should be proposed as part of Wheel 1.1 and everyone should be given a fair chance to discuss it before it is accepted. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA