pip does not work as expected from a zip
When run from a .zip file, pip does not seem to work as expected - it seems to need to be conventionally installed. I zipped up pip 1.4.1, setuptools (1.1.4) and wheel (0.21.0) into a .pyz. I have pip 1.0 and setuptools 0.6 installed in system site-packages. If I run "pip.pyz --version", it prints "1.0" rather than "1.4.1". If I run "pip.pyz wheel", it tells me that it needs setuptools >= 0.8. The problem seems to be that in at least these two places, pip runs code looking like dist = pkg_resources.get_distribution('XXX') which of course will not find the setuptools or pip in the .pyz. Is this behaviour by design? If so, it seems somewhat sub-optimal. If not, should I create an issue on the pip tracker? Regards, Vinay Sajip
It ought to be able to find the .dist-info or .egg-info directories inside the .pyz? On Thu, Sep 12, 2013 at 10:38 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
When run from a .zip file, pip does not seem to work as expected - it seems to need to be conventionally installed.
I zipped up pip 1.4.1, setuptools (1.1.4) and wheel (0.21.0) into a .pyz. I have pip 1.0 and setuptools 0.6 installed in system site-packages.
If I run "pip.pyz --version", it prints "1.0" rather than "1.4.1". If I run "pip.pyz wheel", it tells me that it needs setuptools >= 0.8.
The problem seems to be that in at least these two places, pip runs code looking like
dist = pkg_resources.get_distribution('XXX')
which of course will not find the setuptools or pip in the .pyz.
Is this behaviour by design? If so, it seems somewhat sub-optimal. If not, should I create an issue on the pip tracker?
Regards,
Vinay Sajip
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Daniel Holth <dholth <at> gmail.com> writes:
It ought to be able to find the .dist-info or .egg-info directories inside the .pyz?
Perhaps, but as I didn't actually install those distributions, no .dist-info or .egg-info were available, and so not present in the .pyz. If we want better uptake of .zip-bundled applications as per PEP 441, ISTM we shouldn't *require* such directories to be placed inside .zip files. Does pyzaa create such directories inside .zip files it makes? Regards, Vinay Sajip
On 12 September 2013 15:38, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
The problem seems to be that in at least these two places, pip runs code looking like
dist = pkg_resources.get_distribution('XXX')
which of course will not find the setuptools or pip in the .pyz.
Is this behaviour by design? If so, it seems somewhat sub-optimal. If not, should I create an issue on the pip tracker?
I'd say yes, raise an issue on the pip tracker. At a minimum, it will ensure that the issue is discussed and the conclusion recorded. Personally, I'd say that zipping up pip should work. Whether it's valid to require people to include dist-info directories in zips, as Daniel says, I'm not sure - maybe it should be a requirement, but if so, then the python docs on how to zip up files to make executable zips should document this (potential) requirement. Without seeing the surrounding code, I can't say whether the use of pkg_resources here is valid. Paul
Paul Moore <p.f.moore <at> gmail.com> writes:
I'd say yes, raise an issue on the pip tracker. At a minimum, it will ensure that the issue is discussed and the conclusion recorded.
Okay, will do.
Personally, I'd say that zipping up pip should work. Whether it's valid to require people to include dist-info directories in zips, as Daniel says, I'm not sure - maybe it should be a requirement, but if so, then the python docs on how to zip up files to make executable zips should document this (potential) requirement.
I don't believe it should be a requirement - wherever possible, Python code shouldn't need to be installed into a site-packages in order to work.
Without seeing the surrounding code, I can't say whether the use of pkg_resources here is valid.
Since in each case here it's just checking versions, I would say that it's not really necessary. These are the only two cases in pip where it's searching using string literals 'setuptools' and 'pip' - as these are actually a dependency and pip itself, it shouldn't really need to look for an installed distribution, since the fact that it's running means that (under normal circumstances) it's using imported code rather than installed code (which is what leads to the errors, of course). Regards, Vinay Sajip
On Thu, Sep 12, 2013 at 12:24 PM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Paul Moore <p.f.moore <at> gmail.com> writes:
I'd say yes, raise an issue on the pip tracker. At a minimum, it will ensure that the issue is discussed and the conclusion recorded.
Okay, will do.
Personally, I'd say that zipping up pip should work. Whether it's valid to require people to include dist-info directories in zips, as Daniel says, I'm not sure - maybe it should be a requirement, but if so, then the python docs on how to zip up files to make executable zips should document this (potential) requirement.
I don't believe it should be a requirement - wherever possible, Python code shouldn't need to be installed into a site-packages in order to work.
Without seeing the surrounding code, I can't say whether the use of pkg_resources here is valid.
Since in each case here it's just checking versions, I would say that it's not really necessary. These are the only two cases in pip where it's searching using string literals 'setuptools' and 'pip' - as these are actually a dependency and pip itself, it shouldn't really need to look for an installed distribution, since the fact that it's running means that (under normal circumstances) it's using imported code rather than installed code (which is what leads to the errors, of course).
Regards,
Vinay Sajip
In wheel's case the idea was to have pip verify that the optional dependency is new enough to work. The part inside pip could be changed to try: import wheel; check(wheel.__version__). IIRC the pip process itself does not directly use anything inside wheel and does not need to "import wheel". However "setup.py bdist_wheel ...", run by pip in a subprocess, will not work unless setuptools can find wheel's plugin declaration. That won't work unless wheel's *.dist-info/entry_points.txt on sys.path. It is true that pip has a greater dependency on "knowing what's installed" than most Python software and therefore needs the -info directories that provide the installation database. Programs that are not installers and do not use entry_points usually do fine with just "try: import x".
Daniel Holth <dholth <at> gmail.com> writes:
In wheel's case the idea was to have pip verify that the optional dependency is new enough to work. The part inside pip could be changed to try: import wheel; check(wheel.__version__). IIRC the pip process itself does not directly use anything inside wheel and does not need to "import wheel".
I see what you're saying, but I was commenting about checks for setuptools and pip. There's no analogous check for wheel, since as you say, pip doesn't use wheel directly.
However "setup.py bdist_wheel ...", run by pip in a subprocess, will not work unless setuptools can find wheel's plugin declaration. That won't work unless wheel's *.dist-info/entry_points.txt on sys.path.
Yes, I see. Perhaps it's a shame it was implemented that way, if it means it's harder or not possible to run pip from a .zip :-( Regards, Vinay Sajip
On Thu, Sep 12, 2013 at 1:51 PM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Daniel Holth <dholth <at> gmail.com> writes:
In wheel's case the idea was to have pip verify that the optional dependency is new enough to work. The part inside pip could be changed to try: import wheel; check(wheel.__version__). IIRC the pip process itself does not directly use anything inside wheel and does not need to "import wheel".
I see what you're saying, but I was commenting about checks for setuptools and pip. There's no analogous check for wheel, since as you say, pip doesn't use wheel directly.
However "setup.py bdist_wheel ...", run by pip in a subprocess, will not work unless setuptools can find wheel's plugin declaration. That won't work unless wheel's *.dist-info/entry_points.txt on sys.path.
Yes, I see. Perhaps it's a shame it was implemented that way, if it means it's harder or not possible to run pip from a .zip :-(
I'm suggesting it might be a bug in pkg_resources, or it might be something pkg_resources can already do, if pip.zip is added to PYTHONPATH while executing setup.py in a subprocess.
On Thu, Sep 12, 2013 at 2:14 PM, Daniel Holth <dholth@gmail.com> wrote:
I'm suggesting it might be a bug in pkg_resources, or it might be something pkg_resources can already do, if pip.zip is added to PYTHONPATH while executing setup.py in a subprocess.
pkg_resources can detect subdirectories in a zip that are named "whatever-version.egg/", and treat them like egg files or directories. I don't recall whether it can detect .egg-info in .zip, though; I don't think I specifically implemented that, so if it works, it's by virtue of a good orthogonal design rather than any specific intent for it to work. ;-) My guess, though, is that the "basket" support (i.e. zipped .egg subdirectories) is the only way at the moment to bundle multiple distributions in a .pyz or other archive. (Note: I don't mean putting an .egg file in the .zip, just zipping up an .egg directory and including it as a subdirectory inside the main .zip/.pyz/whatever.)
participants (4)
-
Daniel Holth -
Paul Moore -
PJ Eby -
Vinay Sajip