[Distutils] What role to eggs still play?

[Leonardo Rochael Almeida]

On 24 August 2016 at 12:56, Daniel Holth <dholth at gmail.com> wrote:

> On Wed, Aug 24, 2016 at 10:41 AM Alex Grönholm <alex.gronholm at nextday.fi>
> wrote:
>
>> [...]
>>
>
> 19.08.2016, 20:25, Daniel Holth kirjoitti:
>
> Eggs are the only way to add a zipped distribution to PYTHONPATH and have
> setuptools find the metadata (the Python code can be found with or without
> the metadata; setuptools does not discover *.dist-info inside zip). Eggs
> are used by buildout, especially in the unzipped into a directory form. And
> they could still be used for their originally designed use as a plugin
> format.
>
> Alex kirjoitti: "What is not clear to me is who needs this. Is this not
> what virtualenvs are for? Or could wheels not be used for the same purpose?
> What am I missing?"
>

There is some mixing of concerns here, but to clarify: wheels in general
CANNOT be blindly dropped on PYTHONPATH and be expected to work.

Only wheels that DO NOT contain an entry like `wheel_name-1.0.data/purelib`
or `wheel_name-1.0.data/platlib` can be dropped on PYTHONPATH to make their
code available to a python process.

But that only works for python code. `setuptools`/`pkg_resources` will
currently be unable to read entry points (in particular, console_scripts)
from such wheels dropped in the PYTHONPATH.

So, they cannot be used in general for the purpose of dropping them on the
PYTHONPATH. But the buildout communities (incl. Plone) do not exactly
require wheels dropped on the PYTHONPATH. See belowÇ

What you are missing is that while they could in theory be used for the
> same purpose, in practice there is a large group of Plone & buildout users
> who are not using wheels for that. (And the wheels would have to be
> unzipped into their own individual directories, but buildout almost always
> unzips eggs lately too.)
>

In particular, buildout users on Windows, who cannot easily compile sdists
that contain C extensions and which are currently uploaded as eggs to PyPI
(see lxml and zope.interface as examples of highly used packages in this
regard).

As for wheels, not only would these wheels have to be unzipped, but their
`purelib`/`platlib` contents, if present, copied to the "root" of where
they were unpacked, and the `wheel_name-1.0.dist-info` directory renamed to
`EGG-INFO`

But the most important thing of all, these wheels would have to participate
in the dependency resolution of other packages along with sdists in the
middle of a buildout run.

The last aspect means that having a canonical piece of code that knows how
to pick up a wheel from PyPI and unpack it correctly somewhere in the
PYTHONPATH is not enough.

(and nobody has offered such canonical a piece of software yet, except the
not really useful "call out to pip as a subprocess").

What is needed is for setuptools to be modified to treat wheels the same
way it currently treats bdist_wininst files for installation: as a binary
lump that it knows how to download from PyPI, unpack and convert into an
egg.

On the other hand, if the above happens, we can block .egg files from
upload to PyPI along with the rest of the formats. Wouldn't it be nice?

Any other option for adding wheels support to buildout (calling out to pip,
using distlib,... ) will involve either a lot of code rewrite [1] or the
need to rewrite many currently functioning buildout configurations [2].

[1] to make the `zc.recipe.egg` part of buildout behave exactly like it was
still using setuptools, and pkg_resources would still need to be taught how
to read .dist-info directories.
[2] to use something else than `zc.recipe.egg` to install Python packages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160824/efb58a94/attachment-0001.html>