[Distutils] setup_requires for dev environments

[Daniel Holth]

The wheel spec itself is intentionally designed to be ignorant of the
(setuptools) metadata; you don't have to read that to install the
files where they belong. Wheels are also not too hard to generate
without-setuptools. There's a wscript in the wheel source code that
can generate a wheel of bdist_wheel using waf. wwwwwwwwwww. There's
also an old patch that allows the Bento build system to generate
wheels. Of course wheel currently works by converting all the
setuptools static metadata from the .egg-info directory into a
different format in the .dist-info directory, after setuptools is done
running.

bdist_wheel generates PEP 426 data here:
https://bitbucket.org/pypa/wheel/src/bdf053a70200c5857c250c2044a2d91da23db4a9/wheel/metadata.py?at=default#cl-90

All the files setup.py currently dumps into the .egg-info directory
are generated by setuptools plugins. It would be neat to pull the
dist-info generation out of wheel and put it in one of these plugins
for setuptools. Once the plugin was installed, every setuptools
package would automatically get the new file. However IIRC wheel may
have needed one value that's hard to get at this point in the
execution. Alias the egg-info command to dist-info; have it generate a
.dist-info directory; make sure setuptools treats .dist-info about the
same as .egg-info even in a source checkout.

https://bitbucket.org/pypa/setuptools/src/31b56862b41ce24ffe5e28434b98fa35f34d30b4/setuptools/command/egg_info.py?at=default#cl-378

https://bitbucket.org/pypa/setuptools/src/31b56862b41ce24ffe5e28434b98fa35f34d30b4/setup.py?at=default#cl-117

Recall that setup.py is still moderately OK as a build script. It is
legitimate to need software to build software. We just want the
metadata it generates to always be the same, and for it to not also be
the installer.

setup-requires solves a different problem than pydist.json. You should
be able to use setup-requires in a source checkout even if you don't
have a (complete) pydist.json, install the build system; run the
metadata generation phase of your build system to convert some
metadata "in a file format that can have comments" to the json file;
continue.

On Tue, Mar 17, 2015 at 8:33 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 17 March 2015 at 11:49, Donald Stufft <donald at stufft.io> wrote:
>>> I'm still not clear what you expect pip to *do* with the metadata.
>>> It's just data, there's no functionality specified in the PEP.
>>
>> What pip does now with metadata, Look at it for dependency information when
>> installing the Wheel, show it when doing ``pip show``, handle the Provides
>> metadata making something “Provide” something else, show warnings for the
>> obsoleted-by metadata, handle extensions (including failing if there is a
>> critical extension we don’t understand).
>
> Hmm, OK.
>
> At the moment that stuff (except pip show) is all covered by the
> running of the egg_info command, I guess. So you're saying that pip
> should first check if a requirement has new-style metadata and if it
> does, skip the egg_info command and use pydist.json. I guess that
> would be good - it'd solve the problems we see with numpy-related
> packages that need things installed just to run setup.py egg_info.
>
> It wasn't something I'd particularly considered, but thanks for the
> clarification.
>
> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig