[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Robert Collins]

On 13 October 2015 at 06:23, Nathaniel Smith <njs at pobox.com> wrote:
> On Oct 12, 2015 10:16 AM, "Robert Collins" <robertc at robertcollins.net>
> wrote:
...
>> So until we solve the problems related to unloading something loaded
>> into Python and loading a different version in and all the related
>> pain that can occur - I think using Python function calls is a
>> non-starter.
>
> I don't see the contradiction here. If you look at the original draft PEP
> then it exactly specifies that builds get isolated environments, and build
> tools are supposed to spawn a child and then have that child do a function
> call using whatever mechanism they prefer.

Ok, so here's a worked example to let us debug the disconnect.

Given:

A at 1.0: setup-requires: S~=1.0
             install-requires: B
B at 1.0: setup-requires: S~=2.0
S at 1.0: no dependencies at all.
S at 2.0: no dependencies at all.

and no binaries of A or B... then:

  pip install A

will do the following (key bits related to this proposal only, happy path only):

- download the A at 1.0 sdist
- read pypa.yaml and read the setup-requires + build-tool keys
- download S at 1.0 and prepare a built version of it
- place S at 1.0 into PYTHONPATH and the built versions bin into PATH
   - run the build-tool command to determine how to use it
   - run the resulting wheel-build command to build a wheel for A at 1.0
- read the wheel metadata in to determine A's install-requires
- download the B at 1.0 sdist
- read pypa.yaml and read the setup-requires + build-tool keys
- download S at 2.0 and prepare a built version of it
- place S at 2.0 into PYTHONPATH and the built versions bin into PATH
   - run the build-tool command to determine how to use it
   - run the resulting wheel-build command to build a wheel for B at 1.0
- read the wheel metadata in to determine B's install-requires
- install the B wheel into the target environment
- install the A wheel into the target environment

Note the two places where PYTHONPATH and PATH need to be something
other than the environment of pip itself. pip may not be installed in
the target environment (or may be a different version than the pip
installed there). I don't understand how you propose to have S at 1.0 and
S at 2.0 co-exist in pip's Python process.

-Rob
-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud