[Distutils] add "sourceURL" to the metadata 3.0 PEP.

[Nick Coghlan]

On 24 March 2017 at 13:24, Wes Turner <wes.turner at gmail.com> wrote:

>
> On Thu, Mar 23, 2017 at 2:23 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> This means we're not going to be automating the process of getting an
>> editable checkout in the core tools any time soon - there are already 100k+
>> published packages on PyPI, so anyone that seriously wants to do this is
>> going to have to write their own client utility that attempts to infer it
>> from the metadata that already exists (probably by building atop distlib,
>> since that has all the necessary pieces to read the various metadata
>> formats, both remote and local).
>>
>> Future metadata extensions might help to make such a tool more reliable,
>> but *requiring* metadata changes to be made first will just make it useless
>> (since it wouldn't work at all until after publishers start publishing the
>> new metadata, which would mean waiting years before it covered a reasonable
>> percentage of PyPI).
>>
>
> Here's a way to define Requirements and a RequirementsMap with additional
> data:
> https://github.com/westurner/pyleset/blob/57140bcef5/setup.py#L118
>
> It creates a directory full of requirements[.dev].txt files:
> https://github.com/westurner/pyleset/tree/57140bce/requirements
>
> Additional metadata in Pipfile would be nice;
> but it would be fairly easy to send a PR to:
>
> BLD: setup.py: add the canonical sourceURL
>

PEP 426 already has a source URL field:
https://www.python.org/dev/peps/pep-0426/#source-url

It's just not required to be a *version* control reference - it's free to
be a reference to a tarball or zip archive instead (just not a reference to
the sdist itself, since that will contain a copy of the metadata file).

However, independently of that concern, "send a PR" is only the first step
in updating published metadata to accommodate tasks that package
*consumers* want to perform:

1. Someone has to write and submit the upstream project patch
2. The publisher has to review and accept the change
3. The publisher has to publish the new release
4. Rinse-and-repeat for dozens/hundreds/thousands of projects, depending on
the scope of what you care about

So the lesson we've learned is that for consumer tasks it's *always* better
to start by asking "How can I best achieve my objective without asking
publishers to change *anything*?".

In the case of finding version control references, that's a matter of:

- looking at Download-URL and Project-URL entries for links that "look
like" version control references
- if that doesn't turn up anything useful, scan the long description
- once you have a repository reference, look for promising tag names (if
the link didn't nominate a specific commit)

On the *publisher* side, the equivalent question is "Can publishers already
choose to publish this metadata without having to wait for a metadata
update?".

In this case, the answer is yes, due to the "Project-URL" field: anyone is
free to push for the adoption of a particular convention for tagging the
exact version control reference needed for "pip -e" to retrieve the
corresponding source code.

Putting those two together means that anyone that chooses to do so is
already free to write a tool that:

- downloads a PyPI package
- looks for a "Editable Install" Project-URL, and uses that if defined
- otherwise looks for a promising VCS reference in Download-URL, the
Project-URL definitions, and the long description
- runs `pip -e` based on whatever it finds

And as long as that tool is itself pip installable, there's no particular
reason the feature needs to be built into pip itself.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170324/a34e9030/attachment.html>