<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Mar 24, 2017 at 4:26 AM, Wes Turner <span dir="ltr"><<a href="mailto:wes.turner@gmail.com" target="_blank">wes.turner@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-h5">On Thu, Mar 23, 2017 at 11:59 PM, Nick Coghlan <span dir="ltr"><<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_219496603010817923gmail-">On 24 March 2017 at 13:24, Wes Turner <span dir="ltr"><<a href="mailto:wes.turner@gmail.com" target="_blank">wes.turner@gmail.com</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote"><span class="gmail-m_219496603010817923gmail-"><span class="gmail-m_219496603010817923gmail-m_-5461569875133279154gmail-">On Thu, Mar 23, 2017 at 2:23 AM, Nick Coghlan <span dir="ltr"><<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>></span> wrote:</span><br></span><span class="gmail-m_219496603010817923gmail-"><span class="gmail-m_219496603010817923gmail-m_-5461569875133279154gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>This means we're not going to be automating the process of getting an editable checkout in the core tools any time soon - there are already 100k+ published packages on PyPI, so anyone that seriously wants to do this is going to have to write their own client utility that attempts to infer it from the metadata that already exists (probably by building atop distlib, since that has all the necessary pieces to read the various metadata formats, both remote and local).<br><br></div><div>Future metadata extensions might help to make such a tool more reliable, but *requiring* metadata changes to be made first will just make it useless (since it wouldn't work at all until after publishers start publishing the new metadata, which would mean waiting years before it covered a reasonable percentage of PyPI).<br></div></div></div></div></blockquote><div><br></div></span><div>Here's a way to define Requirements and a RequirementsMap with additional data:</div><div><a href="https://github.com/westurner/pyleset/blob/57140bcef5/setup.py#L118" target="_blank">https://github.com/westurner/p<wbr>yleset/blob/57140bcef5/setup.p<wbr>y#L118</a><br></div><div><br></div><div>It creates a directory full of requirements[.dev].txt files:</div><div><a href="https://github.com/westurner/pyleset/tree/57140bce/requirements" target="_blank">https://github.com/westurner/p<wbr>yleset/tree/57140bce/requireme<wbr>nts</a><br></div><div><br></div><div>Additional metadata in Pipfile would be nice;</div><div>but it would be fairly easy to send a PR to:</div><div><br></div><div>BLD: setup.py: add the canonical sourceURL</div></span></div></div></div></blockquote><div><br></div><div>PEP 426 already has a source URL field: <a href="https://www.python.org/dev/peps/pep-0426/#source-url" target="_blank">https://www.python.org/dev/pep<wbr>s/pep-0426/#source-url</a><br><br></div><div>It's just not required to be a *version* control reference - it's free to be a reference to a tarball or zip archive instead (just not a reference to the sdist itself, since that will contain a copy of the metadata file).<br><br></div><div>However, independently of that concern, "send a PR" is only the first step in updating published metadata to accommodate tasks that package *consumers* want to perform:<br><br></div><div>1. Someone has to write and submit the upstream project patch<br></div><div>2. The publisher has to review and accept the change<br></div><div>3. The publisher has to publish the new release<br></div><div>4. Rinse-and-repeat for dozens/hundreds/thousands of projects, depending on the scope of what you care about</div><br></div></div><div class="gmail_extra">So the lesson we've learned is that for consumer tasks it's *always* better to start by asking "How can I best achieve my objective without asking publishers to change *anything*?".<br><br>In the case of finding version control references, that's a matter of:<br><br></div><div class="gmail_extra">- looking at Download-URL and Project-URL entries for links that "look like" version control references<br></div><div class="gmail_extra">- if that doesn't turn up anything useful, scan the long description<br></div><div class="gmail_extra">- once you have a repository reference, look for promising tag names (if the link didn't nominate a specific commit)<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">On the *publisher* side, the equivalent question is "Can publishers already choose to publish this metadata without having to wait for a metadata update?".<br></div></div></blockquote></div></div></div></div></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br></div><div class="gmail_extra">In this case, the answer is yes, due to the "Project-URL" field: anyone is free to push for the adoption of a particular convention for tagging the exact version control reference needed for "pip -e" to retrieve the corresponding source code.<br></div></div></blockquote></div></div></div></div></div></blockquote><div><br></div><div><a href="https://www.google.com/search?q=python+pep+">https://www.google.com/search?q=python+pep+</a>"project-url" (!)<br></div><div><br></div><div><a href="https://www.python.org/dev/peps/pep-0345/#project-url-multiple-use">https://www.python.org/dev/peps/pep-0345/#project-url-multiple-use</a><br></div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Project-URL (multiple-use)<br>A string containing a browsable URL for the project and a label for it, separated by a comma.<br>Example:<br> Bug Tracker, <a href="http://bitbucket.org/tarek/distribute/issues/">http://bitbucket.org/tarek/distribute/issues/</a><br>The label is a free text limited to 32 signs.</blockquote></div><div> </div><div>- Predicate URIs are often longer than 32 signs.</div><div><br></div><div> (pypi:pkgname, label, URL)</div><div><br></div><div> # RDF triples</div><div> (subject, predicate, object)</div><div> (URI, URI, *)</div><div><br></div><div> # RDF quads</div><div> (graph, s, p, o)</div><div> (URI, URI, URI, *)</div><div><br></div><div><br></div><div>From <a href="http://legacy.python.org/dev/peps/pep-0426/#source-url">http://legacy.python.org/dev/peps/pep-0426/#source-url</a>:<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>For version control references, the VCS+protocol scheme SHOULD be used to identify both the version control system and the secure transport, and a version control system with hash based commit identifiers SHOULD be used. Automated tools MAY omit warnings about missing hashes for version control systems that do not provide hash based commit identifiers.<br>To handle version control systems that do not support including commit or tag references directly in the URL, that information may be appended to the end of the URL using the @<commit-hash> or the @<tag>#<commit-hash> notation.</blockquote><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br></div><div class="gmail_extra">Putting those two together means that anyone that chooses to do so is already free to write a tool that:<br><br></div><div class="gmail_extra">- downloads a PyPI package<br></div><div class="gmail_extra">- looks for a "Editable Install" Project-URL, and uses that if defined<br></div><div class="gmail_extra">- otherwise looks for a promising VCS reference in Download-URL, the Project-URL definitions, and the long description<br></div></div></blockquote></div></div></div></div></div></blockquote><div><br></div><div>> Explicit is better than implicit.<br>> Simple is better than complex.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"></div><div class="gmail_extra">- runs `pip -e` based on whatever it finds</div></div></blockquote></div></div></div></div></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="gmail-h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br></div><div class="gmail_extra">And as long as that tool is itself pip installable, there's no particular reason the feature needs to be built into pip itself.<br></div></div></blockquote><div><br></div></div></div><div>STORY: Users can pull the source code for each installed package (git, [{RPM,} (archive-within-RPM.tar.gz)])</div><div><br></div><div>...</div><div><br></div><div>the npm package.json docs are a pretty good read here:</div><div><br></div><div>- (with {name, description, url} things are already <a href="http://schema.org/Thing" target="_blank">schema.org/Thing</a> s)</div><div>- <a href="https://docs.npmjs.com/files/package.json#bugs" target="_blank">https://docs.npmjs.com/files/<wbr>package.json#bugs</a></div><div>- <a href="https://docs.npmjs.com/files/package.json#repository" target="_blank">https://docs.npmjs.com/<wbr>files/package.json#repository</a></div><div>- <a href="https://docs.npmjs.com/files/package.json#man" target="_blank">https://docs.npmjs.com/<wbr>files/package.json#man</a></div><div><br></div><div>```json</div><div><br></div><div>"bugs": </div><div><div> { "url" : "<a href="https://github.com/owner/project/issues" target="_blank">https://github.com/owner/<wbr>project/issues</a>",</div><div> "email" : "<a href="mailto:project@hostname.com" target="_blank">project@hostname.com</a>"</div><div> }</div></div><div><br></div><div><div>"repository" :</div><div> { "type" : "git"</div><div> , "url" : "<a href="https://github.com/npm/npm.git" target="_blank">https://github.com/npm/npm.<wbr>git</a>"</div><div> }</div><div><br></div><div>"repository" :</div><div> { "type" : "svn"</div><div> , "url" : "<a href="https://v8.googlecode.com/svn/trunk/" target="_blank">https://v8.googlecode.com/<wbr>svn/trunk/</a>"</div><div> }</div></div><div>```</div><span class="gmail-"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"></div><div class="gmail_extra"><br></div><div class="gmail_extra">Cheers,<br></div><span class="gmail-m_219496603010817923gmail-"><div class="gmail_extra">Nick.<br></div><div class="gmail_extra"></div><div class="gmail_extra"><br>-- <br><div class="gmail-m_219496603010817923gmail-m_-5461569875133279154gmail_signature">Nick Coghlan | <a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a> | Brisbane, Australia</div>
</div></span></div>
</blockquote></span></div><br></div></div>
</blockquote></div><br></div></div>