<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jun 2, 2017, at 10:41 AM, Nick Coghlan <<a href="mailto:ncoghlan@gmail.com" class="">ncoghlan@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">On 2 June 2017 at 23:42, Thomas Kluyver <<a href="mailto:thomas@kluyver.me.uk" class="">thomas@kluyver.me.uk</a>> wrote:<br class=""><blockquote type="cite" class="">As was suggested at some point, I have added a build_sdist hook to my<br class="">PR, with the following details:<br class=""><br class="">- A brief definition of the minimal requirements of an sdist.<br class=""> - I have limited the definition to gzipped tarballs. Zip files also<br class=""> work as sdists, but we're moving towards standardising on tarballs, so<br class=""> I think it's simplest to require that of PEP-517 compliant tools.<br class=""></blockquote><br class="">For the sdist case, I'd prefer to leave the actual archive creation in<br class="">the hands of the frontend as far as the plugin API is concerned. That<br class="">lets us completely duck the fact that the sdist naming scheme and<br class="">exact archive format aren't formally defined anywhere, and for pip's<br class="">local build use case, we want the unpacked tree anyway.<br class=""><br class="">In a lot of ways, it's closer in spirit to the wheel metadata<br class="">generation hook than it is to the wheel building hook.<br class=""></div></div></blockquote><div><br class=""></div><div><br class=""></div><div>I’d prefer to leave the actual creation of the archive up to the front end in both the sdist and the wheel case. It will allow some cases of pip to avoid having to round trip through a compression -> decompression cycle and can instead just use it directly. Other cases it won’t allow that, but in those cases it doesn’t really add any more time or complexity, it just shifts the time spent compressing from the backend to the frontend.</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class=""><blockquote type="cite" class="">- The build_sdist hook must be defined, but may not always work (e.g. it<br class="">may depend on a VCS)<br class=""></blockquote><br class="">I was going to object to this aspect, but I realised there's a clear<br class="">marker file that frontends can use to determine if they're working<br class="">with an already exported sdist tree: PKG-INFO<br class=""><br class="">That means the invocation protocol for the additional hook can be:<br class=""><br class="">- if PKG-INFO is present, then just copy the full contents of the<br class="">directory without invoking the backend's sdist export hook<br class="">- if PKG-INFO is *not* present, then invoke the backend's sdist export<br class="">hook to do a filtered export that at least omits any VCS bookkeeping<br class="">files<br class=""><br class=""><blockquote type="cite" class="">- The prepare_build_files hook is optional, and in its absence,<br class="">frontends can use build_sdist and extract the files to create a build<br class="">directory.<br class="">- Backends (like flit) where building an sdist has extra requirements<br class="">should define prepare_build_files.<br class=""></blockquote><br class="">Having two hooks still leaves us open to "VCS -> sdist -> build tree<br class="">-> wheel" and "VCS -> build tree -> wheel" giving different answers,<br class="">and that's specifically the loophole we're aiming to close by<br class="">including this in PEP 517 rather than leaving it until later.<br class=""><br class="">Instead, the flow that I think makes sense is "VCS -> sdist tree [-><br class="">sdist tree -> sdist tree -> ...] -> wheel", and the above model where<br class="">the export filtering is only used when PKG-INFO doesn't exist yet will<br class="">give us that.<br class=""><br class=""></div></div></blockquote><br class=""></div><div>So my preference is that everything goes through the sdist step as I think that is most likely to provide consistent builds everywhere both from a VCS checkout and from a sdist that was released to PyPI. That being said, I am somewhat sympathetic to the idea that generating a sdist might be a slow process for reasons that are unrelated to actually building a wheel (for example, documentation might get “compiled” from some kind of source format to a man page, html docs, etc) so I think I am not against the idea of having an optional hook whose job is to just do the copying needed. The requirements would be:</div><div><br class=""></div><div>* The build_sdist hook is mandatory, but may fail (as any of these commands may fail tbh) if some invariant required by the build backend isn’t satisfied.</div><div>* The copy_the_files hook is optional, if it exists it SHOULD produce a tree that when the build_wheel hook is called in it, will produce a wheel that is equivalent to one that would have been built had the build_sdist hook been called instead.</div><div>* If the copy_the_files hook is not defined, then the build frontend is free to just directory call the build_sdist command instead.</div><div><br class=""></div><div>I think that represents a pretty reasonable trade off, the path of least resistance for a build backend is to just define build_sdist and build_wheel and leave the two optional hooks omitted. I suspect for a lot of pure python packages (although Thomas has said not flit) those two hooks will be fast enough that is all they’ll need to implement. However in cases they’re not we provide both the copy_the_files and the wheel_metadata hook to allow short circuiting a possibly more complex build process to provide a better UX to end users. That kinds of goes against my “good intentions don’t matter” statement from before, but I also think that practicality beats purity ;)</div><br class=""><div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class="">—<br class="">Donald Stufft<br class=""></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""></div><br class="Apple-interchange-newline">
</div>
<br class=""></body></html>