<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Aug 18, 2016 at 4:54 PM, Chris Barker <span dir="ltr"><<a href="mailto:chris.barker@noaa.gov" target="_blank">chris.barker@noaa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Wed, Aug 17, 2016 at 6:45 PM, Daniel Holth <span dir="ltr"><<a href="mailto:dholth@gmail.com" target="_blank">dholth@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">And a while back I argued against setuptools-lite, because I thought it would not solve the poor extensibility problem that stems from its basic distutils derived design... which includes all the classes and subclasses that have to work together to do its thing. My own thinking is that setuptools is fine, except that when you need to extend it you have a big problem; so make it easy to replace.</div></blockquote><div><br></div></span><div>I never had any fantasies that setuptool-lite would improve on this in a way shape or form -- indeed, a better build system is likely to need to be built from scratch, or on a totally different framework, anyway.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Does pip monkey patch setuptools? I thought it just forced setuptools even if a package only asked for distutils.</div></div></blockquote><div><br></div></span><div>good point -- not really a monkey patch, but it does force particular configuration.</div><span class=""><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>So you are suggesting setuptools_lite would not honor the setup_requires = [] and test_requires = [] </div></div></blockquote><div><br></div></span><div>essentially, yes. Though It would be nice if that data could be available for pip -- and I"m not sure how that is done at this point.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>and perhaps "setup.py install" would also throw an error? </div></div></blockquote><div><br></div></span><div>Probably not -- but it could only install from local source -- like distutils. Though if we could re-work it to only do develop mode and build wheels, that may be good -- you'd need pip to install it.</div><span class=""><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>What else? I would suggest that to effectively argue the idea requires reading the source code and enumerating exactly which complexity goes away with the _lite version.</div></div></blockquote><div><br></div></span><div>I don't know that reading the source is the issue -- I'm thinking entirely in terms of stripping out the API, so there may be things that need to stay in, but we don't expose them. though maybe even that's harder than I think.</div><div><br></div><div>But yes -- enumerating what would and wouldn't be supported would be a good start. But here is the guideline:</div><div><br></div><div>It seems we have a (maybe vague) vision of what we want the tool ecosystem to look like that involves separation of concerns, such that each component is replaceable without having to change the others:</div><div><br></div><div>The concerns:</div><div> - building a package</div><div> - installing a package</div><div> - dependency management</div><div> - run-time version management</div><div> - plugin handling, etc.<br></div><div> - resource managment</div><div><br></div><div>setuptools does all of these, yes? but think of these in terms of when they come into play:</div><div><br></div><div>build time: </div><div> - building a package</div></div></div></div></blockquote><div><br></div><div>- building c extensions</div><div>- building NumPy (fortran,)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>install time: installing a package;</div><div><div> - installing a package</div><div> - dependency management </div></div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>run time </div><div><div> - run-time version management</div><div> - plugin handling, etc.<br></div></div></div></div></div></blockquote><div><br></div><div>- running tests</div><div> - (It's much faster/easier/consistent/reproducible to run tests when all of the dependencies are bundled into one executable ZIP (e.g. PEX))</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div></div></div><div><br></div><div>so we have:</div><div><br></div><div> - The mythical build tool</div></div></div></div></blockquote><div><br></div><div>- TBH, I don't know much about pyproject.toml</div><div> - "PEP 518 -- Specifying Minimum Build System Requirements for Python Projects"</div><div> <a href="https://www.python.org/dev/peps/pep-0518/">https://www.python.org/dev/peps/pep-0518/</a></div><div><br></div><div>And then there are a number of build tools which work with multiple languages (because just solving for Python doesn't get it):</div><div><br></div><div>- Conda (Python)</div><div> - meta.yaml, build.sh, build.bat *</div><div> - <a href="http://conda.pydata.org/docs/build_tutorials.html">http://conda.pydata.org/docs/build_tutorials.html</a></div><div> - <a href="http://conda.pydata.org/docs/build_tutorials/pkgs.html#conda-build-skeleton">http://conda.pydata.org/docs/build_tutorials/pkgs.html#conda-build-skeleton</a> (pypkg -> conda)<br></div><div> - <a href="http://conda.pydata.org/docs/building/meta-yaml.html#build-section">http://conda.pydata.org/docs/building/meta-yaml.html#build-section</a></div><div> - <a href="http://conda.pydata.org/docs/bdist_conda.html">http://conda.pydata.org/docs/bdist_conda.html</a> (condapkg -> pypkg)</div><div><br></div><div>- Blaze (Java, Python?; Google) -> Open Sourced as Bazel</div><div><br></div><div>- Pants (Python; Twitter):</div><div> - BUILD files</div><div> - <a href="https://pantsbuild.github.io/python_readme.html">https://pantsbuild.github.io/python_readme.html</a></div><div> - <a href="https://pantsbuild.github.io/3rdparty_py.html">https://pantsbuild.github.io/3rdparty_py.html</a> (pip requirements.txt)</div><div> - <a href="https://pantsbuild.github.io/scala.html">https://pantsbuild.github.io/scala.html</a></div><div> - <a href="https://pantsbuild.github.io/go_readme.html">https://pantsbuild.github.io/go_readme.html</a></div><div><br></div><div>- Buck (Java, Facebook):</div><div> - <a href="https://buckbuild.com/rule/prebuilt_python_library.html">https://buckbuild.com/rule/prebuilt_python_library.html</a> (.egg)</div><div> - <a href="https://buckbuild.com/rule/python_binary.html">https://buckbuild.com/rule/python_binary.html</a></div><div> - <a href="https://buckbuild.com/rule/python_library.html">https://buckbuild.com/rule/python_library.html</a></div><div> - <a href="https://buckbuild.com/rule/python_test.html">https://buckbuild.com/rule/python_test.html</a></div><div> - <a href="https://buckbuild.com/rule/cxx_library.html">https://buckbuild.com/rule/cxx_library.html</a></div><div> - <a href="https://buckbuild.com/rule/go_library.html">https://buckbuild.com/rule/go_library.html</a></div><div> - <a href="https://buckbuild.com/rule/rust_library.html">https://buckbuild.com/rule/rust_library.html</a></div><div><br></div><div>- Bazel (Java, Python; Google)</div><div> - <a href="http://bazel.io/docs/be/python.html">http://bazel.io/docs/be/python.html</a></div><div> - <a href="http://bazel.io/docs/be/python.html#py_binary">http://bazel.io/docs/be/python.html#py_binary</a></div><div> - <a href="http://bazel.io/docs/be/python.html#py_library">http://bazel.io/docs/be/python.html#py_library</a></div><div> - <a href="http://bazel.io/docs/be/python.html#py_test">http://bazel.io/docs/be/python.html#py_test</a></div><div> - <a href="https://bazel.io/docs/be/c-cpp.html">https://bazel.io/docs/be/c-cpp.html</a></div><div> - <a href="https://github.com/bazelbuild/rules_go">https://github.com/bazelbuild/rules_go</a></div><div><br></div><div><br></div><div>... PEX is based on PEP 441:</div><div><br></div><div>- "PEP 441 -- Improving Python ZIP Application Support"</div><div> <a href="https://www.python.org/dev/peps/pep-0441/">https://www.python.org/dev/peps/pep-0441/</a></div><div>- <a href="https://github.com/pantsbuild/pex/blob/master/docs/whatispex.rst">https://github.com/pantsbuild/pex/blob/master/docs/whatispex.rst</a></div><div><br></div><div>- Again, wheel includes a file manifest in the .whl .zip with SHA256 checksums:</div><div> <a href="https://bitbucket.org/pypa/wheel/src/cf4e2d98ecb1/wscript#wscript-69">https://bitbucket.org/pypa/wheel/src/cf4e2d98ecb1/wscript#wscript-69</a></div><div><br></div><div> - AFAIU, if you specify different install paths, there's no way to do something like **debsums** / **rpm -V**</div><div> (which fpm handles well)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> - pip</div><div> - maybe some of the setuptools spun off -- like pkg_resources</div><div><br></div><div>So why aren't we there yet?</div><div><br></div><div>One reason is that setuptools the only thing out there that does the building and some of the run-time stuff. So people use it. And now you can't replace any of it without making a mess.</div><div><br></div><div>However, pip_wheel have indeed addressed much of the install-time and dependency management issues. So folks use them, and they work well.</div><div><br></div><div>So what problem do I want setuptool-lite to solve?</div><div><br></div><div>I actually have spent a lot of time using conda to do my package installing and dependency management. So I want a way to build a python package that JUST builds the package.</div><div><br></div><div>In practice, I've found (most) of the flags and work arounds to diable etuptools "features", but really?</div><div><br></div><div>p<font face="monospace, monospace">ython setup.py install --single-version-externally-<wbr>managed<br></font><br></div><div>Oh right, that's not enough, I need a record file:<font face="monospace, monospace"><br></font></div><div><br></div><div>setup.py install --single-version-externally-<wbr>managed --record record.txt<br></div><div><font face="monospace, monospace"><br></font>Then I want to install in develop mode:</div><div><br></div><div>python setup.py develop</div><div><br></div><div>OOPS! now it's easy installing a bunch of dependencies! arrgg!</div><div><br></div><div>OK, -- no-deps it is.</div><div><br></div><div>Google around, this kind of thing is a pain in a lot of people's butts --even though it call all be worked around.</div><div><br></div><div>So my vision of setuptool-lite is that is simply does not do the things that vision of a build tool should not do. </div><div><br></div><div>i.e.:</div><div><br></div><div>never easy-install anything!</div><div>don't try to resolve dependencies</div><div><br></div><div>others??</div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>One "lite" version of setuptools that I personally think would be cool would be pkg_resources as a separate package on pypi. Setuptools would require it as a dependency.</div></div></blockquote><div><br></div></span><div>Yes, I'd love to disable pkg_resources too, but couldn't until it was spun off.</div><div><br></div><div>In fact, years ago, long before pip, setuptools drove me crazy with how it combined run-time, build time and intall time stuff. A serious problem when I wanted to bundle something up with py2app (or py2exe, or...).</div><div><br></div><div>So what would be accomplished?</div><div><br></div><div>package builders could simply put:</div><div><br></div><div>from setuptools_lite import setup</div><div><br></div><div>(etc.)</div><div><br></div><div>and then their user would never accidentally easy install anything, etc. And it would serve as documentation / notification if a user was using a what-should-be-deprecated feature.</div><div><br></div><div>By the way, this is totally acknowledged to be a hacky, temporary mess that might only help a bit to get us closer from being locked into setuptools.</div><div><br></div><div>(though maybe it would be a place to work through some of the issues with a build API)</div><div><br></div><div>-CHB</div><div><br></div><div><br></div></div><span class="">-- <br><div data-smartmail="gmail_signature"><br>Christopher Barker, Ph.D.<br>Oceanographer<br><br>Emergency Response Division<br>NOAA/NOS/OR&R <a href="tel:%28206%29%20526-6959" value="+12065266959" target="_blank">(206) 526-6959</a> voice<br>7600 Sand Point Way NE <a href="tel:%28206%29%20526-6329" value="+12065266329" target="_blank">(206) 526-6329</a> fax<br>Seattle, WA 98115 <a href="tel:%28206%29%20526-6317" value="+12065266317" target="_blank">(206) 526-6317</a> main reception<br><br><a href="mailto:Chris.Barker@noaa.gov" target="_blank">Chris.Barker@noaa.gov</a></div>
</span></div></div>
<br>______________________________<wbr>_________________<br>
Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org">Distutils-SIG@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/distutils-sig" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/distutils-sig</a><br>
<br></blockquote></div><br></div></div>