<div dir="ltr">OK,<div><br></div><div>I started this thread a while back, as I was getting confused and having issues with intermixing python, setuptools, pip, and Anaconda / conda.</div><div><br></div><div>Now I've figured out where i have my issue:</div><div><br></div><div>I'm using an Anaconda distribution at the moment. I want conda to handle installing my dependencies, etc. for me. OK.</div><div><br></div><div>However, I am also developing various python packages -- this means I can't / don't want o build and install a conda package for them, rather, I'd like to use setuptools "develop" mode.</div><div><br></div><div>So here's the rub:</div><div><br></div><div>When I call "setup.py develop", setuptools apparently looks for the "install_requires" packages. If it doesn't find them, it goes out and decided to apparently pip install them: gets the source form pypi, download, tries to compile, etc....</div><div><br></div><div>Even if it does find them already installed, it does some annoying adding to easy_install.pth magic for them.</div><div><br></div><div>This is all why I've been thinking that dependencies do not belong in setup.py -- but rather outside of setup.py (requirements.txt), and more to the pint, dependency handling ius a pip (or conda, or...) issue - not one that should be handled by aw setuptools at build time.</div><div><br></div><div>Note that conda build usually simply calls setup.py install as well, so this can be a problem even there (though I think it usually satisfies the requirements first, so not so bad)</div><div><br></div><div>At the end of the day, however, I think the issue is not so much where you specify dependencies, but what setuptool develop mode is doing: it should NOT go an auto-install dependencies, particularly not install-dependencies (maybe build dependencies are required...)</div><div><br></div><div>OK -- I just found the --no-deps option. So I can do what I want, but still, I don't think it belongs there and all, and certainly would be better to have the default be no-deps. Let pip (or conda, or...) handle that.</div><div><br></div><div>Any one running these by hand are be definition doing things by hand, let them deal with the deps.<br></div><div><br></div><div>OK, I suppose "casual" users may run setup.py install, so that mode _might_ want to auto install dependencies, if somethign has to. But develop mode is very much for developers, you really don't want this handled there.</div><div><br></div><div>-Chris</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 31, 2014 at 9:41 AM, Chris Barker <span dir="ltr"><<a href="mailto:chris.barker@noaa.gov" target="_blank">chris.barker@noaa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Wed, Dec 31, 2014 at 9:10 AM, Nick Coghlan <span dir="ltr"><<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><span><p dir="ltr">The problem always existed - it's the longstanding conflict between "platform independent, language specific" tooling and "platform specific, language independent" tooling.<br></p></span>
<p dir="ltr">The former is often preferred on the developer side (since the tools are oriented towards building individual custom applications rather than maintaining a suite of applications written by different groups), while the latter is essential on the system integrator side (since you're dealing with inevitable technology sprawl in infrastructure that evolves over the course of years and decades).</p>
<p dir="ltr">One of the best things coming out of the whole "DevOps" notion is the recognition that language independence and platform independence are aimed at solving *different problems*, and that what we need is suitable tools with different roles and responsibilities that interoperate effectively, rather than attempting to provide a single universal tool and having different groups of users yelling at each other for wanting to solve the "wrong" problem.</p>
<p dir="ltr">Tools like conda and zc.buildout fit into that landscape by aiming to provide a platform & language independent approach to doing *application* level integration, which tends to have the effect of making them new platforms in their own right.</p></div></blockquote></span><div>Indeed -- thanks for providing a clear way to talk/think about these systems.</div><div><br></div><div>I guess the trick here is that we want the different level tools to work well with each-other.</div><div><br></div><div>As conda started with python packages in mind, it does a pretty good job with them. But I still find some conflicts between setuptools and conda -- in particular, if you specify dependencies in setup.py (install_requires), it can kind of make a mess of things. conda tries to ignore them, but somehow I've had issues, even though I had specified it all in the conda's meta.yaml. This is probably a conda bug/issue, but I'm still trying to figure out how to best set up a python package so that it can be built installed with the "regular" python tools, and also conda...</div><div><br></div><div>Use case -- developing in a conda environment -- so I want to install dependencies with conda, but the package under development with setuptools "develop" mode. (conda does not (yet) have a develop mode that works...)</div><div><br></div><div>Oh, and there does seem to be an odd (I think non-fatal) issue with setuptools and conda:</div><div><br></div><div>I have package A, with a bunch of stuff listed with "install_requires"</div><div><br></div><div>I have all these dependencies installed with conda.</div><div><br></div><div>When I run setup.py develop, setuptools goes and dumps all the dependencies in easy_install.pth.</div><div><br></div><div>I have no idea why that is, and it's probably only annoying, rather than a problem, but I'm not sure what will happen when I upgrade one of those dependencies with conda.</div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><p dir="ltr">If you compare them to Linux distributions, then zc.buildout is a platform that practices the Gentoo style approach of building everything from source for each application, while conda uses the more common Debian/Fedora style of defining a binary packaging format that allows a redistributor to handle the build process on behalf of their users.</p></div></blockquote></span><div>indeed -- and conda actually provides (to my disappointment) very little in the way of build support -- you need to write platform dependent build scripts to actually build the packages.</div><div><br></div><div>But it does provide a nice way for me to provide a full "just install this" distribution of my complex, ugly, hard to build packages....</div><span class="HOEnZb"><font color="#888888"><div> <br></div><div>-Chris</div><div><br></div></font></span></div><span class=""><div><br></div>-- <br><div><br>Christopher Barker, Ph.D.<br>Oceanographer<br><br>Emergency Response Division<br>NOAA/NOS/OR&R <a href="tel:%28206%29%20526-6959" value="+12065266959" target="_blank">(206) 526-6959</a> voice<br>7600 Sand Point Way NE <a href="tel:%28206%29%20526-6329" value="+12065266329" target="_blank">(206) 526-6329</a> fax<br>Seattle, WA 98115 <a href="tel:%28206%29%20526-6317" value="+12065266317" target="_blank">(206) 526-6317</a> main reception<br><br><a href="mailto:Chris.Barker@noaa.gov" target="_blank">Chris.Barker@noaa.gov</a></div>
</span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><br>Christopher Barker, Ph.D.<br>Oceanographer<br><br>Emergency Response Division<br>NOAA/NOS/OR&R (206) 526-6959 voice<br>7600 Sand Point Way NE (206) 526-6329 fax<br>Seattle, WA 98115 (206) 526-6317 main reception<br><br><a href="mailto:Chris.Barker@noaa.gov" target="_blank">Chris.Barker@noaa.gov</a></div>
</div>