[Distutils] Role of setuptools and eggs in "modern" distributing...

[Chris Barker]

On Wed, Dec 31, 2014 at 9:10 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> The problem always existed - it's the longstanding conflict between
> "platform independent, language specific" tooling and "platform specific,
> language independent" tooling.
>
> The former is often preferred on the developer side (since the tools are
> oriented towards building individual custom applications rather than
> maintaining a suite of applications written by different groups), while the
> latter is essential on the system integrator side (since you're dealing
> with inevitable technology sprawl in infrastructure that evolves over the
> course of years and decades).
>
> One of the best things coming out of the whole "DevOps" notion is the
> recognition that language independence and platform independence are aimed
> at solving *different problems*, and that what we need is suitable tools
> with different roles and responsibilities that interoperate effectively,
> rather than attempting to provide a single universal tool and having
> different groups of users yelling at each other for wanting to solve the
> "wrong" problem.
>
> Tools like conda and zc.buildout fit into that landscape by aiming to
> provide a platform & language independent approach to doing *application*
> level integration, which tends to have the effect of making them new
> platforms in their own right.
>
Indeed -- thanks for providing a clear way to talk/think about these
systems.

I guess the trick here is that we want the different level tools to work
well with each-other.

As conda started with python packages in mind, it does a pretty good job
with them. But I still find some conflicts between setuptools and conda --
in particular, if you specify dependencies in setup.py (install_requires),
it can kind of make a mess of things. conda tries to ignore them, but
somehow I've had issues, even though I had specified it all in the conda's
meta.yaml. This is probably a conda bug/issue, but I'm still trying to
figure out how to best set up a python package so that it can be built
installed with the "regular" python tools, and also conda...

Use case -- developing in a conda environment -- so I want to install
dependencies with conda, but the package under development with setuptools
"develop" mode. (conda does not (yet) have a develop mode that works...)

Oh, and there does seem to be an odd (I think non-fatal) issue with
setuptools and conda:

I have package A, with a bunch of stuff listed with "install_requires"

I have all these dependencies installed with conda.

When I run setup.py develop, setuptools goes and dumps all the dependencies
in easy_install.pth.

I have no idea why that is, and it's probably only annoying, rather than a
problem, but I'm not sure what will happen when I upgrade one of those
dependencies with conda.

> If you compare them to Linux distributions, then zc.buildout is a platform
> that practices the Gentoo style approach of building everything from source
> for each application, while conda uses the more common Debian/Fedora style
> of defining a binary packaging format that allows a redistributor to handle
> the build process on behalf of their users.
>
indeed -- and conda actually provides (to my disappointment) very little in
the way of build support -- you need to write platform dependent build
scripts to actually build the packages.

But it does provide a nice way for me to provide a full "just install this"
distribution of my complex, ugly, hard to build packages....

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20141231/dfe6d947/attachment.html>