[Distutils] Current Python packaging status (from my point of view)

[Chris Barker]

On Wed, Nov 2, 2016 at 7:32 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> > He mentioned that conda allows you to
> > manage the python run-time itself, which is in deed a nice feature, but
> > getting a python run-time as never been the hard part (maybe on Linux if
> you
> > want a different one than your system supplies).
>
> I didn't miss it accidentally, I left it out because it wasn't
> relevant to the post (which was about the ecosystem design direction,
>

I would argue that is is quite relevant -- talking about design decisions
without talking about the motivations and consequences of those decisions
is missing much of the point.

The issue here is that folks might read that post and think: "do I want to
manage my python install or not?" and think that's the only question they
need to ask to determine if pip or conda is right for them. But it's not at
all.

pip is about managing stuff WITHIN python -- that's why it can work with
any conforming python install. So that's the advantage of this design
decision. But it leads to a major limitation, EVEN if you only care about
python, because it can't (properly) manage the stuff outside of python that
python packages may need. I honestly have no idea if the original
motivation was specifically to have a system that could work with any
python install (maybe), but it certainly was designed specifically to
handle python packages, period.

conda started with the motivation of managing complex non-python
dependencies (initially, to support python) -- in order to do that
effectively, it has to operate outside the  python envelope, and and that
means that it really needs to manage python itself. I'm pretty sure that
managing python itself was a consequence of the design goals, not a primary
design goal.

not the end user features that make the desire to use pip
> incomprehensible to a lot of folks).
>
> Designing software assembly tools for interpreted software is a system
> integrator's game, and when people are writing Python code, there is
> one absolutely 100% unavoidable integration task: choosing which
> Python runtimes they're going to support.
>

hmm -- I don't think that's the code-writers job -- it's the deployers job.
Other than choosing which python *version* I want to use, I can happily
develop with system python and pip, and then deploy with conda -- or vice
versa. INdeed, I can develop on Windows and deploy on LInux, or....

though if you meant pypy vs iron python vs cPython when you meant "runtime"
then yes, with the dependency issue, you really do need to make that choice
upfront.


>  conda requires that you use a
> runtime that it provides (similar to the way Linux distro packaging
> works), while pip really doesn't care where the runtime came from, as
> long as it can run pip properly.
>

yes indeed -- and I'm fully aware of the consequences of that -- I work in
a very locked down system -- our system security folks REALLY want us to
use system-supplied packages (i.e. the python supplied by the OS
distributor) where possible. But that doesn't mean that I should try to
manage my applications with pip -- because while that makes the python part
easier, it makes the dependencies a nightmare -- having to custom compile
all sorts of stuff -- much more work, and not any more satisfying to the
security folks.

So you really need to look at your whole system to determine what will best
work for your use case.


> Essentially all the other differences between the two systems stem
> from the different choice made in that core design decision, which is
> why the two aren't substitutes for each other and never will be - they
> solve different problems for different audiences.
>

Indeed they do.

But the different audiences aren't "data science folks" vs "web developers"
-- the different audiences are determined by deployment needs, not domain.

conda environments are kind like mini docker containers -- they really  can
make life easier for lots of use cases, if you can get your IT folks to
accept a "non standard" python.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20161102/89fd499c/attachment.html>