[Distutils] Comments on PEP 426

[Chris Barker - NOAA Federal]

someone wrote:
>  even though plain distutils would work just fine for a very
> large subset of packages.  Most of my own packages
> only use setuptools for its dependency resolution, and
> that's for the most part provided by pip in the future.


Another comment from the peanut gallery: This is very, very true -- I know
nothing of distutils internal design or extensibility, but from the user
end, it is really, really nice!

It's actually quite remarkable that it "just works" most of the time --
even building universal binaries on OS-X, and needing very little platform
dependent code in a setup.py.

So what problems are we really trying to solve?

For my part: I use setuptools for "develop" mode. That's it. Really.

Then there is pip and easy_install -- nice to have the auto-discovery,
download and dependency tracking -- though all of that is actually pretty
over-rated -- it makes it faster to install a complex, dependency-riddled
package (Like a web framework), but it breaks often enough that I end up
doing it by hand anyway. For me, "pip install" really only works well for
single, pure-python packages. (OK, some with external-dependency-free
extensions, but only because I compile my own stuff so have set up
the environment already -- and it's distutils that's working well there...)

On the other hand, setuptools introduced a lot of cruft that seems to get
in the way more than anything else.

- Package versioning is a great idea and _should_ be a standard part of
python, but the community seems to have settled on virtualenv as the
solution to that. And the setuptools solution never seemed to work for me.

- eggs: what's the point? and why mix a distribution format with an
installation, and binary with source, and zipped with unzipped?

- In general, setuptools seems to mix what should be install-time stuff
with run-time stuff -- makes it very hard to use things like py2exe,
py2app, etc. (pkg_resources is a mess, still not sure why it's needed)

OK: so what are problems that really do need to be solved?

Standard binaries that work with virtualenv -- as virtualenv seems to be
the solution to versioning, we really need to be able to either download
and install a binary in a virtualenv, or have pip do it for us -- wheels
and the proper naming schemes seems to be going in the right direction for
this.

The big one: ALL of the problems I have with building compiled stuff is due
to non-python external dependencies. Someone on the thread recently laid
out all the ugly issues with different compilers, etc on OS-X, Windows,
etc, etc. Where those all cause problems is the non-python dependencies:
libpng, libfreetype, libnetcdf, libhdf, etc, etc, etc. That's where it gets
ugly, and AFAIK, no one is trying to build a python solution (the only one
I know of is gattai, which I've started using and hacking on, l but it
really is just away to standardise your build scripts). Granted, I don't
know that it's possible or desirable to have a python solution for that --
that's what apt-get, rpm, homebrew, macports, etc, are for, but that's
where the real hang-ups are. Binary wheels _may_ help that a lot -- so very
few folks actually need to deal with it.

Sorry to sound critical here, but I'm hoping that with all this work, and
failed startes (distutils2) we won't just end up in the same
over-engineered crufty place we've been.

All that said -- I'm looking forward to wheels and a pip that understands
them -- IIUC, we have that, so it's on to the social part -- getting people
to test/use them!

I guess this reads like a rant, which is not really my intent -- it's great
to see all the work getting done, and people thinking hard about good
solutions. I really do appreciate all that effort.

-Chris


Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130906/2d897850/attachment.html>