[Distutils] moving things forward (was: wheel including files it shouldn't)
Chris Barker
chris.barker at noaa.gov
Fri May 6 11:54:10 EDT 2016
On Thu, May 5, 2016 at 4:32 PM, Nathaniel Smith <njs at pobox.com> wrote:
> > You do know that we're on our way to re-writing conda, now, don't you?
> :-)
> >
> > I think we need to be careful of scope-creep...
>
> conda did not invent the idea of creating separate python environments
> for different tasks :-)
I'm not suggesting conda invented anything -- I'm suggesting it has
implemented many of the things being talked about here -- the truth is
conda was designed to solve exactly the problems that scientific python
packages had that pip+wheel+setuptools do not currently solve.
So my point is about scope-creep -- if you want the PyPa tools to solve all
these problems, then you are re-inventing conda -- better to simply adopt
conda (or fork it and fix issues that I'm sure are there....)
On the other hand, improving the PyPa tools while maintaining their current
scope is a lovely idea -- but that means leaving isolation of build
environments etc, to external tools: Docker, VMs, conda......
Actually, conda has a good lesson here: conda is about packaging, NOT
building -- I was quite disappointed that conda provided NO support for
cross platform building at all -- but after using it a bit I realized that
that was a great decision -- if you want to support a wide variety of
packages, you really need to let the package authors use whatever the heck
build system they want -- you really don't want to have to implement (or
even debug) everyone else's builds.
And IIUC, that is the direction we are trying to go with pip now -- making
pip+wheel build-system independent -- good goal.
Which actually give me an idea: it seems we are very bogged down in
struggles with keeping backward compatibility, and coming up with a API for
an arbitrary build system. Maybe we can take a lesson from conda and
essentially punt:
conda works by reading a meta.yaml -- a purely declarative package
configuration. The actual building is done by calling a build script --
conda itself does not need to know or care what the heck is in that build
script -- all it needs to know is that it can be run. Why not keep the API
that simple for pip?
we could do total backward compatible, and simply say: pip will call
"python setup.py install".
And that's it, end of story -- individual packages could do ANYTHING in
that setup.py script -- current packages would be using setuptools, but pip
wouldn't need to care at all what was in there -- only know that it would
install the package.
And, I suppose, a setup.py bdist_wheel for building wheels, and setup.py
sdist for creating source distributions), though I'm not sure that those
need to be a standardized API -- or at least not the SAME standardized API.
-CHB
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160506/64dfdadf/attachment.html>
More information about the Distutils-SIG
mailing list