[Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev)

[Jim Fulton]

On Jul 12, 2009, at 6:24 AM, David Cournapeau wrote:
...
> That relates to the "python needs to be a platform" idea.Taking again
> the numpy/scipy case, I would love to get something like a robust
> easy_install, where scientific toolboxes can be installed, queries,
> uninstalled from pypi.

I don't understand what you mean by "queries, uinstalled from pypi".

> Having our own 'python' for installation, would
> make this much easier - but we loose what makes numpy/scipy so
> attractive in the first place, that is using a lot of great packages
> outside the numpy/scipy community.

Why would you lose anything? You can include any additional tools you  
want.

> Those two goals kind of contradict
> with each other, I am not sure where the trade-off should be.

I'm not sure exactly what you mean, but I think this gets back to my  
original point.  Sometimes you want to provide people a robust   
application that addresses a fixed set of use cases. Sometime you want  
to provide a toolbox that allows people to address ad hoc problems.   
In the later case, you will often trade off robustness for  
flexibility. My impression is that numpy is often used as part of a  
toolbox for ad hoc numerical analysis.

> Maybe
> python makes this impossible.

I doubt it has anything to do with Python.

> I would be surprised if this problem were
> limited to numpy/scipy.

Me too.  For example, a database interface is useful both  in closed  
(from the users point of views) applications, and in open computing  
toolboxes.  The "system python" tends to be used like a open computing  
platform for ad hoc computing and small scripts.  It's not well suited  
to supporting large applications. For ad hoc computing, the  
flexibility that the system Python provides is very useful.   
Applications need a lot more control and predictability than a system  
Python can typically provide.

>> Really, many (I'd guess most) web developers who gravitate to
>> setuptools are drawn by the ability to express and manage
>> dependencies.  They use tools like buildout or virtualenv that avoid
>> the pitfalls you mention above. I suspect you have more in common  
>> with
>> us than you think. :)
>
> I know about virtualenv, pip, yolk (much less about buildout). I even
> use them sometimes (I never use setuptools outside a virtualenv, for
> example). But I don't see how this solve the deployment problem. How  
> is
> virtualenv useful for distribution of software to end users ?

Well, I'm more familiar with buildout. :) Buildout makes it  
straightforward to control precisely the packages in your  
application.  A tool named zc.sourcerelease, let's you build a release  
that contains a particular configuration of packages with an install  
script that installs them.  (This isn't limited to Python packages  
either.  It can include 3rd-party applications such as libraries,  
database servers, etc. -- anything that's installed with a configure- 
make-make-install dance.)  This source release then becomes the basis  
for an installer.  We use it to create RPMs for our applications.   
Because we use RPM, our applications can share a common "clean" Python  
RPM that has what you'd get from a source install of Python.  We don't  
include Python in the application RPMs, although it would be pretty  
easy to do so.

I think something like this is possible with pip/virtualenv.  I could  
be wrong.

> On the
> robustness side of things, I cannot say I am satisfied with the  
> solution
> either. It is almost always workable, but not good enough for people  
> who
> do not care much about programming (which I think is the case of a lot
> of researchers/scientists). We have tons of user problems related to
> installation/deployment issues.

I think buildout might help a lot.  It doesn't give you an installer,  
as that is very platform specific, but it gives you a starting point  
to make building an installer pretty straightforward.

Jim

--
Jim Fulton
Zope Corporation