[Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev)

[David Lyon]

On Thu, 09 Jul 2009 18:04:46 +0900, David Cournapeau
> I am moving this discussion out of python-dev, as it is outside the
> thread topic, but an important discussion nonetheless:
..
>> I think we need to move away from "setup.py install". It's the
> antithesis of static metadata. setup.py needs to go away.
>>
>> If we have a static metadata file called, say "setup.info
> <http://setup.info>", then I'd like to
>> see:
>> "bdist_rpm setup.info <http://setup.info>" produce a .rpm
>> "bdist_msi setup.info <http://setup.info>" produce a .msi
>> "bdist_deb setup.info <http://setup.info>" produce a .deb
>> "bdist_egg setup.info <http://setup.info>" produce a .egg
>> and so on.

Makes perfect sense..

> One such system is the haskell build system, cabal:
> 
> http://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/

Yes, It is well laid out..

> I think it is a good and strong baseline as a starting point. Reusing
> this design in a python equivalent should be much easier than starting a
> system from scratch. 

Yes I agree.

> Under this scheme, backward compatibility with
> distutils could be attempted by automatically converting setup.py files
> to this system, and vice et versa. 

Conversion is a *really* good option.

After examining a small sample of around 120 packages so far, I can
make a generalisation that packages always seem to live either in
their own directory or an .EGG within a package path.

What this means, in simple terms, is that most* packages can be
installed into python simply by a) copying the egg or extracting
the archive (into site-packages) then b) then updating the '
system' (pth, site.path etc)

I concede.. setup.py does look a little magical..

But really, it does no magic.. 

little more than shutil.copytree() and a .pth update...

ok - I'm over simplifying.. but not by much.

My point ? A copytree command is still good for doing a package
installation imho. It means you don't have to worry about specifying
all the files that you want in the package and so forth. That means
a lot less work for the package developer.. a lot less..

PEP 376 on the other hand, does make a lot of good suggestions 
for the internal package format, deinstallation and so forth.

Let's even think about throwing it away yet...

However, he only chronic logic failure that remains now, is 
package dependencies.

That needs to be in metadata somewhere. Easily accessible. Maybe 
it already is. But I thought it was going to be in setup.py. 

Before a package is downloaded, its dependencies need to be 
determined. Not after.. by looking in the setup.py

The easiest solution imho is to extend the PKG-INFO files.

This metadata is on pypi and available via rpc. That's good.

Lets extend it..

So instead of generating the PKG-INFO file (somewhere) as a
result of setup.py running, have the developer or build tool 
(gui2exe,boa,eclipse etc) make it as the first step.

Then, for the 95% of cases that don't do tricky stuff 
with their setup.py, it will mean that setup.py isn't
required.

So let me summarise it all, with one addtion:

 - specify package depends in the PKG-INFO file
   (so we can determine dependencies before download)

 - there's no need for a package manifest (Adds extra 
   developer work - and the system can so easily remove 
   the .egg or directory tree. Why punish the developer)

 - there's no need for setup.py except in complex
   cases. Replace with a post_setup.py

 - (specify) a "doc" directory inside a package to
   hold documentation.

 - keep all the other good things in PEP 376

In closing, I can see all the good work done by lots of people
over time. PEP 376 looks like a pretty challenging piece of work
but its better that it goes ahead rather than the other option in
which nothing gets done.

David