[Catalog-sig] PyPI and setuptools

[Ronald Oussoren]

On 11 Feb, 2013, at 19:38, Daniel Holth <dholth at gmail.com> wrote:

> 
> Sure, and I'd love to see something like pip in the std library.   With wheel files (PEP 427), metadata 1.3 (PEP 426) and the database of installed packages in PEP 376 it should be possible to create a basic, but fully functional, version of a packaging tool in the stdlib that interoperates nicely with more advanced tools like pip and buildout.
> 
> It would be very costly to freeze packaging innovation now.

What kind of innovation do you forsee in the installation space that would conflict with adding a basic installer to the stdlib? In particular, when that basic installer would not be based on distutils, but possibly with a hook for running arbitrary build systems (possibly using a setup.cfg that contains an option that names a PyPI package that knows how to convert that sdist to a wheel archive). 

A basic installer that can install, uninstall and update a wheel (taking care of dependencies) and has a subcommand for listing installed packages should be stable enough, everything else (such as buildout functionality and other bits of pip) can stay outside of the stdlib.

Assuming that distlib and the discussion about securing PyPI settles down before 3.4 I'd expect that most innovation related to packaging would happen on the build part (Improved tools for generating wheel archives from a source archive), and possibly advanced deployment tools (like buildouts features for generating a tree with not just python packages but also supporting software like web- and database servers). Neither of which have to live in the stdlib, while basic installation tool is a useful "included battery" for beginners as wel as advanced users. 

> 
> I'd like to see a version of pip that worked as a script without being importable by the current environment.
> 
> The hardest part would be creating a generic interface for building wheel files that doesn't rely on distutils (but without excluding it).
> 
> It would not be that difficult to write a pure-Python-only wheel builder that did not compile extensions, syntax-checked a hand-written PKG-INFO file (Metadata 1.3), and archived everything up nicely.

I've looked at the spec and it should be easy enough to add a bdist_wheel command to distutils.  I don't recall the consensus on distutils development right now, but if I recall correctly adding a new bdist subcommand would be fine, but I'm not sure if adding support for metadata version 1.3 would be fine as well.  I've experimented a littlle with PEP 390 (static metadata) and that appears to work just fine. I've been using a generic setup.py file in some of my packages that dynamicly constructs a normal call to distribute.setup(), doing something simular in distutils itself should be doable as well although this should be done carefully to avoid unnecessary breakage in distribute/setuptools.

A bdist_wheel command in distutils would be able to build extensions as well, even if the compiler support in distutils is not perfect (although I must admit that I've never run into serious issues beyond distutils tendency to recompile all C files in an extension when I change one of them). 

>  
> Anyways, not using pip doesn't mean having a hopelessly outdated build system :-)
> 
> You could also use buildout 2.0 or Bento :-)

Don't get me started about buildout, I've spent an afternoon gettting nowhere with a buildout script provided by a py2app/pyobjc user (and never getting to the point where I could debug the issue he was having). All tools have advantages and disadvantages, and for my way of working pip is at about neutral w.r.t. distribute.  That doesn't mean pip and buildout are useless, they are just not useful enough for *me*.

Ronald
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130211/e1086d1f/attachment.html>