[Catalog-sig] [Distutils] pre-PEP : Synthesis of previous threads, and irc talks + proposals
Phillip J. Eby
pje at telecommunity.com
Wed Oct 1 20:28:07 CEST 2008
At 07:55 PM 10/1/2008 +0200, Tarek Ziadé wrote:
>On Wed, Oct 1, 2008 at 7:29 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > I'm -1 on all of the above. I think we need a standard for tools interop
> > (ala WSGI), not implementation tweaks for the existing tools. I also think
> > that a concrete metadata format proposal is premature at this time; we've
> > barely begun to gather -- let alone specify -- our requirements for that
> > metadata. (Essentially, only version dependencies have been discussed,
> > AFAICT.)
>What are the other important points we need to discuss at this point
>in your opinion ?
What information needs to be conveyed by a "build" tool to an
"install" tool, and vice versa.
For example, an install tool needs to know what files are
documentation, which are sample data, and what is part of the library
proper, as well as what things are scripts (and in what language
those scripts are written, e.g. Python, shell, batch, etc.). Some
install tools need to know about icons, menus, registry info, cron
jobs, etc. (These are perhaps more properly the domain of
applications than libraries, but I'm going to assume that these
things are in scope.)
The way that this information is communicated needs to be extensible,
so that optional metadata for debs and msi's and rpm's and
whathaveyou can be incorporated, without needing to modify the
standard -- especially if the APIs for reading and writing this data
are in the stdlib.
There needs to be a way for install tools to ask a source package to
"configure" itself, possibly specifying options (and a way for it to
find out what those options are, to be able to present them to the
user). Then there needs to be a way for the install tool to ask the
package to build itself with the configured options, and a standard
for how the build tool(s) communicate errors or other issues back to
the install tools.
There needs to be a way, of course, for the package to specify what
build tools it needs in order to be built, and for those to plug in
to the (again stdlib-contained) build API.
There needs to be a better API for querying C configuration and
compiler details, that's separate from the distutils "ccompiler" stuff.
Last, but not least, there needs to be a definition of core build and
install tools to be both an example/reference implementation of the
standard, and to provide the basics needed by the core.
I think I've mentioned all of these previously in the thread. I also
think that as a matter of technicalities, these things are not
difficult to achieve... but if it they are only achieved
*technically*, then the result is a failure, not a success.
In order for the *real* goal to be achieved (i.e., a flourishing
build/install system for Python), widespread participation and buy-in
is required. If the OS people or the big package people (e.g. Zope
Corp., Enthought) or the distutils aficionados are left out, then the
result won't get used.
I think the best way to ensure that nobody is left out, is to get
them to participate in the design of a standard that ensures that
*they* will be able to control their destiny, by creating their own
build plugins and/or install tools... or at least having a robust
choice of alternatives.
We need a consensus "de jure" standard (ala WSGI), rather than just
an ongoing "de facto" standard (ala distutils/setuptools), or we're
not making any substantial progress, just handing the reins over to
> >> Enforcements:
> >> - a binary distribution cannot be uploaded if a source distrbution
> >> has not been previously provided for the version
> > Note that this doesn't allow closed-source packages to be uploaded; thus it
> > would need to be a warning, rather than a requirement.
>Right. do you agree it is something useful to do ?
> >> New features:
> >> - we should be able to download the metadata of a package without
> >> downloading the package
> >> - PyPI should display the install and test dependencies in the UI
> > It could only do this for specific binaries, since dependencies can be
> > dynamic.
>What dynamic means here ? the python module to static file process or more ?
>can you provide an example ?
Dependencies can be platform-specific as well as python-version
specific. If you want ctypes, you would depend on it in python 2.4
but not in 2.5. Similarly, if on some platform a certain library is
required to implement a feature that is natively accessible on other
platforms. In these cases, you would have logic in setup.py to
detect this and choose the appropriate dependencies.
More information about the Catalog-SIG