[Catalog-sig] egg_info in PyPI

[P.J. Eby]

At 08:43 PM 9/17/2010 +0200, Martin v. Löwis wrote:
>Here at the DZUG conference, we are planning to integrate explicit 
>access to setuptools metadata to the package index.
>
>The current idea is to store the contents of the egg_info directory,
>giving remote access to specific files. By default, PyPI will extract,
>per release, data from the egg that may get uploaded (using the first
>one if multiple eggs get uploaded). If no egg gets uploaded, a VM
>based build service will generate it from a source distributions.
>Tools like setuptools or distribute could also directly upload this
>information, e.g. as part of the register command.
>
>Any opinions?

FYI, egg-info directories can store arbitrary data (see e.g. the 
EggTranslations package, which uses it for localization resources), 
so you may want to impose some restrictions on *which* metadata files 
to include.

Second, if a user uploads a source distribution built with 
setuptools, it will include an .egg-info directory automatically, so 
you can simply extract it from there.  Conversely, if the source 
distribution is *not* built with setuptools, then building it with 
setuptools would not produce much information in the egg-info anyway.

IOW, there might not be much point to using a build service.  A 
project that doesn't use setuptools would have (at most) these 
metadata entries:

PKG-INFO
SOURCES.txt    # a manifest of the sdist contents
top_level.txt  # list of top-level package/module names
scripts/       # source code of scripts in the package)
zip-safe or not-zip-safe   # flag file

I don't think that most of these are useful for PyPI searches, though 
I suppose a listing of the name of the scripts the package includes 
could be useful.

I'm not sure how useful it is to just have URLs for accessing the 
files, though, vs. having actual searches on structured data provided 
in the files.  For example, an index of projects by package or module 
names, or of projects that provide a particular entry point.