[Catalog-sig] egg_info in PyPI

Thomas Lotze thomas at thomas-lotze.de
Sat Sep 18 11:29:32 CEST 2010

Hi there,

I'm going to add my own 2 cents to the discussion as I'm involved in the
matter here at the DZUG conference.

Tarek Ziadé wrote:

> Now you want to publish another metadata format at PyPI ? If PyPI takes
> that direction and adopts, promotes and publishes a standard that is not
> the one we worked on in the past year, this will increase our difficulty
> to push the new format so its adopted by the tools then the community.

Actually, I see it being the other way around: People use what used to
work for them and most people won't switch to using the new standard just
for the fun of it. In order for the new standard to acquire some momentum,
people need to be interested in the subject the standard is about and see
some concrete advantage in switching. Exposing existing metadata will make
it possible to build new applications that use metadata information; it's
those applications' responsibility then to promote the new standard.

Let me take the tl.eggdeps package as a concrete example. The package
currently analyses dependencies between installed packages. I'd like to
expand it to analyse dependencies between any packages on PyPI but I can't
as long as dependency information is not available without actually
installing things. Exposing PEP 345 metadata on PyPI won't be of any
practical use until a critical number of packages actually specify that
information. I simply won't bother implementing the feature either until
some point far in the future or unless I can use legacy metadata as a
fall-back. Having the legacy metadata available enables me to build the
feature now; this could raise interest in metadata as such, and I might
even make visually apparent which dependencies have been specified by PEP
345 means and which I could only guess from legacy information.

> People will just get confused because they will find two competing
> metadata formats  That's exactly the situation where we were at, and
> that's exactly where I don't want to go back.

That's simply a matter of documentation. It has to be clearly communicated
that the legacy information is provided as a transitional service and that
users of that metadata use it only until the newly specified information
is available for a critical number of packages (the exact value of which
may of course depend on the application in question).

> I am not even understanding what's the benefit of doing this since an
> egg_info directory is obtained at *build* time and can differ from a
> machine to another, so it seems pretty useless for me to publish this.

We do understand that legacy metadata has its issues, but taken with the
appropriate amount of salt, it's better than nothing in most cases, enough
so at least to solve some chicken-and-egg problems.

> We worked hard to build some standards, but if PyPI doesn't help us here,
> everything we did and are doing is useless.

Nothing that we're talking about in this thread will contradict what
you want to achieve and what work you've done. By the reasoning I tried to
explain above, what PyPI is going to do will actually help you. OTOH, I
personally don't think it would be a constructive way of helping you if
PyPI tried to enforce the new standard by keeping people from using
information they may at the moment only get through legacy means.


More information about the Catalog-SIG mailing list