[Catalog-sig] [Proposal] Registered packages must provide the source code distribution on PyPI

[M.-A. Lemburg]

Andreas Jung wrote:
> Hi there,
> 
> I propose a policy change for packages registered with PyPI:
> 
>  - packages registered on PyPI have at least one release

I'm not sure what you mean with "release". Every package on
PyPI is a release, since it comes with a version number.

>  - one release of registered package on PyPI _must_ contain
>    a valid source code distribution (sdist)

-100

You'd outrule commercial packages that don't come with a
source distribution. PyPI is for everyone, not only for
open source packages.

Furthermore, not all package authors want to upload their
packages to PyPI.

And lastly, uploading packages to PyPI (still) has a serious
problem: setuptools doesn't know the distinction between
UCS2 and UCS4, so uploading eggs for Unix platforms doesn't
work out in practice. setuptools also doesn't know that
e.g. a Mac OS X fat release may still contain the right binaries
for a non-fat build of Python.

There are other issues as well, e.g. eGenix produces around
50 release files for every package release amounting to
around 150 MB in some cases. It's currently just not feasable to
use PyPI for that.

>  - packages registered on PyPI without releases or without
>    source code release are subject to be removed after N days
>    after the day of registration

Same as above.

> Why?
> 
> Any package registered on PyPI is possibly crucial to any kind of
> development and deployment.
> 
> Packages hosted on external servers (referenced through a download_url)
> are subject to come and go - packages once released should be available
> at any time from a well-known location (PyPI). Dependencies on the
> availability of external downloads servers other than PyPI are hardly
> acceptable for real-world development and deployments.

I think it's for the package users to decide whether they
trust a package author to maintain his or her package.
That's not something PyPI can change.

> As an example: the Plone CMS buildouts depend on python-openid.
> This package is registered with PyPI
> 
> http://pypi.python.org/pypi/python-openid
> 
> but references to
> 
> http://openidenabled.com/files/python-openid/packages/python-openid-2.2.4.tar.gz
> 
> For whatever reason the download URL is no longer working. In fact:
> openidenabled.com now points to http://www.janrain.com.

That's a problem with that particular package, so you should
contact the package author.

Just because one URL goes away doesn't mean that *all* PyPI
package authors who host their software elsewhere are in
poor standing.

> Other reasons for disappearing package in the past:
> 
>  - network or server outages of external servers
>  - users changed their organization and the organization removed
>    content of their former employees

I'd say you open a support request for PyPI and then
let a sys admin add a note to the package or remove the
broken download URL.

> PyPI is a valuable and crucial resource for Python development.
> It must be kept up-to-date and consistent.
> 
> I don't care about the arguments that were made in the past against
> stronger rules ("openness" etc.).

If that's so, but why should we then care about your arguments ?

> There are a lot of Python programmers around that are not Python geeks
> as most of us are and they just become pissed of when packages come and
> go or are not in the place where one would expect them.

That's the nature of the Internet. Besides, would you really want
to use a package that's not being maintained anymore ? Even if you do
have a source or binary distribution for a package on PyPI, would
you really continue to use it if you don't know the author
and it hadn't had any release for 3 years ?

You can't just blindly rely on things that were uploaded to
PyPI and the proposed policy change won't make a difference in
that respect.

> PyPI is a community resource - but community does not mean anarchy where
> everyone should be able to upload its package crap without looking left
> and right and having the community and its needs in mind.

I think that's asked a bit too much of the package authors. PyPI
is just a resource to announce and catalog Python packages, nothing
more.

> PyPI must become a stable package index. Everything registered with PyPI
> must be available at any time (mirrors, distributing PyPI in the cloud...).

I agree that everything uploaded to PyPI should be available
anytime, but not that everything registered with PyPI also
has to be uploaded to PyPI.

Making PyPI more reliable will likely increase the number of
package authors who trust PyPI to host their packages.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 17 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK                31 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/