[Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

[M.-A. Lemburg]

On 11.03.2013 09:18, Lennart Regebro wrote:
> On Mon, Mar 11, 2013 at 9:06 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> But this isn't necessarily true, there is another solution: mirror your requirements locally.
> 
> I do that. This is not a solution, because your requirements yesterday
> is not your requirements tomorrow.
> 
>> Is it even clear why numerous archives aren't hosted on PyPI?
> 
> No, the only one that has mentioned why is Marc-André, I think, whose
> eGenix packages are distributed as binary packages for loads of
> different platforms. It's unclear to me if all these binary packages
> should be uploaded to PyPI, and it is also unclear to me why they
> can't be, it seems to be mostly a case of it being too much work.

I've listed all the reasons in one of the previous emails:

http://mail.python.org/pipermail/catalog-sig/2013-March/005502.html

Others will likely have additional reasons, like e.g.

* the PyPI uploads not being compatible to their release process

* not knowing that it's possible to host files on PyPI - after
  all it's an *index*, not a repository :-)

* still believing that PyPI is an unreliable hosting provider
  due the many downtimes and problems it had in the past - which
  is no longer true today

* not wanting to host and maintain files in several different
  places

* not wanting to host release files at all, i.e. have people
  check out the version from a repository instead of doing
  the download, unzip, install dance

* not wanting to separate associated library or product
  code from the Python wrapper code (think e.g. the
  Python interface for subversion)

* not being allowed to upload files to external servers
  by company policy, or having to deal with a company
  policy that makes this difficult/unattractive

* having issues with the added latency of PyPI downloads compared
  to a simple file based index hosted on a company web server

* having a strong need to know the number of downloads per
  package and associated statistics such as downloads per
  country, per year/month/day/hour

* not wanting to give up access to the download log files

* having a requirement to restrict downloads on a per country
  basis, e.g. for export controlled software or software which
  may not be imported/used in certain countries

* having PyPI not provide the technical means to host the
  release files, e.g. due to the releases using a format
  which is not supported by PyPI (e.g. all the ActiveState
  packages - http://code.activestate.com/pypm/)

* user experience/support issues:
  if the package has external dependencies,
  or needs special setup, it may provide a better user experience
  to host the Python wrapper on the same page as the dependencies
  and instructions on how to install them; rather than having
  them on PyPI which lets people believe that a simple
  "pip install something" will get them a working setup

Those are just a few things that come to mind. I'm sure there
are more issues that keep authors from uploading their
packages to PyPI.

Overall, I think we should encourage people to make their
code available through PyPI and make it attractive to them,
but keep the possibility to continue using external hosting
platforms, should they run into issues that PyPI cannot
solve for them.

> He also mentioned the big Python distributions eGenix does as being
> too large for PyPI, but I don't really see the point of uploading
> Python distributions to PyPI, they can't be installed with Python
> installers anyway.

Not sure what you mean here.

PyPI is also used to index Python projects which are not Python
packages to be installed by pip/easy_install/etc.

Some of those may also want to

>>  IMHO it would be better to remove barriers than force projects to host files on PyPI.
> 
> Nobody has really been able to point out any real barriers, so we
> don't know what they are or if they exist.

Again, please see the email where I listed the ones affecting
at least eGenix.

Most of those can be addressed in one way or another, e.g.
by having PyPI cache the files, provide access to the download
counts by country, provide a way to host separate indexes for
UCS2/UCS4 egg files, etc.

The only issues that need more investigation are the PyPI license
terms and the general issue of not being able to host export
regulated files on PyPI.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 11 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/