[Catalog-sig] PyPI terms

[holger krekel]

On Fri, Mar 01, 2013 at 15:11 +0100, M.-A. Lemburg wrote:
> On 01.03.2013 15:02, Jesse Noller wrote:
> > Okie doke. So we can move on to putting up the CDN and deprecating external
> > links for now?
> 
> I don't think anyone is against putting up a CDN. It should meet
> the same security requirements we have for the pypi server itself,
> ie. HTTPS all the way, proper certificates, operated by the PSF,
> perhaps run on a different domain, and whatever other goodies
> Donald can come up with ;-)
> 
> For the external links we need a migration path... that's in the works.
> 
> See http://wiki.python.org/moin/PyPI/DownloadMetaDataProposal for
> a proposal that allows migrating away from relying on external
> hosts in a backwards compatible and secure way.

The page doesn't describe the current "scraping" situation accurately.
As mentioned in my last post, pip/easy_install do _not_ visit
all links found in simple/PKGNAME. Only the ones with rel="home_page" or
rel="download".  So the proposal effectively says to not visit
"homepage" links by default and use a special format for download ones.
The special format i am not sure about - i guess the SHA256 hash there
is to make sure the target content is the correct one, right?
What about abusing download_url some more and do a multiline-format like 
this:

    HASH1 URL-TO-RELEASE-FILE1
    HASH2 URL-TO-RELEASE-FILE2

This way we can avoid any additional http-requests on the pip/easy_install
client side _and_ allow multiple release files.  The simple/PKGNAME metadata 
would contain all information that is needed (and we could probably introduce
a special syntax for #egg github/bitbucket-style tarballs). Those URLs would 
only be retrieved if the client-side installer determines it needs them because
of the user-required version.  You wouldn't need to create a special
"-download.html" file then, no additional http requests, and it's easy to 
create this format without much tool support.

Can't incorporate this into the wiki right now myself and i'd probably 
like to structure the page differently.  The issue here really is the
(future) behaviour of easy_install and pip, not so much distutils or the
pypi server (apart from the worthwhile-to-consider idea of
pulling/caching things).

On a side note i'd rather prefer this to be a github/bitbucket project
where i can submit a pull request :)

best,
holger


> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, Mar 01 2013)
> >>> Python Projects, Consulting and Support ...   http://www.egenix.com/
> >>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>