[Catalog-sig] Deprecate External Links

Wed Feb 27 18:37:15 CET 2013

On Wednesday, February 27, 2013 at 12:10 PM, M.-A. Lemburg wrote:
> Package installers only need access to the static files in
> the /simple/ index. Those can be put behind a CDN to increase
> uptime.
> 
> PyPI itself doesn't have to be up and running if you just want
> to download the files (unfortunately, that's not true at the
> moment, because the /simple/ index is dynamically generated,
> but that can be changed).
> 
> See http://wiki.python.org/moin/CloudPyPI for details.
I'm aware of that, but that doesn't change the statement. If /simple/
is down you cannot determine the external urls. There is no way
to increase uptime by adding more points of failure. 
> 
> > > Another reason for not uploading files to PyPI are the license
> > > terms you have to agree to on PyPI and the fact that you can no
> > > longer control where your distribution files are made available
> > > by agreeing to them. This could be changed as well, but we'd need
> > > to add more legalese to the PyPI mirror setup for this to work...
> > > not sure whether people providing the mirrors would like this.
> > > 
> > 
> > 
> > The legalese doesn't particularly give any more rights than any
> > free/OSS license does. There's not a requirement currently that
> > packages on PyPI be free/OSS but this change would only actually
> > affect people who want to upload non free code to PyPI.
> > 
> 
> 
> It does affect any package author, regardless of the license.
> Some examples:
> 
> * you may be forced remove a distribution from the net (think DMCA,
> patents, trademarks, etc)
> 
> 

IANAL but I'm pretty sure if any of those things occur you didn't have
the legal right to grant that license to the PSF and the PSF would be
required to take them down anyways. 
> 
> * the distribution may contain a serious bug that you don't want to
> spread
> 
> 

This is a completely separate issue. PyPI supports (and always will)
a method of saying "delete and/or don't install this". This is really just
a strawman.
> 
> * you may want to keep more accurate statistics of the reach of
> your project
> 
> 

What statistics do you want? Let's have PyPI produce them and
properly anonymize them instead of leaking data. 
> 
> > > Security can be had by having installers check the GPG signatures
> > > of distribution file. You don't need to trust the download
> > > site for that.
> > > 
> > 
> > 
> > GPG signatures are good, we don't have them yet. And when we do
> > it's only 1 layer of defense, not the final solution.
> > 
> 
> 
> Sure, you still have to trust the author :-)
But do I need to trust his host? Do I need to trust that his laptop didn't
get swiped and with it his GPG key? Ideally I don't *need* to trust the
author either. I download his package from PyPI and I can review it,
then I know it's fine and I can download that version and use it. PyPI
isn't to the point you can make that assumption but It should get there.
> 
> > > I'm not sure what you meant with privacy in this context.
> > 
> > If I download something from server there is a certain amount
> > of information that by nature of HTTP and networking gets
> > leaked to that host. Additionally if it's done via non TLS connections
> > it also gets leaked to anyone who has a MITM on my connection.
> > 
> > This is especially important in countries where the government
> > actively surveils or modifies the traffic of their citizens.
> > 
> 
> 
> I can see an issue with e.g. trying to download code that
> is illegal to use in a country (e.g. crypto code, exploits,
> hacks, etc.), but the country officials would probably just
> block the complete PyPI site than bother with filtering single
> requests.
> 
> IMO, that's beyond the scope of what we're discussing
> here, though.
> 
> 

It's not just "crypto code", "exploits", "hacks" it's also things like
https://ooni.torproject.org/ and Tor itself which are *good* projects
that certain governments might not particularly like.
> 
> > > Something that would work even with older Python versions is
> > > letting the download URL point to a meta-file which contains
> > > the links to the other distribution files. That way you
> > > avoid having the installers trying to parse arbitrary
> > > websites and you can add more security to the downloads
> > > by providing hash values, etc. in those meta-files.
> > > 
> > > Since installers already know how to parse the /simple/
> > > (HTML) index files, we might use that same format
> > > for those meta-files.
> > > 
> > 
> 
> 
> So what do you think of the above idea ?
If the hashes is on the external system then they are as good as
useless. If I'm able to do something nefarious with the packages
that are hosted I can do something nefarious with the metadata file.

Putting the hashes on PyPI fixes the security issue (because we
have a real SSL cert, and tools are starting to validate it) but doesn't
fix the other issues.
> 
> -- 
> Marc-Andre Lemburg
> eGenix.com (http://eGenix.com)
> 
> Professional Python Services directly from the Source (#1, Feb 26 2013)
> > > > Python Projects, Consulting and Support ... http://www.egenix.com/
> > > > mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/
> > > > mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
> > > > 
> > > 
> > 
> 
> ________________________________________________________________________
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
> eGenix.com (http://eGenix.com) Software, Skills and Services GmbH Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> http://www.egenix.com/company/contact/
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130227/5c333757/attachment.html>