[Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

[Donald Stufft]

On Mar 11, 2013, at 2:09 AM, PJ Eby <pje at telecommunity.com> wrote:

> On Sun, Mar 10, 2013 at 8:25 PM, Donald Stufft <donald at stufft.io> wrote:
>> I don't think anyone is bad here, nor am I arguing against any particular person or group of people. I'm arguing against a practice and a system. You're going out of your way to find excuses to throw all sorts of stop energy here.
> 
> Calling a legitimate disagreement with your point of view "stop
> energy" seems inappropriate to me, since my issue is with you
> derailing the topic of how to get people to *voluntarily* migrate to a
> better situation than the present one, and to develop tools for that
> process.  The only thing I wish you to stop is the repeated assertion
> without proof that 1) external links must go *and* 2) this must be an
> enforced directive rather than a (highly-encouraged) option.

1) Proof of what? That it's insecure? That it harms uptime? That it violates people's privacy? I don't understand what you want here, do you want me to go and find insecure hosts and start boosting malware onto peoples machines?
2) Even a single project remaining causes the entire thing to cascade, Weakest Link Theory.

> 
> I have even gone so far as to suggest, earlier in this thread, what
> evidence I would find at least suggestive of your POV.  But your
> response to that and prior challenges to those assertions, has been
> simply to move your goalpost.  E.g. from "current uptime is bad" to
> "any uptime lower than PyPI's is totally unacceptable".

I outlined all 3 of the major reasons in my very first email. I've never changed them.

> 
> I, on the other hand, have moved in the direction of *your* proposals
> repeatedly, making adjustments as I find actually-convincing evidence
> and/or reasoning, or find ways to deal with the issues.  I have
> compromised quite a bit.  (And have already spent a fair amount of
> time writing setuptools code to lay a foundation for these changes.)
> 
> You, as far as I can tell, have not moved your position in the slightest.
> 
> Which of these is "stop energy"?

I've not been willing to compromise because none of the solutions presented solves all the actual issues. They just rearrange deck chairs on the titanic.

> 
> It is not the case that external links must be removed from PyPI in
> order to ensure security, or uptime.  And it is *especially* not the
> case that you are the BDFL of uptime.  You're definitely not the BDFL
> of uptime for any given project hosted on PyPI, that you *voluntarily
> choose* to make a part of your build process.  If your primary
> argument is that project X must host its files on PyPI because of your
> build process, then I think you misunderstand open source, and also
> the part where you *chose* to make it part of your build process.  It
> certainly doesn't give you the right to force projects Y, Z, and Q --
> that you don't even use! -- to also host their projects on PyPI,
> because project X -- the one you do use -- has a slow or unreliable
> file host!
> 
> It seems disingenuous to then shfit the argument back to security when
> challenged on uptime, and back to uptime when challenged on security.
> We've looped back and forth over those for some time: when I point out
> that wheels have signatures which will make off-site hosting
> relatively unimportant to the security picture, you jump back to
> talking about uptime.  When I point out that uptime is a consensual
> factor that in no way justifies legislating what other people can do
> with their projects, you go back to talking about security.
> 
> Make up your mind.  What problem are you actually trying to solve?

All of them, as outlined in my original email.

> 
> (I expect your response on wheels to be that wheels aren't there yet,
> etc., but that isn't actually a response to the objection unless
> you're going to change your position to, "okay, external links to file
> formats that can be signed can stay," or something of that sort.
> Otherwise, you're not actually compromising, just using the fact that
> wheels aren't in common use yet as an argument to keep the position
> you started with.)

Signed releases solve 1/3 of the original issues and bring with them their own. How do you transmit the signatures? How do you decide which signatures are valid for any given file? There's a pretty complicated system written called TUF which handles some of these issues (but again it only solves 1/3 of them) and until we get that transmission of the signatures in a sane way is unlikely.

> 
> 
>> My analogy served only to put into light that the system that I'm trying to change is insecure, just like allowing anyone to walk into a bank vault and pick up money would be insecure. I fully believe that the people using such a system are completely trustworthy people. But just because *they* are trustworthy doesn't mean that a system which allows *anyone* to attack other Python developers is *ok*.
> 
> And my analogy served only to put into light the part where you're
> insisting that one group of people change for the benefit of a group
> which is already benefiting from their pre-existing generosity.
> 
> That being said, I do see that I could have misinterpreted the intent
> of your analogy -- it sounded like you were saying that the developers
> who host off-PyPI were thieves walking into your bank and taking your
> money (i.e., analogizing theft with inconveniencing you by making your
> builds fail or run slowly).
> 
> Though to be honest, I still don't comprehend how else to make any
> kind of sense to that analogy in its original context.  Who is the
> bank?  Whose money is being taken?  The whole thing is utterly
> confusing to me if I try to take it any other way than the way I did,
> because it doesn't seem to have any other simple 1:1 mapping to the
> situation, as far as I can see.   Your explanation seems terribly
> abstract and tortured to me, as far as analogies go.

Bank == PyPI, People insisting that the bank vault remain open so they can walk in and grab their own money because it's easier == folks arguing for the existing solution because they don't want to change their release process. Combined this leaves the bank (and in the actual situation, PyPI) open to a number of issues.

> 
> 
>> When discussing security of a system it's necessary to divorce yourself from the implementations of things. When you get wrapped up in the implementation you turn things into a Us vs Them game (as evidenced by several of your messages) instead of discussing the merits of the various systems and which ones serve the greatest needs of the community the best.
> 
> I think you've got things backwards here.  It's you who's been arguing
> that the solution to the problem of "improved uptime and security" is
> best implemented by "ban all non-PyPI hosting".  It is I who has been
> arguing that this is a premature judgment and rush to implementation,
> without considering all of the design angles.  And I am the one asking
> you to stop insisting on this one implementation and state your
> *actual* problem with external links.

Read my first email. Security, uptime, privacy. Note security isn't just about changing out files either, there's a whole host of possible problems most of them documented here: https://www.updateframework.com/wiki/Docs/Security . It's true that his won't solve all of those issues immediately but it moves us to a position where we can start trying.

> 
> (By which I mean, a problem stated such that, if you're given a
> solution that *doesn't* involve banning them from PyPI, you aren't
> going to rejigger the problem statement so that it once again requires
> banning.  That's moving the goalposts, and that's what keeps happening
> in this discussion, at least as far as I can see.  I, on the other
> hand, have given you my actual problem with your proposal, and I have
> not moved *my* goalposts.  Instead, I've moved towards your position,
> more than once.  But I've moved as far towards it as I can go at this
> time, without you providing any additional evidence or explanation or
> *some* kind of engagement with the points that I've raised above that
> you've previously ignored, in this thread and others.)


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130311/2ca3dd75/attachment.pgp>