On 8 October 2014 19:44, Donald Stufft <donald@stufft.io> wrote:
On Oct 8, 2014, at 4:44 AM, holger krekel <holger@merlinux.eu> wrote: I am sorry if raising the issue of private/public compromises sounds like FUD to you. From my experience it's a real attack vector. I talked about this at EP2014 (http://youtu.be/aNrrGf-uNUY?t=6m1s ) and people got back to me afterwards, surprised.
And I don't think you can successfully ask people in companies around the work to register private package names publically (let alone the issue of clashes etc.). Admit it, that's even more unlikely than peple using some PEP438 features :)
And yes, if someone forgets to set the private index he could still pull in malicious public links even with devpi or new pip options.
I think raising the issue is FUDish because it has nothing to do with using multi repository support for things that are registered on PyPI. The attack vector you’re describing isn’t possible at all for any project that is effected by PEP 470, which are projects which wish to register themselves in the PyPI index without using PyPI as their repository.
From my perspective, there's also the question of "relative risk". As soon as anyone is installing anything at all directly from PyPI, their perceived threat level should already be through the roof. Why? Because of .pth files.
.pth files run automatically at every interpreter startup, so if you're installing directly from PyPI, then *every single package you install* has the power to completely subvert your application (if you're installing into an application specific virtual environment) or your entire Python installation (if you're installing into a shared Python instance). Even if they don't install malicious .pth files, then many of them are going to get imported at some point anyway, so they're going to able to reach out and do whatever they want to the Python level internal process state. "pip install python-nation" helps illustrate the degree to which we're generally trusting folks uploading stuff to PyPI to not be evil, and that level of trust also extends to folks providing external repositories rather than hosting directly on PyPI. If folks are more worried about the risk of PyPI or a third party repo shadowing their private packages than they are about malicious .pth files or generally malicious runtime behaviour in dependencies, then I strongly believe their threat meters need recalibrating. We focus on MITM attacks in the upstream infrastructure, because if *developers* are actively malicious, then you're already hosed - they don't need to do anything clever, they can just decide to own your system as a side effect of running their code. (Most of them won't, which is why the risk is low in practice. But as far as theoretical attacks go, this is near the top of my personal threat model, just behind third party MITM attacks) If folks are using Python in a context where these risks are unacceptable to them, then they should either be getting their packages via a trusted third party (like a community or commercial Linux distribution, or a commercial Python redistributor), or at least using a PyPI caching proxy with whitelisting support (which is why PEP 470 recommends the use of devpi in conjunction with turning off the default index). Establishing trustworthiness is expensive and relatively slow, which is why PyPI, like a lot of language specific distribution systems, doesn't currently offer it as feature. The generally more limited and older selection of packages in the redistributor channels reflects some of the impact of those additional costs. Bringing those costs down is something we're going to have to fix on the redistributor side - upstream initiatives like PEP 426 may help, but a lot of it is going to be a matter of Linux distros reassessing what services we're able to provide to the wider open source community. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia