[Python-Dev] Our responsibilities (was Re: BDFL ruling request: should we block forever waiting for high-quality random bits?)

[Donald Stufft]

> On Jun 16, 2016, at 7:07 AM, Barry Warsaw <barry at python.org> wrote:
> 
> On Jun 16, 2016, at 06:04 AM, Donald Stufft wrote:
> 
>> Regardless of what we document it as, people are going to use os.urandom for
>> cryptographic purposes because for everyone who doesn’t keep up on exactly
>> what modules are being added to Python who has any idea about cryptography at
>> all is going to look for a Python interface to urandom. That doesn’t even
>> begin to touch the thousands upon thousands of uses that already exist in the
>> wild that are assuming that os.urandom will always give them cryptographic
>> random, who now *need* to write this as:
> 
> [...]
> 
>> Frankly, I think it’s a disservice to Python developers to leave in this
>> footgun.
> 
> This really gets to the core of our responsibility to our users.  Let's start
> by acknowledging that good-willed people can have different opinions on this,
> and that we all want to do what's best for our users, although we may have
> different definitions of "what's best”.


Yes, I don’t think anyone is being malicious :) that’s why I qualified my
statement with “I think”, because I don’t believe that whether or not this
particular choice is a disservice is a fundamental property of the universe,
but rather my opinion influenced by my priorities.


> 
> Since this topic comes up over and over again, it's worth exploring in more
> detail.  Here's my take on it in this context.
> 
> We have a responsibility to provide stable, well-documented, obvious APIs to
> our users to provide functionality that is useful and appropriate to the best
> of our abilities.
> 
> We have a responsibility to provide secure implementations of that
> functionality wherever possible.
> 
> It's in the conflict between these two responsibilities that these heated
> discussions and differences of opinions come up.  This conflict is exposed in
> the os.urandom() debate because the first responsibility informs us that
> backward compatibility is more important to maintain because it provides
> stability and predictability.  The second responsibility urges us to favor
> retrofitting increased security into APIs that for practicality purposes are
> being used counter to our original intent.

Well, I don’t think that for os.urandom someone using it for security is running
“counter to it’s original intent”, given that in general urandom’s purpose is for
cryptographic random. Someone *may* be using it for something other than that,
but it’s pretty explicitly there for security sensitive applications.

> 
> It's not that you think backward compatibility is unimportant, or that I think
> improving security has no value.  In the messy mudpit of the middle, we can't
> seem to have both, as much as I'd argue that providing new, better APIs can
> give us edible cake.

Right. I personally often fall towards securing the *existing* APIs and adding
new, insecure APIs that are obviously so in cases where we can reasonably do
that. That’s largely because given an API that’s both being used in security
sensitive applications and ones that’s not, the “failure” to be properly secure
is almost always a silent failure, while the “failure” to applications that don’t
need that security is almost always obvious and immediate.

Taking os.urandom as an example, the failure case here for the security side is
that you get some bytes that are, to some degree, predictable. There is nobody
alive who can look at some bytes and go “oh yep, those bytes are predictable
we’re using the wrong API”, thus basically anyone “incorrectly” [1] using this
API for security sensitive applications is going to have it just silently doing
the wrong thing.

On the flip side, if someone is using this API and what they care about is it
not blocking, ever, and always giving them some sort of random-ish number no
matter how predictable it is, then both of the proposed failure cases are fairly
noticeable (to varying degrees), either it blocks long enough for it to matter
for those people and they notice and dig in, or it raises an exception and they
notice and dig in. In both cases they get some indication that something is
wrong.

> 
> Coming down on either side has its consequences, both known and unintended,
> and I think in these cases consensus can't be reached.  It's for these reasons
> that we have RMs and BDFLs to break the tie.  We must lay out our arguments
> and trust our Larrys, Neds, and Guidos to make the right --or at least *a*--
> decision on a case-by-case basis, and if not agree then accept.

Right. I’ve personally tried not to personally be the one who keeps pushing
for this even after a decree, partially because it’s draining to me to argue
for the security side with python-dev [2] and partially because It was ruled
on and I lost. However if there continues to be discussion I’ll continue to
advocate for what I think is right :)


[1] I don’t think using os.urandom is incorrect to use for security sensitive
    applications and I think it’s a losing battle for Python to try and fight
    the rest of the world that urandom is not the right answer here.

[2] python-dev tends to favor not breaking “working” code over securing existing
    APIs, even if “working” is silently doing the wrong thing in a security
    context. This is particularly frustrating when it comes to security because
    security is by it’s nature the act of taking code that would otherwise
    execute and making it error, ideally only in bad situations, but this
    “security’s purpose is to make things break” nature clashes with python-dev’s
    default of not breaking “working” code in a way that is personally draining
    to me.

—
Donald Stufft