[Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Barry Warsaw]

On Jun 15, 2016, at 01:01 PM, Nick Coghlan wrote:

>No, this is a bad idea. Asking novice developers to make security
>decisions they're not yet qualified to make when it's genuinely
>possible for us to do the right thing by default is the antithesis of
>good security API design, and os.urandom() *is* a security API
>(whether we like it or not - third party documentation written by the
>cryptographic software development community has made it so, since
>it's part of their guidelines for writing security sensitive code in
>pure Python).

Regardless of what third parties have said about os.urandom(), let's look at
what *we* have said about it.  Going back to pre-churn 3.4 documentation:

    os.urandom(n)
    Return a string of n random bytes suitable for cryptographic use.

    This function returns random bytes from an OS-specific randomness
    source. The returned data should be unpredictable enough for cryptographic
    applications, though its exact quality depends on the OS
    implementation. On a Unix-like system this will query /dev/urandom, and on
    Windows it will use CryptGenRandom(). If a randomness source is not found,
    NotImplementedError will be raised.

    For an easy-to-use interface to the random number generator provided by
    your platform, please see random.SystemRandom.

So we very clearly provided platform-dependent caveats on the cryptographic
quality of os.urandom().  We also made a strong claim that there's a direct
connection between os.urandom() and /dev/urandom on "Unix-like system(s)".

We broke that particular promise in 3.5. and semi-fixed it 3.5.2.

>Adding *new* APIs is also a bad idea, since "os.urandom() is the right
>answer on every OS except Linux, and also the best currently available
>answer on Linux" has been the standard security advice for generating
>cryptographic secrets in pure Python code for years now, so we should
>only change that guidance if we have extraordinarily compelling
>reasons to do so, and we don't.

Disagree.

We have broken one long-term promise on os.urandom() ("On a Unix-like system
this will query /dev/urandom") and changed another ("should be unpredictable
enough for cryptographic applications, though its exact quality depends on OS
implementations").

We broke the experienced Linux developer's natural and long-standing link
between the API called os.urandom() and /dev/urandom.  This breaks pre-3.5
code that assumes read-from-/dev/urandom semantics for os.urandom().

We have introduced churn.  Predicting a future SO question such as "Can
os.urandom() block on Linux?" the answer is "No in Python 3.4 and earlier, yes
possibly in Python 3.5.0 and 3.5.1, no in Python 3.5.2 and the rest of the
3.5.x series, and yes possibly in Python 3.6 and beyond".

We have a better answer for "cryptographically appropriate" use cases in
Python 3.6 - the secrets module.  Trying to make os.urandom() "the right
answer on every OS" weakens the promotion of secrets as *the* module to use
for cryptographically appropriate use cases.

IMHO it would be better to leave os.urandom() well enough alone, except for
the documentation which should effectively say, a la 3.4:

    os.urandom(n)
    Return a string of n random bytes suitable for cryptographic use.

    This function returns random bytes from an OS-specific randomness
    source. The returned data should be unpredictable enough for cryptographic
    applications, though its exact quality depends on the OS
    implementation. On a Unix-like system this will query /dev/urandom, and on
    Windows it will use CryptGenRandom(). If a randomness source is not found,
    NotImplementedError will be raised.

    Cryptographic applications should use the secrets module for stronger
    guaranteed sources of randomness.

    For an easy-to-use interface to the random number generator provided by
    your platform, please see random.SystemRandom.

Cheers,
-Barry