[Security-sig] RFC: PEP: Make os.urandom() blocking on Linux

[Nick Coghlan]

On 22 June 2016 at 17:35, Barry Warsaw <barry at python.org> wrote:
> On Jun 22, 2016, at 11:13 AM, Cory Benfield wrote:
>
>>In a model like this, os.getrandom() would basically need to have, in its
>>documentation, a recipe for using it in a general-purpose, cross-OS
>>manner. That recipe would be, at minimum, an admonition to use the secrets
>>module.
>>
>>However, if we’re going to implement an entire function in order to say “Do
>>not use this, use secrets instead”, why are we bothering? Why add the API
>>surface and a function that needs to be maintained? Why not just make the use
>>of getrandom a private implementation detail of secrets?
>
> Because the os module has traditionally surfaced lower-level operating system
> functions, so os.getrandom() would be an extension of this.  That's also why I
> advocate simplifying os.urandom() so that it reverts more or less to exposing
> /dev/urandom to Python.  With perhaps a few exceptions, os doesn't provide
> higher level APIs.
>
> The point here is that, let's say you're an experienced Linux developer and
> you know you want to use getrandom(2) in Python.  os.getrandom() is exactly
> that.  It's completely analogous to why we provide, e.g. os.chroot() and such.

My own objection (as spelled out in PEP 522) is only to leaving
os.urandom() silently broken when we have the ability to improve on
that - it's an "errors pass silently" and "guessing in the face of
ambiguity" scenario that we previously couldn't sensibly do anything
about, but now have additional options to better handle on behalf of
our users.

As long as os.urandom() is fixed to fail cleanly rather than silently,
I don't object to exposing os.getrandom() as well for the sake of
folks writing Linux specific software that want direct access to the
kernel's blocking behaviour rather than a busy loop. I *do* object to
any solution that proposes that all correct cross-platform code that
needs reliably unpredictable random data necessarily end up looking
like:

    try:
        my_random = os.getrandom
    except AttributeError:
        my_random = os.urandom

WIth the simpler and cleaner "my_random = os.urandom" continuing to
risk silent security failures if the software is used in an
unanticipated context.

Instead, I'm after an outcome for os.urandom() akin to that in PEP
418, where time.time() now looks for several other preferred options
before falling back to _time.time() as a last resort:
https://www.python.org/dev/peps/pep-0418/#time-time

Even if we did add a blocking getrandom() though, I'd still advocate
for secrets and random.SystemRandom to throw BlockingIOError by
default  - with system RNG initialisation being a "once and done"
thing and os.getrandom() exposed, it becomes straightforward to add an
application level "Wait for the random number generator to be ready"
check:

    try:
        wait_for_system_rng = os.getrandom
    except AttributeError:
        pass
    else:
        wait_for_system_rng(1)

The hard part is then knowing that your *need* to wait. If you're
silently getting more-predictable-than-you-expected random data, you
may never realise. If your system hangs, you might eventually figure
it out, but only after a likely frustrating debugging effort. By
contrast, if your application fails with "BlockingIOError: system
random number generator not ready", then you can search for that on
the internet, see the above snippet for "How to wait for the system
random number generator to be ready on Linux" and stick that into your
code.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia