[Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Nathaniel Smith]

On Thu, Jun 9, 2016 at 8:11 PM, Larry Hastings <larry at hastings.org> wrote:

>
> On 06/09/2016 07:58 PM, Nathaniel Smith wrote:
>
> I suspect the crypto folks would be okay with pushing this back to
> 3.6, so long as the final resolution is that os.urandom remains the
> standard interface for, as the docstring says, "Return[ing] a string
> of n random bytes suitable for cryptographic use" using the
> OS-recommended method, and they don't have to go change all their
> code.
>
>
> The Linux core devs didn't like the behavior of /dev/urandom.  But they
> couldn't change its behavior without breaking userspace code.  Linux takes
> backwards-compatibility very seriously, so they left /dev/urandom exactly
> the way it was and added new functionality (the getrandom() system call)
> that had the semantics they felt were best.
>
> I don't understand why so many people seem to think it's okay to break old
> code in new versions of Python, when Python's history has shown a similarly
> strong commitment to backwards-compatibility.  os.urandom() was added in
> Python 2.4, in 2004, and remained unchanged for about thirteen years.
> That's thirteen years of people calling it and assuming its semantics were
> identical to the local "urandom" man page, which was correct.
>

I can only speak for myself, but the the reason it doesn't bother me is
that the documentation for os.urandom has always been very clear that it is
an abstraction over multiple OS-specific sources of cryptographic
randomness -- even in the 2.4 docs [1] we read that its output "depends on
the OS implementation", and that it might be /dev/urandom, it might be
CryptGenRandom, and it might even raise an exception if "a randomness
source is not found". So as a user I've always expected that it will make a
best-effort attempt to use whatever the best source of cryptographic
randomness is in a given environment, or else make a best-effort attempt to
raise an error if it's determined that it can't give me cryptographic
randomness, and it's been doing that unchanged for thirteen years too.

But now Linux has moved forward and provided an improved OS-specific source
of cryptographic randomness, and in particular one that actually signals to
userspace when it doesn't have randomness available. So we have a choice:
either we have to break the guarantee that os.urandom is identical to
/dev/urandom, or we have to break the guarantee that os.urandom uses the
best OS-specific source of cryptographic randomness. Either way we're
breaking some guarantee we used to make. And AFAICT so far 100% of the
people who actually maintain libraries that call os.urandom are asking
python-dev to break the identical-to-/dev/urandom guarantee and preserve
the uses-the-best-OS-specific-cryptographic-randomness guarantee.
Disrupting working code is a bad thing, but in the long run, no-one is
actually asking for an os.urandom that silently falls back on the xkcd #221
PRNG [2].

All that said, on the eve of the 3.5.2 release is a terrible time to be
trying to decide this, and it makes perfect sense to me that maybe 3.5
should kick this can down the road. Your efforts as RM are appreciated and
I'm glad I'm not in your spot :-).

-n

[1] https://docs.python.org/2.4/lib/os-miscfunc.html
[2] https://xkcd.com/221/

-- 
Nathaniel J. Smith -- https://vorpus.org <http://vorpus.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20160609/4c617cc8/attachment.html>