[Tutor] Does the secrets module in Python 3.6 use a hardware RNG like that provided in Intel CPUs?

[Steven D'Aprano]

Hi Simon,

On Fri, Mar 09, 2018 at 10:07:14PM +0000, Simon Connah via Tutor wrote:
> Hi,
> I was reading through the secrets documentation in Python 3.6 and 
> noticed that it uses /dev/urandom but I'm unsure if that means it'll 
> use a hardware RNG or just one provided by the operating system (Linux 
> / Windows / etc) in software.

Getting cryptographic-quality random numbers right is very hard, and 
that's not something which the Python core developers have either the 
experience or desire to do. So the secrets module is an interface to the 
operating system's source of randomness.

If your operating system uses a hardware RNG for /dev/urandom (or the 
equivalent for Windows), then so will the secrets module. If it doesn't, 
then neither will secrets.

In other words, Python trusts the operating system. Generally speaking, 
most people should too. Most major operating systems, including Windows, 
Linux, OS X, and various Unixes have well-respected RNGs which are 
generally considered secure.

But of course there's a lot we don't know about the state of the art of 
*secret* crypto research and the capabilities of major government 
intelligence agencies. What little we do know, we can thank a handful of 
people like Edward Snowden and a few other unnamed whistle-blowers who 
have leaked NSA documents.

And flaws could be discovered at any time. Naturally, if a flaw is 
discovered, the secrets module cannot magically patch it or replace the 
software with something else. That's up to the operating system.


> The question is is it possible to 
> determine the source of the randomness from os.urandom if there was 
> ever a flaw found in a particular hardware RNG?

You have to ask your OS developers about that, but the secrets module 
doesn't support anything like that. It can't peer inside the OS and 
determine how os.urandom works.

Certainly you are right to be cautious about hardware RNGs. Back in 2013 
it has become clear that the NSA at least (if not other intelligence 
agencies) have compromised or inserted backdoors into many if not all 
hardware-based RNGs in common use:

http://www.nytimes.com/2013/09/06/us/nsa-foils-much-internet-encryption.html?pagewanted=all


So if you are using a commercially available hardware RNG, you should 
assume that the Five Eyes countries (the USA, UK, Australia, Canada and 
New Zealand) have compromised it.

In the case of Linux, at least, /dev/urandom will use the output of the 
hardware RNG, but it is mixed in with other sources of cryptographically 
strong randomness and is believed to be safe.

https://plus.google.com/+TheodoreTso/posts/SDcoemc9V3J


> I'm 
> just a bit curious about the whole "will always use the strongest 
> source for pseudo-random numbers" when research could change that 
> assumption overnight based on discovered flaws.

The full quote is:

"The secrets module provides access to the most secure source of 
randomness THAT YOUR OPERATING SYSTEM PROVIDES." [emphasis added]

https://docs.python.org/3/library/secrets.html

So don't imagine that the secrets module has access to the cutting edge 
classified crypto technology used by the NSA :-)

If you have any other questions, please feel free to ask on the mailing 
list, and I will do my best to answer.

By the way, I am the author of the secrets module:

https://www.python.org/dev/peps/pep-0506/

If you haven't already read the PEP (Python Enhancement Proposal), you 
should, it contains a lot of background for why the secrets module was 
invented.


> This is probably a 
> really stupid question and if it is I apologise but I'm somewhat 
> confused.

No need to apologise! Hope I cleared up your confusion, and if not, 
please feel free to ask anything else that concerns you.

Regards,


-- 
Steve