Hi, Warning: believe me or not, I only read the first ~50 messages of the recent discussion about random on the Python bug tracker and then the python-dev mailing list. Warning 2: If this email thread gets 100 emails per day as it was the case on the bug tracker and python-dev, I will have to ignore it again. Sorry, but I don't have the bandwith to read so much messages :-( Here is a concrete proposal trying to make Python 3.6 more secure on Linux, without blocking Python at startup. I suggest to stick to Linux first. Sorry, but I don't have the skills to propose a concrete change for other platforms since I don't know well their exact behaviour, and I'm not that they give access to blocking *and* non-blocking urandom. Victor HTML version: https://haypo-notes.readthedocs.io/pep_random.html +++++++++++++++++++++++++++++++++++ Make os.urandom() blocking on Linux +++++++++++++++++++++++++++++++++++ Headers:: PEP: xxx Title: Make os.urandom() blocking on Linux Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner <victor.stinner@gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 20-June-2016 Python-Version: 3.6 Abstract ======== Modify ``os.urandom()`` to block on Linux 3.17 and newer until the OS urandom is initialized. Rationale ========= Linux 3.17 adds a new ``getrandom()`` syscall which allows to block until the kernel collected enough entropy. It avoids to generate weak cryptographic keys. Python os.urandom() uses the ``getrandom()``, but falls back on reading the non-blocking ``/dev/urandom`` if ``getrandom(GRND_NONBLOCK)`` fails with ``EAGAIN``. Security experts promotes ``os.urandom()`` to genereate cryptographic keys, even instead of ``ssl.RAND_bytes()``. Python 3.5.0 blocked at startup on virtual machines, waiting for the OS urandom initialization, which was seen as a regression compared to Python 3.4 by users. This PEP proposes to modify os.urandom() to more is more secure, but also ensure that Python will not block at startup. Changes ======= * Initialize hash secret from non-blocking OS urandom * Initialize random._inst, a Random instance, with non-blocking OS urandom * Modify os.urandom() to block until urandom is initialized on Linux A new _PyOS_URandom_Nonblocking() private method will be added: read OS urandom in non-blocking mode. In practice, it means that it falls back on reading /dev/urandom on Linux. _PyRandom_Init() is modified to call _PyOS_URandom_Nonblocking(). Moreover, a new ``random_inst_seed`` will be added to the ``_Py_HashSecret_t`` structure (see above). random._inst will be initialized with the ``random_inst_seed`` secret. A flag will be used to ensure that this secret is only used once. If a second instance of random.Random is created, blocking os.urandom() will be used. Alternative =========== Never use blocking urandom in the random module ----------------------------------------------- The random module can use ``random_inst_seed`` as a seed, but add other sources of entropy like the process identifier (``os.getpid()``), the current time (``time.time()``), memory addresses, etc. Reading 2500 bytes from os.urandom() to initialize the Mersenne Twister RNG in random.Random is a deliberate choice to get access to the full range of the RNG. This PEP is a compromise between "security" and "feature". Python should not block at startup before the OS collected enough entropy. But on the regular use case (OS urandom iniitalized), the random module should continue to its code to initialize the seed. Python 3.5.0 was blocked on ``import random``, not on building a second instance of ``random.Random``. Annexes ======= Why using os.urandom()? ----------------------- Since ``os.urandom()`` is implemented in the kernel, it doesn't have some issues of user-space RNG. For example, it is much harder to get its state. It is usually built on a CSPRNG, so even if its state is get, it is hard to compute previously generated numbers. The kernel has a good knowledge of entropy sources and feed regulary the entropy pool. Linux getrandom() ----------------- On OpenBSD, FreeBSD and Mac OS X, reading /dev/urandom blocks until the kernel collected enough entropy. It is not the case on Linux. Basically, if a design choice should be make between usability and security, usability is preferred on Linux, whereas security is preferred on BSD. The new ``getrandom()`` of Linux 3.17 allows users to choose security be blocking until the kernel collected enough entropy. On virtual machines and some embedded devices, it can take longer than a minute to collect enough entropy. In the worst case, the application will block forever because the kernel really has no entropy source and so cannot unblock ``getrandom()``. Copyright ========= This document has been placed in the public domain.