[Security-sig] Implementation of the PEP 524

[Nick Coghlan]

On 18 August 2016 at 22:10, Nick Coghlan <ncoghlan at gmail.com> wrote:
> OK, I'll review it on that basis, and then submit a follow-up RFE and
> patch to add the warning.
>
> I'd like to minimise the divergence between upstream and Fedora, and
> we'll at least have the warning in the downstream version.

It belatedly occurs to me that I should mention a relevant piece of my
work history that may help explain why I'm so strongly in favour of
providing some kind of visible notification as to why the Python
process is blocked waiting for the kernel (and am happy to tackle it
myself as a follow-up patch post-PEP implementation): I spent a couple
of years as the development lead for Red Hat's main hardware
integration testing system, https://beaker-project.org/

While thankfully rare, the single most difficult operating system
level regressions for people to debug in that environment are those
where:

- the system hangs on startup
- before the sshd daemon is running
- with no relevant messages on the system console log

The cases where guest recipes do this aren't *as* bad (since you can
still ssh into the VM host and poke around in the guest via the
hypervisor tooling), but when it happens on bare metal, you either
need to have physical access to the relevant hardware lab yourself, or
get help from a colleague who does.

For the os.urandom change, the first two points on that list are
entirely out of CPython's control (since they depend on what, if
anything, is calling os.urandom() early in the Linux boot cycle), but
we *can* influence the third one (by emitting a warning on stderr
before we block).

Adding such a warning via a downstream patch in Fedora would mostly be
sufficient to address the risk for the users of Fedora's & Red Hat's
Beaker installations (aside from a few situations involving booting
other Linux distros as VM guests), since the two cases where I expect
we may see problems are those I mentioned in PEP 522:

- testing on ARM hardware that doesn't have a proper entropy source set up
- regression tests that run VMs without configuring a proper entropy
source for the VM

and seeing "<module>:<line>: RuntimeWarning: Waiting for system RNG"
in the system console log should be enough to put people on the right
path to identifying the root cause of the problem.

However, other Linux distros tend to have similar regression testing
systems (e.g. it was the Debian one that picked up the issue with
3.5.0), so it makes sense to me to offer this as a general courtesy
upstream, rather than just handling it for Fedora and derivatives
downstream.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia