[Python-ideas] Python's Source of Randomness and the random.py module Redux

[Antoine Pitrou]

Donald Stufft <donald at ...> writes:
> 
> How has it not been taken into account? The current proposal (best summed up
> by Nick in the other thread) will not break compatability for anyone except
> those calling the functions that are specifically about setting a seed or
> getting/setting the current state.

That's a pretty big "except". Paul's and my concern is about compatibility
breakage, saying "it doesn't break compatibility except..." sounds like a
lot of empty rhetoric.

> In looking around I don't see a lot of
> people using those particular functions

Given that when you "look around" you only end up looking around amongst the Web
developer crowd, I may not be surprised.

You know, when I "look around" I don't see a lot of people using the random
module to generate passwords. Your anecdote would be more valuable than other
people's?

> However, one of
> the biggest groups of people who are most likely to be helped by this
change is
> new and inexperienced developers who don't fully grasp the security sensitive
> nature of whatever they are doing with random.

Yes, because generating passwords is a common and reasonable task for new
and inexperienced developers? Really?

Again, why don't you propose a dedicated API for that? That's what we did
for constant-time comparisons. That's what people did for password hashing.
That's what other people did for cryptography. I haven't seen a reasonable
rebuttal to this. Why would generating passwords be any different from all
those use cases? After all, if you provide a convenient API people should
flock to it, instead of cumbersomely reinventing the wheel... That's what
libraries are for.

> However, I/we
> are willing to compromise by sacrificing possible security in order to not
> regress things where we can, in particular a user-space CSPRNG is being
> proposed over SystemRandom because it will provide you with random numbers
> almost as fast as MT will.

Really, it's not so much a performance issue as a compatibility issue.
The random module provides, by default, a *deterministic* stream of random
numbers. That os.urandom() may be a tad slower isn't very important when you're
generating one number at a time and processing it with a slow interpreter
(besides, MT itself is hardly the fastest PRNG out there). That os.urandom()
doesn't give you a way to seed it once and get predictable results is
a big *regression* if made the default RNG in the random module.

And the same can be said for a user-space CSRNG, as far as I understand
the explanations here.

> However, when proposing this possible compromise, we are met with people
> refusing to meet us in the middle.

See, people are fed up with the incompatibilities arising "in the name of
the public good" in each new feature release of Python. When the "middle"
doesn't sound much more desirable than the "extreme", I don't see why I
should call it a "compromise".

Some people have to support code in 4 different Python versions and
further gratuitous breakage in the stdlib doesn't help. Yes, they can
change their code. Yes, they can use the "six" module, the "future" module
or whatever new bandaid exists on PyPI. Still they must change their code
in a way or another because it was deemed "necessary" to break compatibility
to solve a concern that doesn't seem grounded in any reasonable analysis.

Python 3 was there to break compatibility. Not Python 3.4. Not Python 3.5.
Not Python 3.6.

(in case you're wondering, trying to make all published code on the Internet
secure by appropriately changing the interpreter's "behaviour" to match
erroneous expectations - even *documented* as erroneous - is *not* reasonable
- no matter how hard you try, there will always be occurrences of broken code
that people copy and paste around)

> Can you explain what
> compromise you're willing to accept here?

Let's rephrase this: are *you* willing to accept an admittedly "insecure
by default" compromise?

No you aren't, evidently. There's no evidence that you would accept to
leave the top-level random functions intact, even if a new UserSpaceSecureRandom
class was added to the module, right?

So why would we accept a compatibility-breaking compromise? Because we are
more "reasonable" than you?

(which in this context really reads: more willing to quit the discussion
because of boredom, exhaustion, lack of time or any other quite humane
reason; which, btw, sums up of significant part of what the dynamics of
python-ideas have become: "victory of the most obstinate")

Yeah, that's always what you are betting on, because it's not like *you*
will ever be reasonable except if it's the last resort for getting something
accepted. And that's why every discussion about security with security-minded
(read: "obsessed") people is a massive annoyance, even if at the end it
succeeds in reaching a "compromise", after 500+ excruciating backs and forths
on a mailing-list.

Regards

Antoine.