[New-bugs-announce] [issue40682] random.Random.seed() with version=1 does not consistently match Python 2 behavior

[Micah R Ledbetter]

New submission from Micah R Ledbetter <me at micahrl.com>:

When using the random.Random class, using the .seed() method with version=1 does not always reproduce the same results as the .seed() method did in Python 2.

>From the docs, I did expect this, but on closer inspection, I can't tell whether I made a bad assumption or whether there is a bug in the module.

The docs state an intention of compatibility with older versions of Python:
https://docs.python.org/3.9/library/random.html#notes-on-reproducibility

> Most of the random module’s algorithms and seeding functions are subject to change across Python versions, but two aspects are guaranteed not to change:
>
> If a new seeding method is added, then a backward compatible seeder will be offered.
>
> The generator’s random() method will continue to produce the same sequence when the compatible seeder is given the same seed.

It's not clear from the docstring in the code whether this is intended to cover Python 2.7 behavior:
https://github.com/python/cpython/blob/3.9/Lib/random.py#L134

> For version 2 (the default), all of the bits are used if *a* is a str,
> bytes, or bytearray.  For version 1 (provided for reproducing random
> sequences from older versions of Python), the algorithm for str and
> bytes generates a narrower range of seeds.

But the results I've spot checked sometimes do match the Python 2 results, and sometimes are the Python 2 result +1.



I wrote a python script that calls the .seed() method with version=1 under Python 3, and without a version= argument under Python 2. It uses a wordlist I happen to have in /usr/share/dict that I copied to $PWD.

#!/usr/bin/env python
import os, random, sys
mydir = os.path.dirname(os.path.abspath(__file__))
r = random.Random()
maxidx = None
with open('{}/web2'.format(mydir)) as webdict:
  for idx, raw_word in enumerate(webdict.readlines()):
    word = raw_word.strip()
    if sys.version_info[0] == 2:
      r.seed(word)
    elif sys.version_info[0] == 3:
      r.seed(word, version=1)
    else:
      raise Exception("Unexpected python version")
    print("{}: {}".format(word, r.randrange(0, 65535, 1)))
    if maxidx != None and idx >= maxidx:
      break



I also wrote a shell script to run my Python script with the Python versions I happen to have installed locally, along with Python 2.7 and 3.4-3.9 in the ci-image Docker container linked from the Python download page.

#!/bin/sh
set -eux
mkdir -p results
/usr/bin/python test.py > results/macos10.15.4.system.python2.7.16
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3 test.py > results/macos10.15.4.system.python3.8.2
docker run -v $PWD:/testpy:rw -u root -it --rm quay.io/python-devs/ci-image sh -c 'python3.9 /testpy/test.py > /testpy/results/ci-image.python3.9'
docker run -v $PWD:/testpy:rw -u root -it --rm quay.io/python-devs/ci-image sh -c 'python3.8 /testpy/test.py > /testpy/results/ci-image.python3.8'
docker run -v $PWD:/testpy:rw -u root -it --rm quay.io/python-devs/ci-image sh -c 'python3.7 /testpy/test.py > /testpy/results/ci-image.python3.7'
docker run -v $PWD:/testpy:rw -u root -it --rm quay.io/python-devs/ci-image sh -c 'python3.6 /testpy/test.py > /testpy/results/ci-image.python3.6'
docker run -v $PWD:/testpy:rw -u root -it --rm quay.io/python-devs/ci-image sh -c 'python3.5 /testpy/test.py > /testpy/results/ci-image.python3.5'
docker run -v $PWD:/testpy:rw -u root -it --rm quay.io/python-devs/ci-image sh -c 'python2.7 /testpy/test.py > /testpy/results/ci-image.python2.7'



I've made a github repo that contains both scripts and the results:
https://github.com/mrled/random-Random-seed-version-testing

I ran the script on my Mac, which means I used the system installed Python binaries that came with macOS x86_64, but the ci-image Python versions are running under an x86_64 Linux virtual machine (because of how Docker for Mac works).
 
To summarize the results:

* The Python 2.7 on my Mac works the same as the Python 2.7 on the ci-image
* The Python 3.8 on my Mac works the same as Pythons 3.5-3.9 on the ci-image
* Python 3.4 is different from both (although it is now unsupported anyway)

A sample of the results. I haven't programmatically analyzed them, but from my spot checks, they all appear to be like this:

> head results.ci-image.python2.7  |  > head results.ci-image.python3.9
A: 8866                            |  A: 8867
a: 56458                           |  a: 56459
aa: 29724                          |  aa: 29724
aal: 11248                         |  aal: 11248
aalii: 16623                       |  aalii: 16623
aam: 62302                         |  aam: 62303
Aani: 31381                        |  Aani: 31381
aardvark: 6397                     |  aardvark: 6397
aardwolf: 32525                    |  aardwolf: 32526
Aaron: 32019                       |  Aaron: 32019

----------
components: Library (Lib)
messages: 369356
nosy: mrled
priority: normal
severity: normal
status: open
title: random.Random.seed() with version=1 does not consistently match Python 2 behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40682>
_______________________________________