Python 2.2.2 regrtest.py Module Produces Unpredictable Results

[Merton Campbell Crockett]

There seems to be a problem with Lib/test/regrtest.py that was included in
the Python 2.2.2 distribution.

After downloading Python 2.2.2, I modified configure to skip tests of
modules that are not supported under BSD/OS 4.x and 5.x.  I ran the standard
regression test using "gmake test".  Lib/test/regrtest.py reported that the
following modules crashed.

	test_long_future.py(c)
	test_math.py(c)

Setting the TESTOPTS environmental variable, I tested each of the modules
individually.  Lib/test/regrtest.py reported that each test completed
successfully in both passes.

This seemed a little strange.  So I changed TESTOPS to conduct the tests in
a random order.  This time both modules passed but two other modules failed.

At first, it appeared that the tests that failed were whatever ended up at 
positions 86 and 91 in the list.  These positions in the list correspond to
the positions in the default order of execution of the above modules.

Several more random tests showed this not to be the case.  Conducting the
regression tests in random order results in random tests failing with tests
failing in pass 1 being successful in pass 2.

How does one know that Python is built correctly when the regression tests
produce different results from run to run and pass to pass?

Is the random test feature one of those "Doctor.  Doctor.  It hurts when I
do this." scenarios?

Merton Campbell Crockett

-- 
BEGIN:				vcard
VERSION:			3.0
FN:				Merton Campbell Crockett
ORG:				General Dynamics Advanced Information Systems;
				Intelligence Solutions
N:				Crockett;Merton;Campbell
EMAIL;TYPE=internet:		mcc at CATO.GD-AIS.COM
TEL;TYPE=work,voice,msg,pref:	+1(805)497-5045
TEL;TYPE=pager,msg:		+1(877)528-0049
TEL;TYPE=fax,work:		+1(805)497-5050
TEL;TYPE=cell,voice,msg:	+1(805)377-6762
END:				vcard