Testing the tests by modifying the ordering of dict items.
Python code should not depend upon the ordering of items in a dict. Unfortunately it seems that a number of tests in the standard library do just that.
Changing PyDict_MINSIZE from 8 to either 4 or 16 causes the following tests to fail:
test_dis test_email test_inspect test_nntplib test_packaging test_plistlib test_pprint test_symtable test_trace
test_sys also fails, but this is a legitimate failure in sys.getsizeof()
Changing the collision resolution function from f(n) = 5n + 1 to f(n) = n + 1 results in the same failures, except for test_packaging and test_symtable which pass.
Finally, changing the seed in unicode_hash() from (implicit) 0 to an arbitrary value (12345678) causes the above tests to fail plus:
test_json test_set test_ttk_textonly test_urllib test_urlparse
I think this is a real issue as the unicode_hash() function is likely to change soon due to http://bugs.python.org/issue13703.
1. Submit one big bug report?
2. Submit a bug report for each "failing" test separately?
3. Ignore it, since the tests only fail when I start messing about?
On Thu, 05 Jan 2012 13:46:52 +0000 Mark Shannon email@example.com wrote:
Submit one big bug report?
Submit a bug report for each "failing" test separately?
I would say a separate bug report for each failing test file, i.e. one report for test_dis, one for test_email etc. Hope this doesn't eat too much of your time :)
Did you raise bug for this?
-- View this message in context: http://python.6.n6.nabble.com/Testing-the-tests-by-modifying-the-ordering-of... Sent from the Python - python-dev mailing list archive at Nabble.com.