[New-bugs-announce] [issue46647] `test_functools` unexpected failures when C `_functoolsmodule` is missing

Sat Feb 5 05:46:54 EST 2022

New submission from Nikita Sobolev <mail at sobolevn.me>:

Reproduction steps:

1. Add to `Setup.local`:

```
*disabled*
_functoolsmodule
```

2. `.configure && make -j`. Then, ensure that this module is not available:

```
» ./python.exe -c 'import _functools'                                         
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named '_functools'
```

3. Run `test_functools`:

```
======================================================================
ERROR: test_bad_cmp (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 905, in test_bad_cmp
    key = self.cmp_to_key(cmp1)
          ^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

======================================================================
ERROR: test_cmp_to_key (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 869, in test_cmp_to_key
    key = self.cmp_to_key(cmp1)
          ^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

======================================================================
ERROR: test_cmp_to_key_arguments (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 885, in test_cmp_to_key_arguments
    key = self.cmp_to_key(mycmp=cmp1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() got multiple values for argument 'mycmp'

======================================================================
ERROR: test_hash (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 941, in test_hash
    key = self.cmp_to_key(mycmp)
          ^^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

======================================================================
ERROR: test_obj_field (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 920, in test_obj_field
    key = self.cmp_to_key(mycmp=cmp1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() got multiple values for argument 'mycmp'

======================================================================
ERROR: test_sort_int (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 926, in test_sort_int
    self.assertEqual(sorted(range(5), key=self.cmp_to_key(mycmp)),
                                          ^^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

======================================================================
ERROR: test_sort_int_str (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 934, in test_sort_int_str
    values = sorted(values, key=self.cmp_to_key(mycmp))
                                ^^^^^^^^^^^^^^^^^^^^^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

======================================================================
ERROR: test_pickle (test.test_functools.TestPartialC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 258, in test_pickle
    f_copy = pickle.loads(pickle.dumps(f, proto))
                          ^^^^^^^^^^^^^^^^^^^^^^
_pickle.PicklingError: Can't pickle <class 'functools.partial'>: it's not the same object as functools.partial

======================================================================
ERROR: test_recursive_pickle (test.test_functools.TestPartialC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 343, in test_recursive_pickle
    pickle.dumps(f, proto)
    ^^^^^^^^^^^^^^^^^^^^^^
_pickle.PicklingError: Can't pickle <class 'functools.partial'>: it's not the same object as functools.partial

======================================================================
ERROR: test_iterator_usage (test.test_functools.TestReduceC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 843, in test_iterator_usage
    self.assertEqual(self.reduce(add, SequenceClass(5)), 10)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/functools.py", line 249, in reduce
    it = iter(sequence)
         ^^^^^^^^^^^^^^
TypeError: 'builtin_function_or_method' object is not iterable

======================================================================
ERROR: test_reduce (test.test_functools.TestReduceC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 794, in test_reduce
    self.assertEqual(self.reduce(add, ['a', 'b', 'c'], ''), 'abc')
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: reduce() takes from 2 to 3 positional arguments but 4 were given

======================================================================
FAIL: test_disallow_instantiation (test.test_functools.TestCmpToKeyC)
----------------------------------------------------------------------
TypeError: type() takes 1 or 3 arguments

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 955, in test_disallow_instantiation
    support.check_disallow_instantiation(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py", line 2121, in check_disallow_instantiation
    testcase.assertRaisesRegex(TypeError, msg, tp, *args, **kwds)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: "cannot create 'type' instances" does not match "type() takes 1 or 3 arguments"

======================================================================
FAIL: test_attributes_unwritable (test.test_functools.TestPartialC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 402, in test_attributes_unwritable
    self.assertRaises(AttributeError, setattr, p, 'func', map)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: AttributeError not raised by setattr

======================================================================
FAIL: test_attributes_unwritable (test.test_functools.TestPartialCSubclass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 402, in test_attributes_unwritable
    self.assertRaises(AttributeError, setattr, p, 'func', map)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: AttributeError not raised by setattr

----------------------------------------------------------------------
Ran 249 tests in 0.690s

FAILED (failures=3, errors=11)
test test_functools failed
test_functools failed (11 errors, 3 failures)

== Tests result: FAILURE ==

1 test failed:
    test_functools

Total duration: 1.3 sec
Tests result: FAILURE
```

List of individual problems:

1. This function is defined assuming that `c_functools` always has `.lru_cache`: https://github.com/python/cpython/blob/fea7290a0ecee09bbce571d4d10f5881b7ea3485/Lib/test/test_functools.py#L1860-L1862
2. `TestLRUC` is never skipped: https://github.com/python/cpython/blob/fea7290a0ecee09bbce571d4d10f5881b7ea3485/Lib/test/test_functools.py#L1879-L1881 I think it should be, because there's no need to test `_lru_cache_wrapper` twice for just python implementation (default if `_functools` is missing)
3. All similar modules tend to use `fresh=` in `import_fresh_module`, for example: https://github.com/python/cpython/blob/fea7290a0ecee09bbce571d4d10f5881b7ea3485/Lib/test/test_typing.py#L43-L44 But, `test_functools` does not do this: https://github.com/python/cpython/blob/fea7290a0ecee09bbce571d4d10f5881b7ea3485/Lib/test/test_functools.py#L30 So, even if `_functools` is missing, `c_functools` will not be `None`, it will still be `functools.py` module! And this causes multiple unexpected test failures above

Related:
- https://github.com/python/cpython/pull/23405
- https://github.com/python/cpython/pull/23407

I will send a patch for this in a moment.

----------
components: Tests
messages: 412565
nosy: rhettinger, shihai1991, sobolevn
priority: normal
severity: normal
status: open
title: `test_functools` unexpected failures when C `_functoolsmodule` is missing
type: behavior
versions: Python 3.10, Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46647>
_______________________________________