[Pytest-commit] Issue #589: pytest incorrectly considers NotImplementedError when detecting recursion (hpk42/pytest)

[Charles Cloud]

New issue 589: pytest incorrectly considers NotImplementedError when detecting recursion
https://bitbucket.org/hpk42/pytest/issue/589/pytest-incorrectly-considers

Charles Cloud:

Here's an example where this happens:

```python
import operator


class EqList(list):
    def __init__(self, *args):
        super(EqList, self).__init__(*args)

    def __nonzero__(self):
        raise ValueError('this is what numpy does')

    def __bool__(self):
        if len(self) > 1:
            return self.__nonzero__()
        return bool(len(self))

    def __eq__(self, other):
        if not isinstance(other, EqList):
            return EqList([False] * len(self))
        return EqList(map(operator.eq, list(self), list(other)))


_typecache = {}


def dispatch(*types):
    def wrapper(f):
        _typecache[types] = f

        def wrapped(*args, **kwargs):
            func = _typecache[tuple(map(type, args))]
            return func(*args, **kwargs)
        return wrapped
    return wrapper


@dispatch(object)
def flatten(o):
    return [o]


@dispatch(tuple)
def flatten(t):
    return flatten(list(t))


@dispatch(EqList)
def flatten(s):
    raise NotImplementedError('EqList bork')


@dispatch(list)
def flatten(lst):
    for el in lst:
        for e in flatten(el):
            yield e


def test_flatten():
    r = EqList([1, 2, 3])
    x = list(flatten([(r,)]))
    assert x is not None


if __name__ == '__main__':
    test_flatten()
```

The issue is the interaction between

1. `py.test` checking for `NotImplementedError` when detecting recursion followed by
2. comparison of variables in scope between a function appearing to be recursive and
3. any variables in said scopes that do not return a scalar `bool` from their `__eq__` method.

The above code should raise `NotImplementedError`, not a huge `INTERNALERROR` traceback because it couldn't compare variables when trying to detect recursion. Running with

```sh
$ python above_file.py
```
does the right thing, while

```sh
$ py.test above_file.py
```
yields a huge `INTERNALERROR` traceback ending in:

```
INTERNALERROR>   File "/Users/pcloud/test_pytest.py", line 15, in __bool__
INTERNALERROR>     return self.__nonzero__()
INTERNALERROR>   File "/Users/pcloud/test_pytest.py", line 11, in __nonzero__
INTERNALERROR>     raise ValueError('this is what numpy does')
INTERNALERROR> ValueError: this is what numpy does
```

I propose that instead of checking for general `RuntimeError`s (which includes subclasses), that `py.test` check for the *exact* type of `RuntimeError`. `NotImplementedError` doesn't seem like an exception that would indicate recursion.