[Pytest-commit] Issue #596: xdist is not executing tests if their parametrizations were collected in a different order (hpk42/pytest)

[liori]

New issue 596: xdist is not executing tests if their parametrizations were collected in a different order
https://bitbucket.org/hpk42/pytest/issue/596/xdist-is-not-executing-tests-if-their

liori:

Related to #594, but slightly different.

I generated by parametrizations by iterating over a set of strings. However, this iteration depends on the specific values the strings hash to, and these values might be different for every process. Therefore, whereas I was always generating exactly the same parametrizations, they were in different order.

A simple test case that shows the problem:

```
#!python

import pytest

my_names = {'john', 'kate', 'alfred', 'paul', 'mary'}

@pytest.mark.parametrize('name', list(my_names), ids=list(my_names))
def test_is_name_short(name):
    assert len(name) < 7
```

Run with `PYTHONHASHSEED=random py.test -n 4` to make sure you trigger randomized hashing for strings.

I think that a simple `sort` in `report_collection_diff` before running `unified_diff` might help with this problem.