On Sun, Nov 26, 2017 at 5:30 PM, Glyph <glyph@twistedmatrix.com> wrote:


Quite often—as I believe is the case for the patch you’re referring to here—patches are adjusting behaviors which would be difficult to cleanly integration-test on the platform in question anyway, and the appropriate thing to do is to make some mock match the observed behavior of the platform in question.

Just because there's enough misunderstanding about how to do this kind of thing and the words that describe it, I wanted to point out that this isn't really a "mock" - and it's certainly not something you would do with a library with "mock" in its name.

For those who don't know (not Glyph), a mock is an object that somehow records an exact sequence of operations (like "my foo method was called with bar" or "my baz attribute was accessed").  These tend to result in very fragile tests because they operate at such a specific level.  Quite often entirely legitimate implementation changes invalidate the expectations of some mock and force corresponding test suite maintenance.

A better approach is to make sure the platform-specific interface being tested has been reduced to its smallest possible scope and then create a verified fake which implements that interface.  The verification is performed by a test suite which can run against both implementations (the fake and the real - with the real version only being tested in the necessary environment).  All other related tests can use the verified fake instead of the real implementation and will run on all platforms.

Jean-Paul