Hello All,
Postorius, as you all know, is the Web UI for Mailman 3 and is based on Django Web Framework. For some time now, we have been using VCRPy, a HTTP Stubbing library to record test interactions with Mailman Core' API so that the code can be tested in CI environments with the need to have Core.
However, in the past, we have observed that it creates more problems than it solves. First, we test against the state of Mailman Core in past, since the interactions are recorded once and replayed all the time. There are tapes dating back years, which could in theory have changed (though, API's backward compatibility guarantees avoids breaking changes to happen without notice, but even with notice, tests would continue to pass).
Other problem is that it creates friction when writing tests, which was the biggest pain point for myself. You have to write tests, record tapes and then make sure that you keep re-recording till the tests pass. It is not easy to explain to early stage developers either.
Pytest is a new test runner and is pretty popular these days. I decided to use it mostly due to strong support for fixtures and plugins. I wrote a new fixture to make sure that Core's API is started before tests start and all data is erased before each test class.
The changes are in this MR1.
I am also working on moving Core's tests to use pytest instead of nose2,but that process is going to be longer since I am going to have to rewrite some of the nose2 plugins we use there. Doctests also need some attention. Here is the WIP MR2 for that.
-- thanks, Abhilash Raj (maxking)