Re: [Mailman-Developers] Sample / dummy / generated data for mailman?
On Sep 05, 2012, at 02:01 PM, Ethan Fremen wrote:
Is there any way to automatically generate lists/messages/members etc?
If not, how do you (existing devs) go about setting up mailman with said data?
I am interested in working on https://bugs.launchpad.net/postorius/+bug/1004049 , and so need to add some emails in "pending" state. More generally, having a bunch of lists/users/etc would be helpful in developing the UI further.
In general, I'm not a fan of sample data for tests. Back in my Launchpad days, we had tons of sample data that the tests relied on and we had a terrible experience with it. It makes tests brittle, difficult to understand the initial state, very difficult to change or update, etc. We were slowly weening ourselves off of sample data by the time I moved on to other projects.
The alternative is for the tests themselves to explicitly set up whatever minimal state they are testing, usually in their setUp() methods. They typically use the internal APIs (e.g. interfaces and a few other convenience functions) to set things up, including mailing lists, users, pending requests, emails in queues, etc. There's also a test helper that ensures all that state is cleaned up after each test, and this is hooked into the test layers (though eventually I'd like to get rid of zope.testing).
The mm3 does have a small amount of sample data, but only for the schema migration tests. Because the model can't be used to set up the test state, we basically create an unmigrated database and populate it from sample data, then run through the migrations, and test the results through the model. The model can't be used to set up the initial state because it already reflects the migrated schema and because it's Python code, it's not easy to change in the middle of a test.
I'm not sure how it's done in Postorius, but for the core, you can look through the substantial amounts of existing tests to see how things are done. There's probably a bit of "style skew" with the older tests doing things a bit out of what I now consider good style, but I've tried to spend some time getting the tests in a consistent state.
Hopefully that helps. -Barry
Hi Ethan,
if you work on the Python side of Postorius, like Barry said, you don't really need sample data in form of "real" lists/held messages/etc. *Ideally*, the code is covered by tests that don't rely on sample data (or even the mailman core) to be present (see an example in the source[1]).
That being said, Postorius is in some kind of transition when it comes to testing. We used to have some tests based on the Django test HTTP client which tested more or less the whole stack, using a temporary mailman database. But this was slow, complicated and error-prone.
All communication with the mailman core happens through mailman.client and the data expected to come out of it is pretty well documented in the docs [2][3]. That makes it pretty easy to use mock API return values to test our business logic (like in the example above). There are also nice ways in Django to test login status without touching the database and to test response/request objects without making actual HTTP calls. All of this can make testing more efficient and fun. I hope we can design our future tests more like that (suggestions are welcome!).
OTOH: If you work on the HTML/CSS side it can make your life easier to have *some* real data present. You can add lists, domains, users etc. to a local mailman installation without having to care about working MTA settings. Feeding real emails into the local system is a little bit more complicated. Also, it is little fun to compose actual mails for every scenario that might be relevant for the UI (overlong texts etc.). I prefer to test a design directly in the templates or change the data in Firebug or even temporarily manipulate the output of the view code.
Hope this helps...!
Cheers Florian
[1] http://bazaar.launchpad.net/~mailman-coders/postorius/trunk/view/head:/src/p... [2] http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/files/head:/src/mail... [3] http://bazaar.launchpad.net/~mailman-coders/mailman.client/trunk/view/head:/...
On 09/05/2012 03:37 PM, Barry Warsaw wrote:
On Sep 05, 2012, at 02:01 PM, Ethan Fremen wrote:
Is there any way to automatically generate lists/messages/members etc?
If not, how do you (existing devs) go about setting up mailman with said data?
I am interested in working on https://bugs.launchpad.net/postorius/+bug/1004049 , and so need to add some emails in "pending" state. More generally, having a bunch of lists/users/etc would be helpful in developing the UI further.
In general, I'm not a fan of sample data for tests. Back in my Launchpad days, we had tons of sample data that the tests relied on and we had a terrible experience with it. It makes tests brittle, difficult to understand the initial state, very difficult to change or update, etc. We were slowly weening ourselves off of sample data by the time I moved on to other projects.
The alternative is for the tests themselves to explicitly set up whatever minimal state they are testing, usually in their setUp() methods. They typically use the internal APIs (e.g. interfaces and a few other convenience functions) to set things up, including mailing lists, users, pending requests, emails in queues, etc. There's also a test helper that ensures all that state is cleaned up after each test, and this is hooked into the test layers (though eventually I'd like to get rid of zope.testing).
The mm3 does have a small amount of sample data, but only for the schema migration tests. Because the model can't be used to set up the test state, we basically create an unmigrated database and populate it from sample data, then run through the migrations, and test the results through the model. The model can't be used to set up the initial state because it already reflects the migrated schema and because it's Python code, it's not easy to change in the middle of a test.
I'm not sure how it's done in Postorius, but for the core, you can look through the substantial amounts of existing tests to see how things are done. There's probably a bit of "style skew" with the older tests doing things a bit out of what I now consider good style, but I've tried to spend some time getting the tests in a consistent state.
Hopefully that helps. -Barry
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/f%40state-of-mind....
Security Policy: http://wiki.list.org/x/QIA9
participants (2)
-
Barry Warsaw -
Florian Fuchs