[Mailman-Developers] MM3 REST API testing [was: Python 3]

Andrew Stuart andrew.stuart at supercoders.com.au
Sat Jan 3 23:44:21 CET 2015

FWIW some notes on mitmproxy and ngrep - helpful tools when intercepting and recording HTTP traffic.

mitmproxy/mitmdump: I know doesn’t serve the same purpose as VCR but its helpful to be able to capture and see what is going on between server and client when Mailman tests are running. 


mitmproxy itself seems to have Python 3 as its next major initiative.

I configured Mailman tests to run through a proxy (I sent an email to this list a few weeks back with instructions for how to do this).


mitmdump -p 7001 -P -w outfile

Captures traffic to outfile.  It can be loaded back into mitmproxy and modified and replayed.

ngrep is useful when digging deeper.  it will show you traffic in real time on a given port. 

for example sudo ngrep -W byline -d lo port 9001 will show what is happening on the Mailman REST API port when configured for port 9001

Dunno if this is useful for anyone but everything helps when digging into HTTP.


On 2 Jan 2015, at 3:40 pm, Barry Warsaw <barry at python.org> wrote:

On Dec 27, 2014, at 01:16 PM, Florian Fuchs wrote:

> So far we haven't found a perfect solution for testing packages that
> rely heavily on the MM3 REST API. As far as mailman.client, Postorius
> and HK are concerned, testing without *some* core integration doesn't
> make too much sense IMO, because any changes in the API would go unnoticed.

I'm playing with a solution that involves the use of the 'vcr' package, which
as its name implies (well, at least for us, um, mature programmers ;) is a
facility to record and replay HTTP traffic.  It's compatible with both Python
2 and 3 and is available on the Cheeseshop.

So the basic idea is to set up an MM3 installation (`tox --notest -r`
suffices), then start up its REST server with a fresh mailman.db, and run the
test suite against it.   This will capture the traffic in a yaml file, which
we could ship with mailman.client.

Then to run the test suite, you'd use vcr in replay mode against the captured
yaml file.  Any time the API changes, you'd have to re-record the traffic
against a fresh database.

I have a branch that's working in this direction and it's promising, but I've
also had to rewrite the testing infrastructure for mailman.client, and make
some other changes.  It also points out what I think are some bugs in Mailman
core, so I'm working those out on my py3 branch in parallel.

It's not yet ready for review, but if you want to keep an eye on things, look
here:  lp:~barry/mailman.client/bilingual

Feel free to provide feedback on the approach, or the changes so far, but I
will submit a merge proposal when/if it's ready to go.

Mailman-Developers mailing list
Mailman-Developers at python.org
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

More information about the Mailman-Developers mailing list