
I'm curious about how the REST interface scales out. I've posted a bug to Gitlab about Postorius making a REST request for every list member on the list info page [1], but aside from waiting for a fix for that, I've also been looking more into the performance of the Mailman3 REST interface itself.
Testing with siege (15 concurrent users) on a production VM, I get 29 requests/sec and a concurrency of just 1.3x requesting the "/3.0/domains" URL. For "/3.0/system/versions" I get only about 0.2x concurrency. This may not scale too well to a very active Postorius, especially if 1 Postorious request results in many Mailman API requests.
Is anyone using anything besides wsgiref for the REST interface? (uwsgi, gunicorn, ?) I thought I could get some extra performance that way...
Eric

On Jul 08, 2015, at 05:13 PM, Eric Searcy wrote:
You're probably the first person to experiment with this, so please let us know how it goes. If you get one of the alternative wsgi servers working and it improves things, please do contribute some documentation. I'd definitely like for this to be a viable option. wsgiref is there because it's all stdlib.
Cheers, -Barry

I’ve fumbled around here to see if I can make it work with gunicorn.
Correct me if I’m wrong but effectively gunicorn would need to run the rest server within itself, so it would be necessary to run the rest server outside the mailman runners? I’m not sure how to do so.
rest/wsgiapp.py says:
# Both the REST server and the signal handlers must run in the main # thread; the former because of SQLite requirements (objects created # in one thread cannot be shared with the other threads), and the # latter because of Python's signal handling semantics.
rest/wsgiapp.py also has the following function which seems to be the right wsgi app function to pass into gunicorn. I couldn’t find the right incantation to get it to run standlone.
def make_application(): """Create the WSGI application.
Use this if you want to integrate Mailman's REST server with your own WSGI
server.
"""
return RootedAPI(Root())

On Jul 10, 2015, at 10:47 AM, Andrew Stuart wrote:
Most of the RESTRunner is probably not applicable to the gunicorn case.
Runners themselves are forked from the master watcher, so the warnings about
threads is there solely because we have to run the .server_forever() call in a
separate thread within the rest runner process, otherwise the main thread will
block and we won't be able to use the Runner infrastructure to kill it when
mailman stop
is issued.
I'm honestly not sure that SQLite restriction is applicable any more, at least for this particular case. Maybe it was once when we were using Storm. But I think it's generally acknowledged that production systems should probably be backed by a client/server database, although SQLite may work fine for smaller sites.
The intention is that this would work; what problems have you seen?
My thinking is that a gunicorn -or other external wsgi server- based REST
runner wouldn't do much more than run gunicorn with the proper commands in a
subprocess, and manage its PID so that the runner itself could response to
mailman stop
and restart
commands.
If that works, I could see adding runner subclasses to runners/rest.py such
that a site could change [runner.rest]class
in their mailman.cfg file to
select say the gunicorn based REST server over the default wsgiref one.
Cheers, -Barry

On Jul 10, 2015, at 10:43 AM, Barry Warsaw wrote:
I took a slightly different approach. Basically don't start gunicorn with
mailman start
but start it in a shell after the main system is up and
running. Here's a very experimental and almost completely untested branch
that could get you started:
https://gitlab.com/warsaw/mailman/tree/gunicorn
Read the docstring in this commit for details.
https://gitlab.com/warsaw/mailman/commit/3fbfde8793c9c5bf12b0f250a58b3e3134f...
Cheers, -Barry

On 7/10/15 11:16 AM, Barry Warsaw wrote:
This didn't work for me without modification, both because gunicorn was trying to pass 2 arguments into make_application, and because the initialization was trying to happen each request, which caused issues (mailman crashing trying to add the same language twice).
Here's how I resolved it (API running under gunicorn 19.3.0 correctly responds to requests; didn't test much beyond that):
https://gitlab.com/emsearcy/mailman/commit/33fa949cfa190d32e3fd68ed02e787cc0...
It's curious to me that Mailman Bundler makes references to Gunicorn support, but it seems there code would run into the same problems I did (multiple initiations and argument count mismatch). Is anybody using this from Bundler currently and can help me figure out why the difference?
https://mailman-bundler.readthedocs.org/en/latest/ https://gitlab.com/mailman/mailman-bundler/blob/master/mailman_web/wsgi.py https://gitlab.com/mailman/mailman-bundler/blob/master/templates/deployment/...
Best regards,
Eric

On Jul 08, 2015, at 05:13 PM, Eric Searcy wrote:
You're probably the first person to experiment with this, so please let us know how it goes. If you get one of the alternative wsgi servers working and it improves things, please do contribute some documentation. I'd definitely like for this to be a viable option. wsgiref is there because it's all stdlib.
Cheers, -Barry

I’ve fumbled around here to see if I can make it work with gunicorn.
Correct me if I’m wrong but effectively gunicorn would need to run the rest server within itself, so it would be necessary to run the rest server outside the mailman runners? I’m not sure how to do so.
rest/wsgiapp.py says:
# Both the REST server and the signal handlers must run in the main # thread; the former because of SQLite requirements (objects created # in one thread cannot be shared with the other threads), and the # latter because of Python's signal handling semantics.
rest/wsgiapp.py also has the following function which seems to be the right wsgi app function to pass into gunicorn. I couldn’t find the right incantation to get it to run standlone.
def make_application(): """Create the WSGI application.
Use this if you want to integrate Mailman's REST server with your own WSGI
server.
"""
return RootedAPI(Root())

On Jul 10, 2015, at 10:47 AM, Andrew Stuart wrote:
Most of the RESTRunner is probably not applicable to the gunicorn case.
Runners themselves are forked from the master watcher, so the warnings about
threads is there solely because we have to run the .server_forever() call in a
separate thread within the rest runner process, otherwise the main thread will
block and we won't be able to use the Runner infrastructure to kill it when
mailman stop
is issued.
I'm honestly not sure that SQLite restriction is applicable any more, at least for this particular case. Maybe it was once when we were using Storm. But I think it's generally acknowledged that production systems should probably be backed by a client/server database, although SQLite may work fine for smaller sites.
The intention is that this would work; what problems have you seen?
My thinking is that a gunicorn -or other external wsgi server- based REST
runner wouldn't do much more than run gunicorn with the proper commands in a
subprocess, and manage its PID so that the runner itself could response to
mailman stop
and restart
commands.
If that works, I could see adding runner subclasses to runners/rest.py such
that a site could change [runner.rest]class
in their mailman.cfg file to
select say the gunicorn based REST server over the default wsgiref one.
Cheers, -Barry

On Jul 10, 2015, at 10:43 AM, Barry Warsaw wrote:
I took a slightly different approach. Basically don't start gunicorn with
mailman start
but start it in a shell after the main system is up and
running. Here's a very experimental and almost completely untested branch
that could get you started:
https://gitlab.com/warsaw/mailman/tree/gunicorn
Read the docstring in this commit for details.
https://gitlab.com/warsaw/mailman/commit/3fbfde8793c9c5bf12b0f250a58b3e3134f...
Cheers, -Barry

On 7/10/15 11:16 AM, Barry Warsaw wrote:
This didn't work for me without modification, both because gunicorn was trying to pass 2 arguments into make_application, and because the initialization was trying to happen each request, which caused issues (mailman crashing trying to add the same language twice).
Here's how I resolved it (API running under gunicorn 19.3.0 correctly responds to requests; didn't test much beyond that):
https://gitlab.com/emsearcy/mailman/commit/33fa949cfa190d32e3fd68ed02e787cc0...
It's curious to me that Mailman Bundler makes references to Gunicorn support, but it seems there code would run into the same problems I did (multiple initiations and argument count mismatch). Is anybody using this from Bundler currently and can help me figure out why the difference?
https://mailman-bundler.readthedocs.org/en/latest/ https://gitlab.com/mailman/mailman-bundler/blob/master/mailman_web/wsgi.py https://gitlab.com/mailman/mailman-bundler/blob/master/templates/deployment/...
Best regards,
Eric
participants (5)
-
Abhilash Raj
-
Andrew Stuart
-
Barry Warsaw
-
Barry Warsaw
-
Eric Searcy