Could you open an issue on Core for this?

There you go: https://gitlab.com/mailman/mailman/issues/327

Have you done any other analysis on where the bottlenecks are? Is it CPU, db, I/O?

I haven't investigated properly but it seems to be the database (which is on another host).

Currently the REST server uses Python's stdlib wsgiref WSGIServer which I'm positive is not performant enough for your lists. There are a couple of options that would be worth investigating. I played with trying to use gunicorn, but deleted the branch a while ago (IIRC, there were problems and I didn't have time to investigate further). At the very least, I want to look into asyncio-based REST server (e.g. aiohttp, uvloop), but maybe there are other parts of the stack we need to look at too.

Interesting. I wonder if we can have SQLAlchemy play well with asyncio, I haven't tried that before. There is still a gunicorn.py file in the contrib directory, but I haven't tested it either.

Aurélien

Could you open an issue on Core for this?

There you go: https://gitlab.com/mailman/mailman/issues/327

Have you done any other analysis on where the bottlenecks are? Is it CPU, db, I/O?

I haven't investigated properly but it seems to be the database (which is on another host).

Currently the REST server uses Python's stdlib wsgiref WSGIServer which I'm positive is not performant enough for your lists. There are a couple of options that would be worth investigating. I played with trying to use gunicorn, but deleted the branch a while ago (IIRC, there were problems and I didn't have time to investigate further). At the very least, I want to look into asyncio-based REST server (e.g. aiohttp, uvloop), but maybe there are other parts of the stack we need to look at too.

Interesting. I wonder if we can have SQLAlchemy play well with asyncio, I haven't tried that before. There is still a gunicorn.py file in the contrib directory, but I haven't tested it either.

Aurélien