On Mon, Jun 17, 2019 at 5:18 PM Antoine Pitrou
On Mon, 17 Jun 2019 11:15:29 +0900 Inada Naoki
wrote: Increasing pool size is one obvious way to fix these problems. I think 16KiB pool size and 2MiB (huge page size of x86) arena size is a sweet spot for recent web servers (typically, about 32 threads, and 64GiB), but there is no evidence about it.
Note that the OS won't give a huge page automatically, because memory management becomes much more inflexible then.
When we used contiguous 2MB pages, Linux may use Huge Page
implicitly when Transparent Huge Page is enabled.
Then, if we munmap one page of the 2MB, the kernel will split
the huge page into small many pages again.
I don't know it really happens on Python applications,
but using 2MB arena will reduce the risk of performance penalty
of page splitting.
In web applications, it's common to one Python worker process
use 50~100+ MiB RSS. 2MB arena seems reasonable for those
applications.
I am not proposing making this default.
I just meant this may be a reasonable configuration for many Python
users who create medium~large size web apps.
Regards,
--
Inada Naoki