On Thu, Jul 4, 2019 at 11:32 PM Inada Naoki
On Thu, Jul 4, 2019 at 8:09 PM Antoine Pitrou
wrote: Ah, interesting. Were you able to measure the memory footprint as well?
Hmm, it is not good. mimalloc uses MADV_FREE so it may affect to some benchmarks. I will look it later.
``` $ ./python -m pyperf compare_to pymalloc-mem.json mimalloc-mem.json -G Slower (60): - logging_format: 10.6 MB +- 384.2 kB -> 27.2 MB +- 21.3 kB: 2.58x slower (+158%) - logging_simple: 10028.4 kB +- 371.2 kB -> 22.2 MB +- 24.9 kB: 2.27x slower (+127%)
I think I understand why the mimalloc uses more than twice memory than the
pymalloc + glibc malloc in logging_format and logging_simple benchmarks.
These two benchmarks does like this:
buf = [] # in StringIO
for _ in range(10*1024):
buf.append("important: some important information to be logged")
s = "".join(buf) # StringIO.getvalue()
s.splitlines()
mimalloc uses size segregated allocator for ~512KiB. And size class
is determined
top three bits.
On the other hand, list increases capacity by 9/8. It means next size
class is used
on each realloc. At last, all size classes has1~3 used/cached memory blocks.
This is almost worst case for mimalloc. In more complex application,
there may be
more chance to reuse memory blocks.
In complex or huge application, this overhead will become relatively small.
It's speed is attractive.
But for memory efficiency, pymalloc + jemalloc / tcmalloc may be better for
common cases.
Regards,
--
Inada Naoki