<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix"><br>
<br>
On 06/02/2017 02:46 AM, Victor Stinner wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAMpsgwYMnV9eCt93+7QV=a9Aukfz4vZBrMtK=1GhpVUYBdSQAg@mail.gmail.com">
<pre wrap="">I would be curious of another test: use pymalloc for objects larger
than 512 bytes. For example, allocate up to 4 KB?
In the past, we already changed the maximum size from 256 to 512 to
support most common Python objects on 64-bit platforms. Since Python
objects contain many pointers: switching from 32 bit to 64 bit can
double the size of the object in the worst case.</pre>
</blockquote>
<br>
You've already seen Tim Peters' post about why we must leave pool
size set to 4k. Obviously This in turn means using obmalloc for
larger objects will mean more and more wasted memory.<br>
<br>
For example, let's say we use obmalloc for allocations of 2048
bytes. Pool size is 4096 bytes, and there's a 48-byte "pool_header"
structure on the front (on 64-bit platforms, if I counted right).
So there are only 4048 bytes usable per pool. After the first 2048
allocation, we're left with 2000 bytes at the end. You can't use
that memory for another allocation class, that's impossible given
obmalloc's design. So that 2000 bytes is just wasted.<br>
<br>
Currently obmalloc's maximum allocation size is 512 bytes; after 7
allocations, this leaves 464 wasted bytes at the end. Which isn't
*great* exactly but it's only 11% of the overall allocated memory.<br>
<br>
Anyway, I'm not super excited by the prospect of using obmalloc for
larger objects. There's an inverse relation between the size of
allocation and the frequency of allocation. In Python there are
lots of tiny allocations, but fewer and fewer as the size
increases. (A similarly-shaped graph to what retailers call the
"long tail".) By no small coincidence, obmalloc is great at small
objects, which is where we needed the help most. Let's leave it at
that.<br>
<br>
<br>
A more fruitful endeavor might be to try one of these fancy new
third-party allocators in CPython, e.g. tcmalloc, jemalloc. Try
each with both obmalloc turned on and turned off, and see what
happens to performance and memory usage. (I'd try it myself, but
I'm already so far behind on watching funny cat videos.)<br>
<br>
<br>
<i>/arry</i><br>
</body>
</html>