[Tutor] object size in python is in what units?

[eryksun]

On Tue, Jul 23, 2013 at 10:21 AM, Oscar Benjamin
<oscar.j.benjamin at gmail.com> wrote:
>
> The reason for the patterns that you observe are related to CPython's
> memory pool allocator. You can read the long source comment describing
> this here:
> http://hg.python.org/cpython/file/a5681f50bae2/Objects/obmalloc.c#l367
>
> Essentially anything over 512 bytes will be allocated using a call to
> the system (OS/runtime) malloc. Anything under 512 bytes will be
> allocated into a memory pool managed by the interpreter. The code I
> linked to shows the exact strategy used for this and if you read this
> you'll have a vague understanding of the results you see.

Prior to 3.3 the upper limit is 256 bytes. Here's the link for 2.7.5:

http://hg.python.org/cpython/file/ab05e7dd2788/Objects/obmalloc.c#l20

Pools for a given storage class are 4 KiB, and are allocated from the
system in 256 KiB arenas. On 32-bit Windows Python, the pool overhead
is 32 bytes. So for 40-byte blocks, a pool will allocate 101 objects,
with 24 bytes left unused, i.e. 32 + 101*40 + 24 == 4096.

3.3 also switched to using anonymous mmap on POSIX (where supported),
and as your link shows -- 3.4 looks to be switching to VirtualAlloc on
Windows. These changes bypass malloc to avoid high watermark effects
that prevent the heap from shrinking. On my Debian box 3.3 does a much
better job releasing memory back to the system after allocating and
deallocating thousands of objects.

> You can check the size in bytes of an object with sys.getsizeof e.g.:
>
>>>> import sys
>>>> sys.getsizeof(lardKronk)
> 36

sys.getsizeof() includes the GC header that precedes the object base
address. Only container objects (such as normal class instances) have
this header, which is used by gc to resolve circular references. The
header is 12 bytes on 32-bit Windows Python. If you want to exclude
it, use the object's __sizeof__() method instead.