[Python-ideas] Length hinting and preallocation for container types

Daniel Holth dholth at gmail.com
Fri Mar 8 14:18:07 CET 2013

On Fri, Mar 8, 2013 at 5:07 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Fri, 08 Mar 2013 08:16:59 +0900,
> "Stephen J. Turnbull" <stephen at xemacs.org>
> a écrit :
>>  > > But why do you think allocation is slow in the general case?
>>  > > Sure, it involves system calls which indeed do slow things down.
>>  >
>>  > It depends what one calls a system call. A memory allocation
>>  > shouldn't always incur a call to the kernel. Library calls can be
>>  > quite fast.
>> If the process itself is growing, eventually it does.
> I think most allocators would request big chunks of memory from the
> kernel, and then carve out the small blocks requested by the user from
> that. Therefore, my intuition is that a long-running process, if not
> leaky, should end up not stressing mmap / sbrk system calls too much.
>> And I suspect that Alex (and Christian!) would consider saving a few
>> CPU cycles in an inner loop very important.
> They probably would, but that is not a design point for Python.
> Regards
> Antoine.

I am a fan of the proposal. Imagine you are programming for a
memory-constrained system. By telling the list how big it needs to be
you can save precious RAM. It's a pretty standard feature to be able
to hint and trim the size of data structures, just like you can
usually choose the buffer size for stream operations.

More information about the Python-ideas mailing list