[Python-ideas] Length hinting and preallocation for container types

[Daniel Holth]

On Fri, Mar 8, 2013 at 5:07 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Fri, 08 Mar 2013 08:16:59 +0900,
> "Stephen J. Turnbull" <stephen at xemacs.org>
> a écrit :
>>
>>  > > But why do you think allocation is slow in the general case?
>>  > > Sure, it involves system calls which indeed do slow things down.
>>  >
>>  > It depends what one calls a system call. A memory allocation
>>  > shouldn't always incur a call to the kernel. Library calls can be
>>  > quite fast.
>>
>> If the process itself is growing, eventually it does.
>
> I think most allocators would request big chunks of memory from the
> kernel, and then carve out the small blocks requested by the user from
> that. Therefore, my intuition is that a long-running process, if not
> leaky, should end up not stressing mmap / sbrk system calls too much.
>
>> And I suspect that Alex (and Christian!) would consider saving a few
>> CPU cycles in an inner loop very important.
>
> They probably would, but that is not a design point for Python.
>
> Regards
>
> Antoine.

I am a fan of the proposal. Imagine you are programming for a
memory-constrained system. By telling the list how big it needs to be
you can save precious RAM. It's a pretty standard feature to be able
to hint and trim the size of data structures, just like you can
usually choose the buffer size for stream operations.