[Python-ideas] Length hinting and preallocation for container types

[Stephen J. Turnbull]

Antoine Pitrou writes:
 > Le Wed, 06 Mar 2013 11:23:26 +0900,
 > "Stephen J. Turnbull" <stephen at xemacs.org>
 > a écrit :
 > > 
 > > Not at all coincidentally.  As a compiler writer (which he refers to
 > > in the slides several times) he is offended by poor performance, as
 > > measured in CPU/invocation.  Good for him!  I'm glad he's working on
 > > PyPy!  But when compiler writers start talking language design for
 > > performance, we inevitably end up with C<0.5 wink/>.
 > 
 > I think that's a strong argument indeed.

Thank you, but it's actually a fallacy (argumentem ad hominem).  I think
it's true that Alex's complaints and Christian's proposals are colored
by that tendency, but it deserves discussion in terms of the proposals
themselves.  (Thus, the 0.5 wink.  I maybe should have deleted it.)

 > > But why do you think allocation is slow in the general case?  Sure, it
 > > involves system calls which indeed do slow things down.
 > 
 > It depends what one calls a system call. A memory allocation shouldn't
 > always incur a call to the kernel. Library calls can be quite fast.

If the process itself is growing, eventually it does.  I don't know
what best practice is, do libraries try to arrange for bounded time
cost of allocation by default?  And are best practices universally
implemented?  What about user tuning of the allocator that happens to
be a pessimization for Python?  As a cross-platform application,
Python needs to worry about that.

And I suspect that Alex (and Christian!) would consider saving a few
CPU cycles in an inner loop very important.  My point is that I don't
consider that appropriate for user-visible language features in Python.
I realize that performance-oriented applications will chafe at the
bureaucracy necessary to get optimizations like comprehensions into
the language, though.  I just think it's a necessary sacrifice (but
it's not my sacrifice, I admit).