[Cython] Cython syntax to pre-allocate lists for performance

Zaur Shibzukhov szport at gmail.com
Thu Mar 7 14:39:33 CET 2013


2013/3/7 Stefan Behnel <stefan_ml at behnel.de>

> Yury V. Zaytsev, 07.03.2013 12:16:
> > Is there any syntax that I can use to do something like this in Cython:
> >
> >     py_object_ = PyList_New(123); ?
>
> Note that Python has an algorithm for shrinking a list on appending, so
> this might not be sufficient for your use case.
>
>
> > If not, do you think that this can be added in one way or another?
> >
> > Unfortunately, I can't think of a non-disruptive way of doing it. For
> > instance, if this
> >
> >     [None] * N
> >
> > is given a completely new meaning, like make an empty list (of NULLs),
> > instead of making a real list of Nones, it will certainly break Python
> > code. Besides, it would probably be still faster than no pre-allocation,
> > but slower than an empty list with pre-allocation...
> >
> > Maybe
> >
> >     [NULL] * N ?
>
> What do you need it for?
>
> Won't list comprehensions work for you? They could potentially be adapted
> to presize the list.
>
>
I guess the problem is to construct new (even empty) list with
pre-allocated memory exactly for N elements.

N*[NULL] - changes semantics because there can't be list with N elements
and filled by NULL.
N*[None] - more expansive for further assignments because of Py_DECREFs.

I suppose that N*[] could do the trick. It could be optimized so that N*[]
is equal to an empty list but with preallocated memory exactly for N
elements. Could it be?

Zaur Shibzukhov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20130307/b16893ac/attachment.html>


More information about the cython-devel mailing list