[Python-Dev] reference counting in Py3K

Greg Ewing greg.ewing at canterbury.ac.nz
Thu Sep 8 06:31:17 CEST 2005

Phillip J. Eby wrote:

> Just an FYI; Pyrex certainly makes it relatively painless to write code 
> that interfaces with C, but it doesn't do much for performance, and 
> naively-written Pyrex code can actually be slower than 
> carefully-optimized Python code.  So, for existing modules that were 
> written in C for performance reasons, Pyrex isn't currently a substitute.

If the performance-critical parts of the C code were
translated into Pyrex-C (i.e. Pyrex code that only
performs C operations) it shouldn't be significantly
different. Alternatively, Pyrex could be used to create
a wrapper around the existing C code, freeing it from having
to deal with the Python/C API, thus making it easier to

> One of the reasons for this is that Pyrex code uses the generic Python/C 
> APIs, like PySequence_GetItem, even in cases where PyList_GetItem or its 
> macro form would be more appropriate.  Pyrex has no way currently to 
> say, "this is type X's C API, so use it when you have a variable that's 
> of type X, instead of using the generic object protocols."

I'm thinking of teaching it about some of the builtin types
(e.g. list, dict) so it can use more efficient APIs when

In the meantime, you can declare and call most of the APIs
explicitly if you want. The exception is some of the macro
forms which have nonstandard refcount behaviour. I'm also
thinking of adding some declarations to help with that.

> There are other issues that contribute to the inefficiency as well, like 
> redundant refcounting, assigning None to temporary variables, etc.

Yes, it could probably do with a bit of flow analysis and
peephole optimisation to deal with things like that.

> I haven't used the absolute latest version of Pyrex, but older versions 
> also used C strings for attribute lookups, which was horribly slow.  I 
> think the latest version now creates string objects at module 
> initialization to avoid this issue, though.

Yes, it now precreates and interns all identifier strings,
which should help considerably.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

More information about the Python-Dev mailing list