[Python-Dev] IMPORTANT 2.5 API changes for C Extension Modules and Embedders

Neal Norwitz nnorwitz at gmail.com
Wed Apr 12 08:30:07 CEST 2006


If you don't write or otherwise maintain Python Extension Modules
written in C (or C++) or embed Python in your application,
you can stop reading.

Python 2.5 alpha 1 was released April 5, 2006.  The second alpha
should be released in a few weeks.  There are several changes
which can cause C extension modules or embedded applications
to crash the interpreter if not fixed.  Periodically, I will send out
these reminders with updated information until 2.5 is released.

  * support for 64-bit sequences (eg, > 2GB strings)
  * memory allocation modifications

64-bit changes
--------------
There are important changes that are in 2.5 to support 64-bit systems.
The 64-bit changes can cause Python to crash if your module is not upgraded
to support the changes.  Python was changed internally to use 64-bit
values on 64-bit machines for indices.  If you've got a machine with
more than 16 GB of RAM, it would be great if you can test Python with
large (> 2GB) strings and other sequences.

For more details about the Python 2.5 schedule:
http://www.python.org/dev/peps/pep-0356/
For more details about the 64-bit change:
http://www.python.org/dev/peps/pep-0353/
How to fix your module:
http://www.python.org/dev/peps/pep-0353/#conversion-guidelines

The effbot wrote a program to check your code and find potential
problems with the 64-bit APIs.
http://svn.effbot.python-hosting.com/stuff/sandbox/python/ssizecheck.py

Memory Allocation Modifications
-------------------------------
In previous versions of Python, it was possible to use different
families of APIs (PyMem_* vs. PyObject_*) to allocate and free
the same block of memory.  APIs in these families include:

  PyMem_*:    PyMem_Malloc, PyMem_Realloc, PyMem_Free,
  PyObject_*: PyObject_Malloc, PyObject_Realloc, PyObject_Free

There are a few other APIs with similar names and also the macro variants.

In 2.5, if allocate a block of memory with one family, you must reallocate
or free with the same family.  That means:

  If you allocate with PyMem_Malloc (or MALLOC), you must reallocate
      with PyMem_Realloc (or REALLOC) and free with PyMem_Free (or FREE).
  If you allocate with PyObject_Malloc (or MALLOC), you must reallocate
      with PyObject_Realloc (or REALLOC) and free with PyObject_Free (or FREE).

Using inconsistent APIs can cause double frees or otherwise crash
the interpreter.  It is fine to mix and match functions or macros
within the same family.

Please test and upgrade your extension modules!

Cheers,
n


More information about the Python-Dev mailing list