[Python-Dev] New PEP: Using ssize_t as the index type

M.-A. Lemburg mal at egenix.com
Tue Jan 10 10:18:58 CET 2006


Martin v. Löwis wrote:
> Armin Rigo wrote:
>> This would do the right thing for <= 2.4, using ints everywhere; and the
>> Python.h version 2.5 would detect the #define and assume it's a
>> 2.5-compatible module, so it would override the #define with the real
>> thing *and* turn on the ssize_t interpretation of the '#' format
>> character.
> 
> This would be very similar to the PY_SIZE_T_CLEAN approach, except
> that it would also help to detect spelling mistakes.
> 
>>From an implementation point of view, the real challenge is to
> give PyArg_ParseTuple a different meaning; I do this be #defining
> it to PyArg_ParseTupleSsize_t (to preserve binary compatibility
> for the original interpretation of ParseTuple). Putting
> additional flags arguments in the entire code is also quite
> hackish.
> 
>> I still don't like the idea of a magic #define that changes the behavior
>> of '#include <Python.h>', but I admit I don't find any better solution.
>> I suppose I'll just blame C.
> 
> More precisely, the printf style of function calling, and varargs
> functions. ISO C is pretty type safe, but with varargs functions,
> you lose that completely.
> 
> I still hope I can write a C parser some day that does
> ParseTuple/BuildValue checking.

At least gcc does check the parameter types and generates
warnings for wrong vararg parameters passed to printf() et al.

We definitely need a clean solution for this. Otherwise,
authors who want to support more than just the latest Python
release will run into deep trouble.

Note that the branch also has other changes of output parameter
types which will cause problems in extensions (not only extensions
implementing the sequence protocol as the PEP suggests, but
also ones using it as well as extensions implementing or
using the buffer protocol and the slicing protocol). These are not
as easily avoidable as the PyArg_ParseTuple() problem which
could be dealt with by a a new parser marker for ssize_t
lengths (with the '#' marker casting the argument to an int).

I don't see a good solution for these other than introducing
a set of new APIs (and maybe doing some macro magic as Martin
did for PyArg_ParseTuple()). Due to the fact that changes in
the types of output parameters require changes in the
extension variable type layout itself, they introduce a large
number of type changes in the extension and make writing
backwards compatible extensions harder than necessary.

Furthermore, all extensions for Python 2.4 would have to be
ported to the new Python API and porting is not going to
be a simple recompile, but will require C skills.

We'd also have to make sure that old extensions don't
just import with a warning, since the change will introduce
buffer overflows and seg faults in extensions that are not
aware of the change.

Martin, please add the above points to the PEP. I'd also
like to see it published, because it's hard to track a PEP
in the mailing

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


More information about the Python-Dev mailing list