[Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...)

[M.-A. Lemburg]

Martin v. Löwis wrote:
> M.-A. Lemburg wrote:
>> ... and then the type change of that variable propagates
>> throughout the extension.
> 
> That depends on the usage of the code. If the variable
> is passed by value, no further changes are necessary.
> If a pointer to the variable is passed, you could replace
> it with
> 
>   Py_ssize_t x64; int x;
>   foo(&x64);
>   x = x64;
> 
> Then use x as you did with the original code.

What if x64 has a 64-bit value ? How do you catch
and process the truncation error ?

To do this down-casting correctly, you'd need to write
code that does bounds checks and integrate into the
functions error handling.

>> You basically end up having to convert the whole extension
>> to Py_ssize_t.
> 
> That is not necessary. Can you give in example of a module
> where you think it is necessary?

If you want to port the extension to Py_ssize_t, this
is essentially the case.

You might want to take the unicodeobject.c file as
example.

>> Don't get me wrong: I don't mind doing this for the eGenix
>> extensions, but I'm worried about the gazillion other
>> useful extensions out there which probably won't get
>> upgraded in time to be used with Python 2.5.
> 
> I agree that it is not acceptable to require immediate
> whole-sale updates of every modules. However, I do
> believe that the number of modules that need any change
> at all is small, and that those modules can be modified
> with minimal effort to "get them working again,
> backwards-compatible" (i.e. with the only exception that
> they would fail if indices run above 2**31).
> 
>> I think all it takes is a set of new APIs for functions
>> that use Py_ssize_t as output parameter and which are
>> mapped to the regular API names if and only if the
>> extension #defines PY_SSIZE_T_CLEAN (or some other
>> capability flag).
> 
> That is not enough. You also need to deal with the
> function pointers that change.

We could use the type flags for these. much like we
do for the new style numbers.

> Also, others have rejected/expressed dislike of the
> PY_SIZE_T_CLEAN macro already, so they would probably
> dislike further hackishness in that direction.

That's easy to say... by not providing an easy way
to upgrade extensions, you basically just move the
requirement for hacks into the extensions.

I wouldn't consider that a good solution.

If you don't like the macro approach, why not simply
leave the two separate sets of APIs visible. Old
extensions would then use and link against the existing
APIs. All Py_ssize_t aware and compatible extensions
would use the new APIs instead. The old-style APIs
should then take care of the proper down-casting and
error reporting.

In Py3K we could then remove the old-style ones
and rename the new ones to the old names. Porting
an already Py_ssize_t compatible extension to the
renamed new style APIs would then become a simple
task of search&replace.

> Anyway, I have installed the PEP onsite, and
> added an "Open Issues" section, recording your
> comments.

Thanks.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::