[Python-Dev] Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...)

[M.-A. Lemburg]

Martin v. Löwis wrote:
> M.-A. Lemburg wrote:
>> What if x64 has a 64-bit value ? How do you catch
>> and process the truncation error ?
> 
> We were *both* discussing a scenario where no sizes
> exceed 2**31, right?

Right, but I don't see the point of each and every
extension having to go through these hoops when you
could add support for these checks (including error
reporting) to the few APIs in question, in particular
the PyArg_ParseTuple() API.

> Under such a scenario, this just won't happen.
> 
> OTOH, if you were discussing a scenario where sizes
> might exceed 2**31, then I don't see why you are worried
> about Py_ssize_t* parameters alone: Even
> 
>   PyString_Size()
> 
> might (of course!) return a value > 2**31 - so it
> is not just output parameters, but also return values.

Indeed - which is why I'm going to convert our tools to
Py_ssize_t throughout.

I don't expect this to happen any time soon for the ten or
twenty other 3rd party extensions we regularly use and this
would prevent an upgrade to Python 2.5.

> For more information, please read "Conversion guidelines"
> in
> 
> http://www.python.org/peps/pep-0353.html

BTW, the open issue should be reworded:

In particular, functions that currently take int* output
parameters should continue to do so. New functions should be
revised to enable Py_ssize_t* output arguments and preseve
backwards compatibility.

(This also includes the strategy to be used, so you can
remove the note on strategy)

Please also add a comment on the fact that extensions
which haven't been recompiled for the Python 2.5 API
will not get imported (the API_LEVEL check should consistently
fail for these instead of just issuing a warning).

>>> That is not necessary. Can you give in example of a module
>>> where you think it is necessary?
>>
>> If you want to port the extension to Py_ssize_t, this
>> is essentially the case.
>>
>> You might want to take the unicodeobject.c file as
>> example.
> 
> unicodeobject.c is not an extension. We were discussing
> existing extension modules.
> 
>> We could use the type flags for these. much like we
>> do for the new style numbers.
> 
> Would you like to write a specification for that?

Sure, if there's hope to get this into the code.

>> If you don't like the macro approach, why not simply
>> leave the two separate sets of APIs visible.
> 
> To simplify the porting.

Sorry, I lost you there:

I'm saying that two sets of APIs (one for int*, one
for Py_ssize_t*) will make porting easier since
updated code will only have to rename the APIs
used. Furthermore, it will allow to write code
that easily works in Python 2.1-2.4 as well as
Python 2.5.

>> All Py_ssize_t aware and compatible extensions
>> would use the new APIs instead. The old-style APIs
>> should then take care of the proper down-casting and
>> error reporting.
> 
> That is not possible. Some API does not support
> error reporting (like PyString_Size()). So callers
> don't expect it to fail.

I'm talking about the few APIs that use output parameters.
Those do support error reporting.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 13 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::