[Python-Dev] API bloat

[Mark Shannon]

M.-A. Lemburg wrote:
> Mark Shannon wrote:
>> Nick Coghlan wrote:
>>> On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon <marks at dcs.gla.ac.uk>
>>> wrote:
>>>> Doing a search for the regex:  "PyAPI_FUNC\([^)]*\) *Py" in .h files,
>>>> which should match API functions (functions starting _Py are
>>>> excluded) gives
>>>> the following result:
>>>>
>>>> Version  matches
>>>> 3.0       717
>>>> 3.1.3     728
>>>> 3.2b2     743
>>>>
>>>> It would appear the API  bloat is real,
>>>> not just an artefact of updated docs.
>>> Since it doesn't account for #ifdef, a naive count like that isn't a
>>> valid basis for comparison.
>>>
>> OK. How about this:
>>
>> egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h
>> finds no matches.
>>
>> egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u
>>
>> This finds all matches and removes duplicates, so anything defined
>> multiple time in branches of #ifdef blocks, will only be counted once.
>>
>> Version  matches
>> 3.0       714
>> 3.1.3     725
>> 3.2b2     739
> 
> Given these numbers, I don't think the subject line really
> captures the problem accurately enough ... a 2% increase
> in number of API function per release can hardly be called
> API bloat :-)
> 
>> So given, the revised numbers;
>>
>> The "what's new for 3.2" API section:
>> http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes
>> lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2.
> 
> Could you identify the ones that are not yet documented ?
> 
> That would be useful.

Here's the details:

The following API functions were removed from 3.1.3:

PyAST_Compile
PyCObject_AsVoidPtr
PyCObject_FromVoidPtr
PyCObject_FromVoidPtrAndDesc
PyCObject_GetDesc
PyCObject_Import
PyCObject_SetVoidPtr
PyCode_CheckLineNumber
Py_CompileStringFlags
PyEval_CallObject
PyOS_ascii_atof
PyOS_ascii_formatd
PyOS_ascii_strtod
PyThread_exit_prog
PyThread__PyThread_exit_prog
PyThread__PyThread_exit_thread
PyUnicode_SetDefaultEncoding

And the following were added to 3.2,
of which only 2 are documented:

PyArg_ValidateKeywordArguments
PyAST_CompileEx
Py_CompileString
Py_CompileStringExFlags
PyErr_NewExceptionWithDoc    (documented)
PyErr_SyntaxLocationEx
PyErr_WarnFormat
PyFrame_GetLineNumber
PyImport_ExecCodeModuleWithPathnames
PyImport_GetMagicTag
PyLong_AsLongLongAndOverflow    (documented)
PyModule_GetFilenameObject
Py_SetPath
PyStructSequence_GetItem
PyStructSequence_NewType
PyStructSequence_SetItem
PySys_AddWarnOptionUnicode
PySys_AddXOption
PySys_FormatStderr
PySys_FormatStdout
PySys_GetXOptions
PyThread_acquire_lock_timed
PyType_FromSpec
PyUnicode_AsUnicodeCopy
PyUnicode_AsWideCharString
PyUnicode_EncodeFSDefault
PyUnicode_FSDecoder
Py_UNICODE_strcat
Py_UNICODE_strncmp
Py_UNICODE_strrchr
PyUnicode_TransformDecimalToASCII

For added confusion PySys_SetArgvEx is documented as
new in 3.2, but exists in 3.1.3

That should keep someone busy ;)

Note that this only include functions.
The API also includes a number of macros such as
Py_False and Py_RETURN_FALSE, types ,
and data like PyBool_Type.

I've not tried to analyse any of these.

Mark.

> 
> Thanks,