[Python-Dev] API bloat

[M.-A. Lemburg]

Mark Shannon wrote:
> M.-A. Lemburg wrote:
>> Mark Shannon wrote:
>>> Nick Coghlan wrote:
>>>> On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon <marks at dcs.gla.ac.uk>
>>>> wrote:
>>>>> Doing a search for the regex:  "PyAPI_FUNC\([^)]*\) *Py" in .h files,
>>>>> which should match API functions (functions starting _Py are
>>>>> excluded) gives
>>>>> the following result:
>>>>>
>>>>> Version  matches
>>>>> 3.0       717
>>>>> 3.1.3     728
>>>>> 3.2b2     743
>>>>>
>>>>> It would appear the API  bloat is real,
>>>>> not just an artefact of updated docs.
>>>> Since it doesn't account for #ifdef, a naive count like that isn't a
>>>> valid basis for comparison.
>>>>
>>> OK. How about this:
>>>
>>> egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h
>>> finds no matches.
>>>
>>> egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u
>>>
>>> This finds all matches and removes duplicates, so anything defined
>>> multiple time in branches of #ifdef blocks, will only be counted once.
>>>
>>> Version  matches
>>> 3.0       714
>>> 3.1.3     725
>>> 3.2b2     739
>>
>> Given these numbers, I don't think the subject line really
>> captures the problem accurately enough ... a 2% increase
>> in number of API function per release can hardly be called
>> API bloat :-)
>>
>>> So given, the revised numbers;
>>>
>>> The "what's new for 3.2" API section:
>>> http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes
>>>
>>> lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2.
>>
>> Could you identify the ones that are not yet documented ?
>>
>> That would be useful.
> 
> Here's the details:
> 
> The following API functions were removed from 3.1.3:
> 
> PyAST_Compile
> PyCObject_AsVoidPtr
> PyCObject_FromVoidPtr
> PyCObject_FromVoidPtrAndDesc
> PyCObject_GetDesc
> PyCObject_Import
> PyCObject_SetVoidPtr
> PyCode_CheckLineNumber
> Py_CompileStringFlags
> PyEval_CallObject
> PyOS_ascii_atof
> PyOS_ascii_formatd
> PyOS_ascii_strtod
> PyThread_exit_prog
> PyThread__PyThread_exit_prog
> PyThread__PyThread_exit_thread
> PyUnicode_SetDefaultEncoding
> 
> And the following were added to 3.2,
> of which only 2 are documented:
> 
> PyArg_ValidateKeywordArguments
> PyAST_CompileEx
> Py_CompileString
> Py_CompileStringExFlags
> PyErr_NewExceptionWithDoc    (documented)
> PyErr_SyntaxLocationEx
> PyErr_WarnFormat
> PyFrame_GetLineNumber
> PyImport_ExecCodeModuleWithPathnames
> PyImport_GetMagicTag
> PyLong_AsLongLongAndOverflow    (documented)
> PyModule_GetFilenameObject
> Py_SetPath
> PyStructSequence_GetItem
> PyStructSequence_NewType
> PyStructSequence_SetItem
> PySys_AddWarnOptionUnicode
> PySys_AddXOption
> PySys_FormatStderr
> PySys_FormatStdout
> PySys_GetXOptions
> PyThread_acquire_lock_timed
> PyType_FromSpec
> PyUnicode_AsUnicodeCopy
> PyUnicode_AsWideCharString
> PyUnicode_EncodeFSDefault
> PyUnicode_FSDecoder
> Py_UNICODE_strcat
> Py_UNICODE_strncmp
> Py_UNICODE_strrchr
> PyUnicode_TransformDecimalToASCII
> 
> For added confusion PySys_SetArgvEx is documented as
> new in 3.2, but exists in 3.1.3
> 
> That should keep someone busy ;)
> 
> Note that this only include functions.
> The API also includes a number of macros such as
> Py_False and Py_RETURN_FALSE, types ,
> and data like PyBool_Type.
> 
> I've not tried to analyse any of these.

Thanks.

I opened http://bugs.python.org/issue11173 for this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 10 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/