[Python-Dev] Split unicodeobject.c into subfiles

[Victor Stinner]

> Such a restructuring should not result in compilers
> no longer being able to optimize code by inlining functions
> in one of the most important basic types we have in Python 3.

I agree that performances are important. But I'm not convinced than
moving functions has a real impact on performances, not that such
issues cannot be fixed.

I tried to limit changes impacting performances. Inlining is (only?)
interesting for short functions. PEP 393 introduces many macros for
this. I also added some "Fast" functiions
(_PyUnicode_FastCopyCharacters() and _PyUnicode_FastFill()) which
don't check parameters and do the real work. I don't think that it's
really useful to inline _PyUnicode_FastFill() in the caller for
example.

I will check performances of all str methods. For example, str.count()
is now calling PyUnicode_Count() instead of the static count().
PyUnicode_Count() adds some extra checks, some of them are not
necessary, and it's not a static function, so it cannot(?) be inlined.
But I bet that the overhead is really low.

Note: Since GCC 4.5, Link Time Optimization are possible. I don't know
if GCC is able to inline functions defined in different files, but C
compilers are better at each release.

--

I will check the impact of performances on _PyUnicode_Widen() and
_PyUnicode_Putchar(), which are no more static. _PyUnicode_Widen() and
_PyUnicode_Putchar() are used in Unicode codecs when it's more
expensive to compute the exact length and maximum character of the
output string. These functions are optimistic (hope that the output
will not grow too much and the string is not "widen" too much times,
so it should be faster for ASCII).

I implemented a similar approach in my PyUnicodeWriter API, and I plan
to reuse this API to simplify the API. PyUnicodeWriter uses some macro
to limit the overhead of having to check before each write if we need
to enlarge or widen the internal buffer, and allow to write directly
into the buffer using low level functions like PyUnicode_WRITE.

I also hope a performance improvement because the PyUnicodeWriter API
can also overallocate the internal buffer to reduce the number of
calls to realloc() (which is usually slow).

> Also note that splitting the file in multiple smaller ones will
> actually create more maintenance overhead, since patches will
> likely no longer be easy to merge from 3.3 to 3.4.

I'm a candidate to maintain unicodeobject.c. In your check
unicodeobject.c (recent) history, I'm one of the most active developer
on this file since two years (especially in 2012). I'm not sure that
merges on this file are so hard.

Victor