[Python-ideas] discouraging direct use of the C-API

[Andrew Barnert]

On May 6, 2015, at 15:00, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> 
>> On Wed, May 6, 2015 at 1:59 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>>> On May 6, 2015, at 09:23, Eric Snow <ericsnowcurrently at gmail.com> wrote:
>>> 
>>> A big blocker to making certain sweeping changes to CPython (e.g.
>>> ref-counting) is compatibility with the vast body of C extension
>>> modules out there that use the C-API.  While there are certainly
>>> drastic long-term solutions to that problem, there is one thing we can
>>> do in the short-term that would at least get the ball rolling.  We can
>>> put a big red note at the top of every page of the C-API docs that
>>> encourages folks to either use CFFI or Cython.
>> 
>> Does this mean you also want to discourage boost::python, SIP, SWIG, etc., which as far as I know come down to automatically building C API extensions, and would need to be completely rewritten if you wanted to make them work a different way?
> 
> Not really.  I mentioned CFFI and Cython specifically because they are
> the two that kept coming up in previous discussions related to
> discouraging use of the C-API.  If C extensions were always generated
> using tools, then only tools would have to adapt to (drastic) changes
> in the C-API.  That would be a much better situation than the status
> quo since it drastically reduces the impact of changes.

OK, that makes sense to me. Even if there are a dozen wrappers and wrapper generators (and I think it's more like 4 or 5...), and we had to get buy-in from all of them (or get buy-in from most of them and reluctantly decide to screw over the last one), that's still orders of magnitude easier than getting buy-in from (or screw over) the 69105 people who are currently maintaining or building a C API extension, so it's still a huge win.

I'm not sure it would do nearly enough, at least not for a long time (how many of the current top 100 projects on PyPI use C API extensions and would be non-trivial to rewrite?), but obviously you can make the point that if we don't do anything, we'll _never_ get there.