[Python-Dev] Making PyInterpreterState an opaque type

[Antoine Pitrou]

Le 21/02/2019 à 12:58, Paul Moore a écrit :
> On Thu, 21 Feb 2019 at 11:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> On Thu, 21 Feb 2019 12:13:51 +0100
>> Victor Stinner <vstinner at redhat.com> wrote:
>>>
>>> Premature optimization is the root of all evil. Most C extensions use
>>> premature optimization
>>
>> How do you know it's premature?  Some extensions _are_ meant for speed.
> 
> Extensions that need to squeeze every bit of speed out of the C API
> are the exception rather than the rule. Making it easier for extension
> authors to naturally pick portable options seems reasonable to me.

Actually, it would be interesting to have some kind of survey of C
extensions (through random sampling? or popularity?) to find out why the
developers had to write a C extension in the first place and what their
concerns are.

Intuitively there are three categories of C extensions:

1. extensions whose entire purpose is performance (this includes all
scientific computing C extensions - including Numpy or Pandas -, but
also C accelerators such as SQLAlchemy's or simplejson's)

2. extensions wrapping third-party APIs that are not
performance-critical.  If you are exposing a wrapper to e.g. the
posix_spawn() system call, probably the wrapper performance isn't very
important.

3. extensions wrapping third-party APIs that are performance-critical.
For example, in a database wrapper, it's important that your native
DB-to-Python and Python-to-native DB conversions are as fast as
possible, because users are going to convert a *lot* of data.

Note that category 2 may be taken care of by ctypes or cffi.

Regards

Antoine.