[Python-Dev] C-level duck typing

[Dag Sverre Seljebotn]

On 05/16/2012 11:50 AM, "Martin v. Löwis" wrote:
>> Agreed in general, but in this case, it's really not that easy. A C
>> function call involves a certain overhead all by itself, so calling into
>> the C-API multiple times may be substantially more costly than, say,
>> calling through a function pointer once and then running over a
>> returned C
>> array comparing numbers. And definitely way more costly than running over
>> an array that the type struct points to directly. We are not talking
>> about
>> hundreds of entries here, just a few. A linear scan in 64 bit steps over
>> something like a hundred bytes in the L1 cache should hardly be
>> measurable.
>
> I give up, then. I fail to understand the problem. Apparently, you want
> to do something with the value you get from this lookup operation, but
> that something won't involve function calls (or else the function call
> overhead for the lookup wouldn't be relevant).

In our specific case the value would be an offset added to the 
PyObject*, and there we would find a pointer to a C function (together 
with a 64-bit signature), and calling that C function (after checking 
the 64 bit signature) is our final objective.

> I still think this is out of scope for python-dev. If this is something
> you want to be able to do for Python 2.4 as well, then you don't need
> any change to Python - you can do whatever you come up with for all
> Python versions, no need to (or point in) changing Python 3.4 (say).

We can go ahead and hijack tp_flags bit 22 to make things work in 
existing versions. But what if Python 3.8 then starts using that bit for 
something else?

> As this is apparently only relevant to speed fanatics, too, I suggest
> that you check how fast PyPI works for you. Supposedly, they have very
> efficient lookup procedures, supported by the JIT. If this doesn't work
> for some reason, I suggest that you'll have to trade speed for
> convenience: a compile-time fixed layout will beat any dynamic lookup
> any time. Just define a common base class, and have all interesting
> types inherit from it.

Did you mean PyPy? Me and Stefan are Cython developers, so that's kind 
of our angle... And I'm a Cython developer because it solves a practical 
need (in my case in scientific computation), not because I think it's 
that beautiful. PyPy won't work for me (let's not go down that road now...)

Defining a common base class is what NumPy already does, and Cython 
would be forced to without this proposal. We just think it has 
significant disadvantages and were looking for something else.

Dag