Extending Python: rewriting a single method in C

[Alex Martelli]

"Jacek Generowicz" <jmg at ecs.soton.ac.uk> wrote in message
news:g0ofv6xbj4.fsf at scumbag.ecs.soton.ac.uk...
    [snip]
> AM> > Yes, but I was trying to avoid the function call overhead
> AM> > (particularly as one of the functions I need to rewrite is very
short
> AM> > indeed).
> AM>
> AM> No way to do that -- the function call overhead is there even
> AM> if you call directly from Python to the C extension.
>
> But isn't the function call overhead greater for calling python
> functions than it is for calling the C extensions ?

On my old box (Pentium-II 266) I measure about 1 microsecond
difference per call, yes: a loop of 1,000 'pass' takes about
1.1 msec, one of 1,000 calls to an empty built-in (which
ignores its args and returns None) about 2.2, one of 1,000
calls to a similarly-empty (takes *args, body is 'pass')
Python-coded function about 3.2 (that's in WinNT4 SP6, with
Python 2).  Much of that is overhead of global-name lookup,
of course (for uniformity I assigned either kind of function
to a global variable and called through that); moving it to
a local name is an important optimization:

def timecals(afun):
    start = time.clock()
    for i in range(1000):
        afun()
    stend = time.clock()
    print stend-start

Here, it's about 1.8 msec if the empty C-coded function is
passed as the argument, about 2.8 if the Python one is --
so, the global name lookup overhead was about 0.4 msec in
each case (and the rest of the empty-loop overhead another
0.7 msec or so, leaving about 0.7 vs 1.7 for C vs Python
coded function's _sheer_ call overhead from Python).

If microseconds (small fractions thereof on more modern
machines, actually) are so crucial, then, yes, you can
shave some further tiny fractions this way.


> AM> If the function/method is called from a few places, you might
> AM> manually inline in there, maybe.
>
> Not sure I understand what you mean exactly. Are you suggesting that I
> call an extension function wherever I would previously have called and
> instance method ?

Yes.

> How do I get at the members of a class in C if that class was defined
> in python ?

By passing the reference to the instance to the function, and
receiving it on the C side as a PyObject* -- you can then call
methods and access attributes on it from C, just as you could
from Python.  See the "O" format code in the documentation of
PyArg_ParseTuple (1.7 in the "Extending and Embedding" manual).


> JE> In fast_a and fast_b, you can get arguments by using the ParseTuple
string
> JE> "(Oii)" and then use PyObject_SetAttrString and PyObject_GetAttrString
> JE> to store or retrieve attributes (including, if necessary, instance
> JE> methods) from the instance by their name.
>
> Is there an organized reference where I can find the specifications of
> all these functions ?  All I have come across shows example code, but
> no full descriptions of what can/must be done.

It's somewhat disorganized.  I think Beazley's "Python Essential
Reference" (New Riders) is your best bet among current books (but
it only covers Python 1.5.2).


> Maybe some kind soul could show me what the correct code for a minimal
> extension type is ?
>
> For example, I tried to write an extension type equivalent to
>
> class foo:
>     def __init__ ( self, a ):
> self.a = a
>     def show ( self ):
> print a
>
> My various attempts, almost work, but there is always something not
> quite right.

Getting equivalence to this is not going to be easy -- the
type/class separation hits hard, and calling Python's "print"
statement from C is not quite trivial either.


> Incidentally, why does the xxobject.c file in the Objects directory of the
> Python distribution contain no initxxobject function ?

Hmmm, because it's meant for a "real builtin type" (in a hypothetical
modification of Python) rather than for an extension...?


Alex