[Python-Dev] Need advice, maybe support

[Christian Tismer]

Guido van Rossum wrote:

[me, about how to add _nr cfunction versions in a compatible way]

> I don't think we can just add an extra field to PyMethodDef, because
> it would break binary incompatibility.  Currently, in most cases, a
> 3r party extension module compiled for an earlier Python version can
> still be used with a later version.  Because PyMethodDef is used as an
> array, adding a field to it would break this.

Bad news. I hoped you would break binary compatibility between
major versions (like from 2.2 to 2.3), but well, now I also
understand why there are so many flags in typeobjects :-)

> I have less of a problem with extending PyTypeObject, it grows all the
> time and the tp_flags bits tell you how large the one you've got is.
> (I still have some problems with this, because things that are of no
> use to the regular Python core developers tend to either confuse them,
> or be broken on a regular basis.)

For the typeobjects, I'm simply asking for reservation
of a bit number. What I used is

#ifdef STACKLESS
#define Py_TPFLAGS_HAVE_CALL_NR (1L<<15)
#else
#define Py_TPFLAGS_HAVE_CALL_NR 0
#endif

but I think nobody needs to know about this, and maybe
it is better (requiring no change of Python) if I used
a bit from the higer end (31) or such?

> Maybe you could get away with defining an alternative structure for
> PyMethodDef and having a flag in tp_flags say which it is; there are
> plenty of unused bits and I don't mind reserving one for you.  Then
> you'd have to change all the code that *uses* tp_methods, but there
> isn't much of that; in fact, the only place I see is in typeobject.c.

The problem is that I need to give extra semantics to
existing objects, which are PyCFunction objects.
I think putting an extra bit into the type object
doesn't help, unless I use a new type. But then I don't
need the flag.
An old extension module which is loaded into my Python
will always use my PyCFunction, since this is always
borrowed.

> If this doesn't work for you, maybe you could somehow fold the two
> implementation functions into one, and put something special in the
> argument list to signal that the non-recursive version is wanted?
> (Thinking aloud here -- I don't know exactly what the usage pattern of
> the nr versions will be.)

This is hard to do. I'm adding _nr versions to existing
functions, and I don't want to break their parameter lists.


Ok, what I did is rather efficient, quite a bit ugly of
course, but binary compatible as much as possible.
It required to steal some bits of ml_flags as a small
integer, which are interpreted as "distance to my sibling".
I'm extending the MethodDef arrays in a special way
by just adding some extra records without name fields
at the end of the array, which hold the _nr pointers.

An initialization functions initializes the small integer
in ml_flags with the distance to this "sibling", and
the nice thing about this is that it will never fail
if not initialized:
A distance of zero gives just the same record.

So what I'm asking for in this case is a small number
of bits of the ml_flags word which will not be used,
otherwise.

Do you think the number of bits in ml_flags might ever
grow beyond 16, or should I just assume that I can
safely abuse them?

thanks a lot -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/