[Python-Dev] Need advice, maybe support

[Guido van Rossum]

> Guido van Rossum wrote:
> 
> [me, about how to add _nr cfunction versions in a compatible way]
> 
> > I don't think we can just add an extra field to PyMethodDef, because
> > it would break binary incompatibility.  Currently, in most cases, a
> > 3r party extension module compiled for an earlier Python version can
> > still be used with a later version.  Because PyMethodDef is used as an
> > array, adding a field to it would break this.
> 
> Bad news. I hoped you would break binary compatibility between
> major versions (like from 2.2 to 2.3), but well, now I also
> understand why there are so many flags in typeobjects :-)
> 
> > I have less of a problem with extending PyTypeObject, it grows all the
> > time and the tp_flags bits tell you how large the one you've got is.
> > (I still have some problems with this, because things that are of no
> > use to the regular Python core developers tend to either confuse them,
> > or be broken on a regular basis.)
> 
> For the typeobjects, I'm simply asking for reservation
> of a bit number. What I used is
> 
> #ifdef STACKLESS
> #define Py_TPFLAGS_HAVE_CALL_NR (1L<<15)
> #else
> #define Py_TPFLAGS_HAVE_CALL_NR 0
> #endif
> 
> but I think nobody needs to know about this, and maybe
> it is better (requiring no change of Python) if I used
> a bit from the higer end (31) or such?
> 
> > Maybe you could get away with defining an alternative structure for
> > PyMethodDef and having a flag in tp_flags say which it is; there are
> > plenty of unused bits and I don't mind reserving one for you.  Then
> > you'd have to change all the code that *uses* tp_methods, but there
> > isn't much of that; in fact, the only place I see is in typeobject.c.
> 
> The problem is that I need to give extra semantics to
> existing objects, which are PyCFunction objects.
> I think putting an extra bit into the type object
> doesn't help, unless I use a new type. But then I don't
> need the flag.
> An old extension module which is loaded into my Python
> will always use my PyCFunction, since this is always
> borrowed.
> 
> > If this doesn't work for you, maybe you could somehow fold the two
> > implementation functions into one, and put something special in the
> > argument list to signal that the non-recursive version is wanted?
> > (Thinking aloud here -- I don't know exactly what the usage pattern of
> > the nr versions will be.)
> 
> This is hard to do. I'm adding _nr versions to existing
> functions, and I don't want to break their parameter lists.
> 
> 
> Ok, what I did is rather efficient, quite a bit ugly of
> course, but binary compatible as much as possible.
> It required to steal some bits of ml_flags as a small
> integer, which are interpreted as "distance to my sibling".
> I'm extending the MethodDef arrays in a special way
> by just adding some extra records without name fields
> at the end of the array, which hold the _nr pointers.
> 
> An initialization functions initializes the small integer
> in ml_flags with the distance to this "sibling", and
> the nice thing about this is that it will never fail
> if not initialized:
> A distance of zero gives just the same record.
> 
> So what I'm asking for in this case is a small number
> of bits of the ml_flags word which will not be used,
> otherwise.
> 
> Do you think the number of bits in ml_flags might ever
> grow beyond 16, or should I just assume that I can
> safely abuse them?
> 
> thanks a lot -- chris

It's better to reserve bits explicitly.  Can you submit a patch to SF
that makes reservations of the bits you need?  All they need is a
definition of a symbol and a comment explaining what it is for;
"reserved for Stackless" is fine.

--Guido van Rossum (home page: http://www.python.org/~guido/)