[Python-Dev] Code to prevent modification on builtins classes also abusively (IMHO) prevents modifications on extensions modules, some ideas on this.

Guido van Rossum guido at python.org
Fri Nov 7 12:16:57 EST 2003


> I look into the archives and didn't see any debate on the question, hope 
> I didn't miss something.
> 
> My point concerns limitations on extensions module due to checks aiming 
> the builtins.
> The main point is settable extension classes.
> In Python code there is some checks against TPFLAGS_HEAPTYPE, extension 
> modules should'nt have this flag, so the normal type->tp_setattro doesnt 
> allow the user to
> set new attributes on your extension classes. There is a way around, 
> write a special MetaClass which redefine setattr.

Or you can create a Python subclass that doesn't add any features but
inherits from your extension class -- the user can set attributes on
the Python class to their heart's content and everything will work as
needed.

> In the extension module I'm writing (I'm porting some Python code to 
> Python-C for speed issues) the user can set attributes and slots on my 
> classes.
> What I need is the complete type->tp_setattro behaviour, without the 
> check. I didn't see a way to have this behaviour using only Python API 
> (is rereadying the type a work around ?), so I copy paste all the code 
> to make update_slots work (ouch 2500 lines).

A much simpler approach would be to have a metaclass whose tp_setattro
clears the HEAPTYPE flag, calls type->tp_setattro, and then restores
the HEAPTYPE flag.  Yes, that might be considered cheating, but so is
copying 2500 lines of code. :-)

> This is now almost working, every kind of attribute can be set but the 
> __setattr__ one, the hackcheck prevents the user from calling another 
> __setattr__ from its new setattr:
> example of my extension class hierachy:
> Class A(object)
> Class B(A)
> 
> In the extension, there is a tp->setattro on B, if the user want to 
> redefine it, he can't call the A __setattr__:
> def myBSetattr(self,k,v):
>     super(B,self).__setattr__(k,v)
>     ## Do here my special stuff
> This won't work, the hachcheck will see some kind of hack here, 'you 
> cant' call the A.__setattr__ function from a B object' :).

I don't understand this -- does any of my suggestions above handle it?

> First question, Is there a known way around ?
> 
> Possible Improvments :
> 
> In the python code there is in function function checks to see if you 
> are not modying builtins classes, unfortunately this code is also 
> concerning extension modules.
> I think the Heaptype flag is abusively used in differents cases mostly, 
> in type_setattro, object_set_bases, object_set_classes, the checks have 
> nothing to do with the HeapType true definition as stated in the 
> comments in Include/Object.h , it is used, I think, only because this is 
> the only one that makes a difference between builtins and user classes. 
> Unfortunately with  this flag extension classes  fall into the 
> 'builtin'  part.
> 
> A way to solve the problem without backward compatibility problems, 
> would be to have a new TPFLAGS_SETABLE flag, defaulting to 0 for 
> builtins/extension classes and 1 for User,Python classes. This flag 
> would be check in place of the heaptype one when revelant.
> 
> I'm ready to write the code for this if there is some positive votes, 
> won't bother if everybody is against it.

This seems to be a reasonable suggestion, however I want you to
consider what happens if you are using multiple interpreters.  When
you set a function attribute builtin or extension type, the function
references to the environment of the interpreter where it was defined,
but it is visible from all interpreters.  This is likely not what you
want, and that's why the HEAPTYPE flag exists.  I would strongly
advise using my first suggestion above (derive a class in Python)
rather than mess with HEAPTYPE.

--Guido van Rossum (home page: http://www.python.org/~guido/)



More information about the Python-Dev mailing list