[Python-bugs-list] [ python-Bugs-723540 ] slots broken in 2.3a with ("dict", )

Sun, 25 May 2003 06:40:31 -0700

Bugs item #723540, was opened at 2003-04-18 07:35
Message generated for change (Comment added) made by pje
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=723540&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Douglas Napoleone (derivin)
Assigned to: Nobody/Anonymous (nobody)
Summary: __slots__ broken in 2.3a with ("__dict__", )

Initial Comment:
I LOVE using __slots__. I love using properties.
I made my own extension and metaclass so I could 
define type safe properties in classes. I could stick 
properties on a class and have them accessed 
seperatly from slots (because they are properties and 
not attributes). The properties call setter/getters and 
store the data in __dict__ (a cheezy means of making 
pickle still work with no effort). thus I had __slots__ = 
('__dict__', ) in my base class. this worked great in 2.2 
and 2.2.2.  Im migrating all my modules to 2.3. I just 
fixed a bug in my code that was due to a typo. Thats 
when I realized that __slots__ was no longer working in 
2.3a. I cant find anything on this change in behavior.

if slots contains "__dict__" slots is turned OFF!!!
simplified version of the bug:

$ python
Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> class foo(object):
...   __slots__ = ("__dict__", )
...
>>> a = foo()
>>> a.bad = 3
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'foo' object has no attribute 'bad'
>>> ^Z

$ python23
Python 2.3a2 (#39, Feb 19 2003, 17:58:58) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> class foo(object):
...   __slots__ = ("__dict__", )
...
>>> a = foo()
>>> a.bad = 3
>>> dir(a)
['__class__', '__delattr__', '__dict__', '__doc__', '__getattr
ibute__', '__hash_
_', '__init__', '__module__', '__new__', '__reduce__', '__re
duce_ex__', '__repr_
_', '__setattr__', '__slots__', '__str__', 'bad']
>>> ^Z

----------------------------------------------------------------------

Comment By: Phillip J. Eby (pje)
Date: 2003-05-25 13:40

Message:
Logged In: YES 
user_id=56214

>From discussion on Python-Dev, this behavior is a feature,
not a bug.  Or to be precise, the 2.2 behavior of __slots__
is a bug, or at least a misfeature.  If an object has a
__dict__, it is supposed to allow setting arbitrary
attributes.  The behavior was fixed in 2.3 (see
http://www.python.org/2.2.3/descrintro.html).

A backward-compatible solution to your problem is to create
a slot with a different name than __dict__, store a
dictionary in it, and add __setstate__/__getstate__ methods
to your base class that operate on it.  This should work
correctly for both 2.2 and 2.3, and preserve the fixed
namespace behavior you're looking for, as long as all
subclasses set __slots__ to an empty list.  (You can always
have your metaclass do that in its __new__ method, if you
don't want to have to remember to do it manually.  Just be
sure not to overwrite __slots__ if present, since your base
class will need to set up the slot for the "hidden" dictionary.)

Hmm.  I just read all the followups in more detail and
noticed you are already using a __fakedict__ slot, but refer
to the pickle handling as "yuck"; note that this:

class MyBase(object):

    __slots__ = '__fakedict__'

    def __getstate__(self):
        return self.__fakedict__

    def __setstate__(self,state):
        self.__fakedict__ = state

...should be all you need to get pickling working correctly
under 2.2, assuming that you already have an __init__ that
sets __fakedict__ to a dictionary.  Since in 2.2, a __dict__
slot was not set to a dictionary automatically, I presume
you must have this code already.

For Python 2.3, the __getstate__/__setstate__ methods are
*not* required, as long as you use the new pickling protocol
(protocol #2) See the 2.3 pickle module docs.  Note that the
new protocol is also more space-efficient for new-style
classes, so you may want to upgrade to using it, anyway.

----------------------------------------------------------------------

Comment By: Douglas Napoleone (derivin)
Date: 2003-04-21 18:31

Message:
Logged In: YES 
user_id=541557

from that followup of mine, you can clearly see why I use 
__slots__; sorry for the bad spelling and worse grammar.

I *think* I have a solution, but I an not really sure if it is the 
proper behavior for __slots__.

I can not find any documentation, PEP, bug, request, or 
discussion on how __slots__ should behave in boundry 
cases such as "__dict__", "__weakref__", etc. 

Should these be an error if used?
Are they assumed?
what behavior does having __dict__ as an allowed attribute 
have? (Does it allow access to the proxy __dict__ and use 
proper __slots__ conventions and attribute checking? does it 
circumvent __slots__, i.e. a back door?)

__slots__ and inheritance is not clearly specified either.
If these things are documented, please flame me with the 
links.

----------------------------------------------------------------------

Comment By: Douglas Napoleone (derivin)
Date: 2003-04-18 18:44

Message:
Logged In: YES 
user_id=541557

Well looking deeper at the problem, is seems that __dict__ 
and __weakref__ are called out as special in 
Objects/typeobject.c at some point after the 2.2.2 release.

looking at teh code, I would expect the followint ot be 
triggered:

   if (!may_add_dict || add_dict) { 
          PyErr_SetString(PyExc_TypeError, 
                                    "__dict__ slot disallowed: " 
                                    "we already got one"); 
          goto bad_slots; 
    }

Ignoring the attrocious goto logic in this section of code, I 
would have expected to see this triggered.
I can understand the IDEA behind always having __slots__ 
on all classes/types and setting it to contain "__dict__" as 
teh equivolent of not having slots... But the code does not 
look like it implements this, and I dont see how this is even 
working that way yet. Has anyone heard anything about 
this???

(for now in my code I am having to use __fakedict__ and 
override all the pickle handling (yuck).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=723540&group_id=5470

[Python-bugs-list] [ python-Bugs-723540 ] __slots__ broken in 2.3a with ("__dict__", )

[Python-bugs-list] [ python-Bugs-723540 ] slots broken in 2.3a with ("dict", )