Hello all, I've been meta-reflecting a lot lately: reflecting on reflection. My recent post on __slots__ not being picklable (and the resounding lack of response to it) inspired me to try my hand at channeling Guido and reverse- engineer some of the design decisions that went into the new-style class system. Unfortunately, the more I dug into the code, the more philosophical my questions became. So, I've written up some questions that help lay bare some of basic design questions that I've been asking myself and that you should be aware of. While there are several subtle issues I could raise, I do want some feedback on some simple and fundamental ones first. Please don't disqualify yourself from commenting because you haven't read the code or used the new features yet. I've written my examples assuming only a basic and cursor understanding of the new Python 2.2 features. [In this discussion I am only going to talk about native Python classes, not C-extension or native Python types (e.g., ints, lists, tuples, strings, cStringIO, etc.)] 1) Should class instances explicitly/directly know all of their attributes? Before Python 2.2, all object instances contained a __dict__ attribute that mapped attribute names to their values. This made pickling and some other reflection tasks fairly easy. e.g.: class Foo: def __init__(self): self.a = 1 self.b = 2 class Bar(Foo): def __init__(self): Foo.__init__(self) self.c = 3 bar = Bar() print bar.__dict__ > {'a': 1, 'c': 3, 'b': 2} I am aware that there are situations where this simple case does not hold (e.g., when implementing __setattr__ or __getattr__), but let's ignore those for now. Rather, I will concentrate on how this classical Python idiom interacts with the new slots mechanism. Here is the above example using slots: e.g.: class Foo(object): __slots__ = ['a','b'] def __init__(self): self.a = 1 self.b = 2 class Bar(Foo): __slots__ = ['c'] def __init__(self): Foo.__init__(self) self.c = 3 bar = Bar() print bar.__dict__ > AttributeError: 'Bar' object has no attribute '__dict__' We can see that the class instance 'bar' has no __dict__ attribute. This is because the slots mechanism allocates space for attribute storage directly inside the object, and thus does not use (or need) a per-object instance dictionary to store attributes. Of course, it is possible to request that a per-instance dictionary by inheriting from a new-style class that does not list any slots. e.g. continuing from above: class Baz(Bar): def __init__(self): Bar.__init__(self) self.d = 4 self.e = 5 baz = Baz() print baz.__dict__ > {'e': 5, 'd': 4} We have now created a class that has __dict__, but it only contains the attributes not stored in slots! So, should class instances explicitly know their attributes? Or more precisely, should class instances always have a __dict__ attribute that contains their attributes? Don't worry, this does not mean that we cannot also have slots, though it does have some other implications. Keep reading... 2) Should attribute access follow the same resolution order rules as methods? class Foo(object): __slots__ = ['a'] self.a def __init__(self): self.a = 1 class Bar(Foo): __slots__ = ('a',) def __init__(self): Foo.__init__(self) self.a = 2 bar = Bar() print bar.a > 2 print super(Bar,bar).a # this doesn't actually work > 2 or 1? Don't worry -- this isn't a proposal and no, this doesn't actually work. However, the current implementation only narrowly escapes this trap: print bar.__class__.a.__get__(bar) > 2 print bar.__class__.__base__.a.__get__(bar) > AttributeError: a Ok, let me explain what just happened. Slots are implemented via the new descriptor interface. In short, descriptor objects are properties and support __get__ and __set__ methods. The slot descriptors are told the offset within an object instance the PyObject* lives and proxy operations for them. So getting and setting slots involves: # print bar.a a_descr = bar.__class__.a print a_descr.__set__(bar) # bar.a = 1 a_descr = bar.__class__.a a_descr.__set__(bar, 1) So, above we get an attribute error when trying to access the 'a' slot from Bar since it was never initialized. However, with a little ugliness you can do the following: # Get the descriptors for Foo.a and Bar.a a_foo_descr = bar.__class__.__base__.a a_bar_descr = bar.__class__.a a_foo_descr.__set__(bar,1) a_bar_descr.__set__(bar,2) print bar.a > 2 print a_foo_descr.__get__(bar) > 1 print a_bar_descr.__get__(bar) > 2 In other words, the namespace for slots is not really flat, although there is no simple way to access these hidden attributes since method resolution order rules are not invoked by default. 3) Should __slots__ be immutable? The __slots__ attribute of a new-style class lists all of the slots defined by that class. It is represented as whatever sequence type what given when the object was declared: print Foo.__slots__ > ['a'] print Bar.__slots__ > ('a',) This allows us to do things like: Foo.__slots__.append('b') foo = Foo() foo.b = 42 > AttributeError: 'Foo' object has no attribute 'b' So modifying the slots does not do what one may expect. This is because slot descriptors and the space for slots are only allocated when the classes are created (i.e., when they are inherited from 'object', or from an object that descends from 'object'). 4) Should __slots__ be flat? bar.__slots__ only lists the slots specifically requested in bar, even though it inherits from 'foo', which has its own slots. Which would be the preferable behavior? class Foo(object): __slots__ = ('a','b') class Bar(object): __slots__ = ('c','d') print Bar.__slots__ > ('c','d') # current behavior or > ('a','b','c','d') # alternate behavior Clearly, this issue goes back to the ideas addressed in question 1. If slot descriptors are not stored in a per-instance dictionary, then the assumptions on how to do object reflection must change. However, which version of the following code do you prefer to print all attributes of a given object: Old style or if descriptors are stored in obj.__dict__: if hasattr(obj,'__dict__'): print ''.join([ '%s=%s' % nameval for nameval in obj.__dict__ ]) Currently in Python 2.2 (and still not quite correct): def print_slot_attrs(obj,cls=None): if not cls: cls = obj.__class__ for name,obj in cls.__dict__.items() if str(type(obj)) == "<type 'member_descriptor'>": if hasattr(obj, name): print "%s=%s" % (name,getattr(obj, name)) for base in cls.__bases__: print_slot_attrs(obj,base) if hasattr(obj,'__dict__'): print [ '%s=%s' % nameval for nameval in obj.__dict__ ] print_slot_attrs(obj) Flat and immutable slot namespace: a = [ '%s=%s' % nameval for nameval in obj.__dict__ ] a += [ '%s=%s' % (name,val) for name,val in obj.__slots__ \ if hasattr(obj, name) ] print ''.join(a) So, which one of these do you want to support or explain to a new user? Thanks, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
I haven't even finished reading this yet. This is good stuff! --- Kevin Jacobs <jacobs@penguin.theopalgroup.com> wrote:
Hello all,
I've been meta-reflecting a lot lately: reflecting on reflection.
My recent post on __slots__ not being picklable (and the resounding lack of response to it) inspired me to try my hand at channeling Guido and reverse- engineer some of the design decisions that went into the new-style class system. Unfortunately, the more I dug into the code, the more philosophical my questions became. So, I've written up some questions that help lay bare some of basic design questions that I've been asking myself and that you should be aware of.
While there are several subtle issues I could raise, I do want some feedback on some simple and fundamental ones first. Please don't disqualify yourself from commenting because you haven't read the code or used the new features yet. I've written my examples assuming only a basic and cursor understanding of the new Python 2.2 features.
[In this discussion I am only going to talk about native Python classes, not C-extension or native Python types (e.g., ints, lists, tuples, strings, cStringIO, etc.)]
1) Should class instances explicitly/directly know all of their attributes?
Before Python 2.2, all object instances contained a __dict__ attribute that mapped attribute names to their values. This made pickling and some other reflection tasks fairly easy.
e.g.:
class Foo: def __init__(self): self.a = 1 self.b = 2
class Bar(Foo): def __init__(self): Foo.__init__(self) self.c = 3
bar = Bar() print bar.__dict__ > {'a': 1, 'c': 3, 'b': 2}
I am aware that there are situations where this simple case does not hold (e.g., when implementing __setattr__ or __getattr__), but let's ignore those for now. Rather, I will concentrate on how this classical Python idiom interacts with the new slots mechanism. Here is the above example using slots:
e.g.:
class Foo(object): __slots__ = ['a','b'] def __init__(self): self.a = 1 self.b = 2
class Bar(Foo): __slots__ = ['c'] def __init__(self): Foo.__init__(self) self.c = 3
bar = Bar() print bar.__dict__ > AttributeError: 'Bar' object has no attribute '__dict__'
We can see that the class instance 'bar' has no __dict__ attribute. This is because the slots mechanism allocates space for attribute storage directly inside the object, and thus does not use (or need) a per-object instance dictionary to store attributes. Of course, it is possible to request that a per-instance dictionary by inheriting from a new-style class that does not list any slots. e.g. continuing from above:
class Baz(Bar): def __init__(self): Bar.__init__(self) self.d = 4 self.e = 5
baz = Baz() print baz.__dict__ > {'e': 5, 'd': 4}
We have now created a class that has __dict__, but it only contains the attributes not stored in slots! So, should class instances explicitly know their attributes? Or more precisely, should class instances always have a __dict__ attribute that contains their attributes? Don't worry, this does not mean that we cannot also have slots, though it does have some other implications. Keep reading...
2) Should attribute access follow the same resolution order rules as methods?
class Foo(object): __slots__ = ['a'] self.a def __init__(self): self.a = 1
class Bar(Foo): __slots__ = ('a',) def __init__(self): Foo.__init__(self) self.a = 2
bar = Bar() print bar.a > 2 print super(Bar,bar).a # this doesn't actually work > 2 or 1?
Don't worry -- this isn't a proposal and no, this doesn't actually work. However, the current implementation only narrowly escapes this trap:
print bar.__class__.a.__get__(bar) > 2 print bar.__class__.__base__.a.__get__(bar) > AttributeError: a
Ok, let me explain what just happened. Slots are implemented via the new descriptor interface. In short, descriptor objects are properties and support __get__ and __set__ methods. The slot descriptors are told the offset within an object instance the PyObject* lives and proxy operations for them. So getting and setting slots involves:
# print bar.a a_descr = bar.__class__.a print a_descr.__set__(bar)
# bar.a = 1 a_descr = bar.__class__.a a_descr.__set__(bar, 1)
So, above we get an attribute error when trying to access the 'a' slot from Bar since it was never initialized. However, with a little ugliness you can do the following:
# Get the descriptors for Foo.a and Bar.a a_foo_descr = bar.__class__.__base__.a a_bar_descr = bar.__class__.a a_foo_descr.__set__(bar,1) a_bar_descr.__set__(bar,2)
print bar.a > 2 print a_foo_descr.__get__(bar) > 1 print a_bar_descr.__get__(bar) > 2
In other words, the namespace for slots is not really flat, although there is no simple way to access these hidden attributes === message truncated ===
__________________________________________________ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com
I think that you're making useful points, but I think that it's worth stepping even further back and deciding what the reflection API should be like from a "what's it for" POV? This relates to much of the discussion about what dir() should do on new-style classes, as well as why some Python objects have 'members', some have 'methods', etc. In my opinon, __dict__ is mostly an implementation detail, and it makes sense to me that the slot names dont' show up in there (after all, it's not a dictionary!). What I'd propose is that the inspect module grow some "abstract" reflection APIs which make it possible for folks who don't need to know about implementation details to get away with it. Looking at it, maybe it already has everything we need. I'm not quite sure why inspect.getmembers is called that, but maybe I'm the only one who's not sure what 'members' mean in Python. --david
On Mon, 18 Feb 2002, David Ascher wrote:
I think that you're making useful points, but I think that it's worth stepping even further back and deciding what the reflection API should be like from a "what's it for" POV?
Exactly! However, having a meta-discussion on meta-reflection is a little too abstract for the disinterested to jump in on. However most people who read python-dev use and come to rely on using __dict__ as The Python Reflection API for instance attributes.
This relates to much of the discussion about what dir() should do on new-style classes, as well as why some Python objects have 'members', some have 'methods', etc.
Sure, except that I've _NEVER_ assumed dir() was anything more than a quick-and-dirty ultra-high level hack that was occaisonally useful for doing reflection. One call does not a reflection API make.
In my opinon, __dict__ is mostly an implementation detail, and it makes sense to me that the slot names dont' show up in there (after all, it's not a dictionary!).
I think so too, though I don't want to ram my own views down people's throats on the matter. However, it is potentially valid to view __dict__ as the one true reflection API for getting access to all attributes. This isn't too outlandish since it effectively is in Python 2.2. Pickle and cPickle 2.2 (among several dozen other examples I've found) are currently implemented assuming this. If we wanted to keep this existing API we could support reflection on slots by extending object instances with only slot attributes to share a common read-only __dict__. New style class instances with per-instance __dict__'s should start with a mutable copy when instantiated. For the record, I don't think this is the right way to go, even though it is a valid way way of defining the Python reflection API.
What I'd propose is that the inspect module grow some "abstract" reflection APIs which make it possible for folks who don't need to know about implementation details to get away with it.
Great idea! I've already got a stack of suggestions and patches that clean up other various bits of it. However, there was an unstated and important question left out of my last e-mail: We need to decide if slots are really 'attributes' or "something else". The "something else" being akin to __set/getattr__ virtual attributes, pure properties, and other techniques that will almost always require explicit hooks to into reflection APIs. My preference is the former, that slot declarations simply affect the allocation choices made by an object, and not the semantics of what may be done with that object (modulo issues when per-instance dicts are not allocated). Thanks, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
[ David Ascher <DavidA@ActiveState.com> wrote:]
I think that you're making useful points, but I think that it's worth stepping even further back and deciding what the reflection API should be like from a "what's it for" POV?
This relates to much of the discussion about what dir() should do on new-style classes, as well as why some Python objects have 'members', some have 'methods', etc.
I think his points were more than useful. His examples expose serious flaws with the use of slots, which I hardly see as satisfactory behavior. Particularly, are slot attributes to be treated like MRO or not? Hummm... Does each class have a separate set of slot attributes? class A: __slots__=('foo','bar') class B(A): __slots__=('foo','spam') What are we supposed to expect here? I believe things would be greatly simplified if the inheritance tree was traversed and all slot attributes were concatenated and regarded as unique for a given instance. So in the above example we would expect (breadth first then depth), __slots__ =('foo', 'spam', 'bar'). A note on dir... Since we have no other introspection tools aside from dir, I as of this revision of python dir should correctly display slot attributes with dict attributes. Does "dir" stand for directory or does it mean __dict__.keys()? I believe this is an implementation detail. Dir should lookup every attribute, but now we need two additional functions: slotdir(), dictdir(). Maybe better names: slots(), dictdir(). Another argument for why dir should be changed, is that cPickle and pickle would problably function correctly as a result of changing dir's implementation to reveal slot attributes. But even on another note which digs deeper into the philosophy of __slots__, why not use slots for class methods? Can this already be done? It can be used everywhere, modules, imported modules, classes, instances, etc.
In my opinion, __dict__ is mostly an implementation detail, and it makes sense to me that the slot names dont' show up in there (after all, it's not a dictionary!).
What I'd propose is that the inspect module grow some "abstract" reflection APIs which make it possible for folks who don't need to know about implementation details to get away with it.
Looking at it, maybe it already has everything we need. I'm not quite sure why inspect.getmembers is called that, but maybe I'm the only one who's not sure what 'members' mean in Python.
--david
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev
__________________________________________________ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com
Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
1) Should class instances explicitly/directly know all of their attributes?
Since types are classes, this is the same question as "should type instances know all their attributes?" I don't think they should, in general: For example, there is no way to find out whether a string object has an interned pointer, and I don't think there should be. The __slots__ aren't really different here. In fact, if you do class Spam(object): __slots__ = ('a','b') s = Spam() s.a = {} del Spam.a you loose access to s.a, even though it is still available (I guess it is actually a bug that cyclic garbage collection won't find cycles involving slots).
2) Should attribute access follow the same resolution order rules as methods?
Yes, I think so.
4) Should __slots__ be flat?
Yes. They should also be a property of the type, not a member of the dict of the type, and they should be a tuple of member object, not a list of strings. It might be reasonable to call this property __members__.
> ('c','d') # current behavior or > ('a','b','c','d') # alternate behavior
Neither, nor; assuming you meant Bar to inherit from Foo, it should be (<member 'a' of 'Foo' objects>, <member 'b' of 'Foo' objects>, <member 'c' of 'Bar' objects>, <member 'd' of 'Bar' objects>) Regards, Martin
On 18 Feb 2002, Martin v. Loewis wrote:
Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
1) Should class instances explicitly/directly know all of their attributes?
Since types are classes, this is the same question as "should type instances know all their attributes?" I don't think they should, in general: For example, there is no way to find out whether a string object has an interned pointer, and I don't think there should be.
I explicitly made note that my discussion of slots was in the context of native new-style Python class and not C-types, even ones that can be used as bases class for other new-style classes. We will always need to hide C implementation details behind Python objects, but we are not talking about reflection on such hidden state. My belief is that slots should be treated as much as possible like normal attributes and not as "hidden object state".
class Spam(object): __slots__ = ('a','b')
s = Spam() s.a = {} del Spam.a
you loose access to s.a, even though it is still available (I guess it is actually a bug that cyclic garbage collection won't find cycles involving slots).
Not exactly -- the semantics are the same as regular attributes in this case. Continuing your example, you can then do s.a = 5 so access to the slot is not lost, only to the value.
2) Should attribute access follow the same resolution order rules as methods?
Yes, I think so.
Ouch! This implies a great deal more than you may be thinking of. For example, do you really want to be able to do this: class Foo(object): __slots__ = ('a',) class Bar(Foo): __slots__ = ('a',) bar = Bar() bar.a = 1 super(Bar, bar).a = 2 print bar.a > 1 This violates the traditional Python idiom of having a flat namespace for attributes, even in the presence of inheritance. This has very profound implications to Python semantics and performance.
4) Should __slots__ be flat?
Yes. They should also be a property of the type, not a member of the dict of the type, and they should be a tuple of member object, not a list of strings. It might be reasonable to call this property __members__.
> ('c','d') # current behavior or > ('a','b','c','d') # alternate behavior
Neither, nor; assuming you meant Bar to inherit from Foo, it should be
(<member 'a' of 'Foo' objects>, <member 'b' of 'Foo' objects>, <member 'c' of 'Bar' objects>, <member 'd' of 'Bar' objects>)
An interesting idea that I had not considered. Currently the slot descriptor objects to not directly expose the name or type of the object except in the repr. This could easily be fixed. However this brings up another issue. The essence of a slot (or, more correctly, a slot descriptor) is to store an offset into a PyObject* that represents a value within an object. The name to which the slot is bound is not the intrinsic and defining characteristic. So it would be somewhat illogical to mandate static name bindings to slots. This supports the notion rebinding slot names during object inheritance (this is already partially implemented), or storing the descriptor objects in a __slots__ tuple and providing an interface to query and reset the name binding for each of them. Comments? Thoughts? Thanks, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
Hi. From: Kevin Jacobs <jacobs@penguin.theopalgroup.com>
On 18 Feb 2002, Martin v. Loewis wrote:
Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
2) Should attribute access follow the same resolution order rules as methods?
Yes, I think so.
Ouch! This implies a great deal more than you may be thinking of. For example, do you really want to be able to do this:
class Foo(object): __slots__ = ('a',)
class Bar(Foo): __slots__ = ('a',)
bar = Bar() bar.a = 1 super(Bar, bar).a = 2 print bar.a > 1
This violates the traditional Python idiom of having a flat namespace for attributes, even in the presence of inheritance. This has very profound implications to Python semantics and performance.
Probably I have not followed the discussion close enough. The variant with super does not work but
bar=Bar() bar.a=1 Foo.a.__set__(bar,2) bar.a 1 Foo.a.__get__(bar) 2
works. Slots are already not flat. They have basically a similar behavior to fields in JVM object model (and I presume in .NET). Given that implementing slots with fields is one of the possibility for Jython (and some possible Python over .NET), with indeed some practical advantages [Btw for the moment I don't see obstacles to such an approach but I have not considered all the details], it is probably reasonable to keep things as they are. Consider also:
class Goo(object): ... __slots__ = ('a',) ... class Bar(Goo,Foo): pass ... Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: multiple bases have instance lay-out conflict
that helps and reinforces that model. Samuele.
On Tue, 19 Feb 2002, Samuele Pedroni wrote:
Slots are already not flat. They have basically a similar behavior to fields in JVM object model (and I presume in .NET).
I agree, but do we want slots to be non-flat? It goes very much against the traditional Python idiom. In my opinion, I believe that slots should have exactly the same semantics as normal instance attributes, except for how/where they are allocated.
Given that implementing slots with fields is one of the possibility for Jython
This is possible for flat slot namespaces too; just remap new slots to existing ones when they overlap, instead of allocating a new one.
Consider also:
class Goo(object): ... __slots__ = ('a',) ... class Bar(Goo,Foo): pass ... Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: multiple bases have instance lay-out conflict
that helps and reinforces that model.
I'll contend that the current implementation is flawed for this and several other reasons I've stated in my previous e-mails. Of course, we're waiting to hear back from Guido when he returns, since his opinion is infinitely more important than mine in this matter. Thanks, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
From: Kevin Jacobs <jacobs@penguin.theopalgroup.com>
On Tue, 19 Feb 2002, Samuele Pedroni wrote:
Slots are already not flat. They have basically a similar behavior to fields in JVM object model (and I presume in .NET).
I agree, but do we want slots to be non-flat? It goes very much against the traditional Python idiom. In my opinion, I believe that slots should have exactly the same semantics as normal instance attributes, except for how/where they are allocated.
Personally I don't expect slots to behave like attributes. I mean, the different naming is a hint.
Given that implementing slots with fields is one of the possibility for Jython
This is possible for flat slot namespaces too; just remap new slots to existing ones when they overlap, instead of allocating a new one.
Yes, but from the POV of fields this is less natural. There's a trade-off issue here.
Consider also:
class Goo(object): ... __slots__ = ('a',) ... class Bar(Goo,Foo): pass ... Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: multiple bases have instance lay-out conflict
that helps and reinforces that model.
I'll contend that the current implementation is flawed for this and several other reasons I've stated in my previous e-mails. Of course, we're waiting to hear back from Guido when he returns, since his opinion is infinitely more important than mine in this matter.
It is not flawed, it is just single-inheritance-of-struct-like-layout-based. I'm fine with that. To be honest I would find very annoying that what we are about to implement in Jython 2.2 should be somehow radically changed for Jython 2.3. We have not the necessary amount of human resources to happily play that kind of game. I hope and presume that Guido did know what he was designing, and I had that impression too. OTOH I agree that pickle should work for new-style classes too. regards, Samuele Pedroni.
On Tue, 19 Feb 2002, Samuele Pedroni wrote:
Personally I don't expect slots to behave like attributes. I mean, the different naming is a hint.
For me, slot declarations are a hint that certain attributes should be allocated 'statically' versus the normal Python 'dynamic' attribute allocation. In virtually all other ways I expect them to act like attributes. The question is should the static allocation introduce all the complex scoping rules that come with Java fields or C++ instance variables. If we go by the "principle of least surprise", it seems much better to keep the normal Python attribute rules than those of Java or C++.
Given that implementing slots with fields is one of the possibility for Jython
This is possible for flat slot namespaces too; just remap new slots to existing ones when they overlap, instead of allocating a new one.
Yes, but from the POV of fields this is less natural. There's a trade-off issue here.
Less natural for Java maybe, but not for Python.
I'll contend that the current implementation is flawed for this and several other reasons I've stated in my previous e-mails. Of course, we're waiting to hear back from Guido when he returns, since his opinion is infinitely more important than mine in this matter.
It is not flawed, it is just single-inheritance-of-struct-like-layout-based. I'm fine with that.
Please read some of my earlier messages. There are other 'warts'.
To be honest I would find very annoying that what we are about to implement in Jython 2.2 should be somehow radically changed for Jython 2.3. We have not the necessary amount of human resources to happily play that kind of game.
Well, we are dealing with an implementation that is not documented _at all_. So, in virtually all respects, Jython 2.2 could ignore their existence totally and still function correctly. I hope that you will be pleased by the in-depth discussions on this topic, since it will likely lead to the formulation of refined documentation for many of these very fuzzily defined features. As an implementer, that kind of clarification can be invaluable since it means you don't have to guess at the semantics and have to change it later.
I hope and presume that Guido did know what he was designing, and I had that impression too. OTOH I agree that pickle should work for new-style classes too.
He knew what he was designing, but was focused on achieving other goals with this infrastructure (like class/type unification). I have the feeling that slots were more of an experiment than anything else. Don't get me wrong -- they are insanely useful even in their current state. On the other hand, I don't think they're ready for prime-time until we smooth over the picky semantic issues relating to slot overloading, reflection and renaming. Just look at the Python standard library -- you'll notice that slots are not used anywhere. I predict that we will be using them extensively, especially in the standard library, as soon as they are deemed ready for prime-time. Best regards, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
From: Kevin Jacobs <jacobs@penguin.theopalgroup.com>
On Tue, 19 Feb 2002, Samuele Pedroni wrote:
Personally I don't expect slots to behave like attributes. I mean, the different naming is a hint.
For me, slot declarations are a hint that certain attributes should be allocated 'statically' versus the normal Python 'dynamic' attribute allocation.
Interesting but for the implementation: class f(file): __slots__ = ('a',) slot a and file.softspace are in the same league, which is not attributes' league. They are struct member and the descriptor logic to access them exploit this.
On Tue, 19 Feb 2002, Samuele Pedroni wrote:
From: Kevin Jacobs <jacobs@penguin.theopalgroup.com>
On Tue, 19 Feb 2002, Samuele Pedroni wrote:
Personally I don't expect slots to behave like attributes. I mean, the different naming is a hint.
For me, slot declarations are a hint that certain attributes should be allocated 'statically' versus the normal Python 'dynamic' attribute allocation.
Interesting but for the implementation:
class f(file): __slots__ = ('a',)
slot a and file.softspace are in the same league, which is not attributes' league.
Currently this is true, though only you and Martin v. Loewis have replied agreeing that this should be the case. Everyone else I've spoken to _wants_ slots to act more like instance attributes.
Yes, but changing the whole impl design is probably not the only solution. I mean this literally.
I realize this. That's why I'm trying to build a consensus until some sort of clarity emerges. That's also why I'm asking for feedback on what should be the correct semantics of slots instead of assuming that the current implementation is the One True Bible of slots. If you think that slots are implemented correctly, then I welcome you to work with me to make exactly that case. Unless you (or others) step up to do that, I will continue to feel that the current slot implementation is flawed and will continue to advocate their reform. Unfortunately, repeatedly pointing out that my suggestions are not how they are implemented doesn't advance either of our cases.
Well, we are dealing with an implementation that is not documented _at all_.
The 2.2 type/class unification tutorial has references to __slots__:
http://www.python.org/2.2/descrintro.html
What is true is that the surface aspects of the undelying design are not documented.
A tutorial is not documentation. It is certainly suggestive of what will eventually be documented, but it is not documented until it is part of the Python Reference Manual. For example, I would not be surprised if large hunks of the descrintro ceases to work after the 'super' and 'property' syntax changes slated for Python 2.3.
So, in virtually all respects, Jython 2.2 could ignore their existence totally and still function correctly.
False. See above.
Don't take my word for it -- ask Guido when he gets back.
I hope that you will be pleased by the in-depth discussions on this topic, since it will likely lead to the formulation of refined documentation for many of these very fuzzily defined features. As an implementer, that kind of clarification can be invaluable since it means you don't have to guess at the semantics and have to change it later.
This one is insolent.
Please, lets not descend into name calling. I truly believe that I am providing a service to the general Python community by engaging in these discussions. If you feel that it is insolent to question the language implementers just because I am a newcomer and have some controversial issues, then I recommend that you rapidly get used to it. I do it all the time and don't plan to stop.
A possible approach: write a patch implementing your preferred semantics. You can keep it orthogonal from the rest, using a name different than "__slots__", for the first cut.
I fully intend to provide a reference implementation of some of these ideas. In fact, its likely to be a fairly small patch. However, I still don't know what the ideal semantics are. I would very much value your input on the matter, even if on a purely theoretical level. So, lets start with the premise that __attrs__ is a declaration like __slots__ except that: 1) the namespace of these 'attrs' is flat -- repeated names in descendant classes either results in an error or silently re-using the existing slot. This maintains the traditional flat instance namespace of attributes. 2) A complete and immutable list of slots is available as a member of each type to allow for easy and efficient reflection. (though I am also in favor of working on better formal reflection APIs) 3) These 'attrs' are to otherwise have the same semantics as normal __dict__ instance attributes. e.g., they should be automatically picklable, they can have properties assigned to them, etc. Regards, -Kevin -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
On 18 February 2002, Kevin Jacobs said:
My recent post on __slots__ not being picklable (and the resounding lack of response to it)
Certainly caught my attention, but I had nothing to add.
1) Should class instances explicitly/directly know all of their attributes?
I'm not sure you're asking the right question. If you're concerned with introspection, shouldn't the question be: "Should arbitrary code be able to find out the set of attributes associated with a given object?" The Pythonic answer is clearly yes. And if "attribute" means "something that follows a dot", then you can do this using dir(). Unfortunately, the expansion of dir() to include methods means it's no longer very useful for getting just instance attributes, whether they're in a __dict__ or some other method. So the obvious answer is to use vars(), which works on classic classes and __slots__-less new-style classes. (I think vars(x) is just a more sociable way to spell x.__dict__.) But it bombs on classes with __slots__:
class C(object): ... __slots__ = ['a', 'b'] ... c = C() vars(c) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: vars() argument must have __dict__ attribute
Uh-oh. This is a problem.
3) Should __slots__ be immutable?
Yes, definitely. Clearly __slots__ is a property of the type (class), not of the instance, and once the class is defined, that's it. (Or that should be it.) It looks as though you can modify __slots__, but it has no effect; that's mildly bogus.
4) Should __slots__ be flat?
Hmmmm... probably. That's certainly consistent with "... once the class is defined, that's it". Greg -- Greg Ward - geek-at-large gward@python.net http://starship.python.net/~gward/ If you and a friend are being chased by a lion, it is not necessary to outrun the lion. It is only necessary to outrun your friend.
participants (6)
-
David Ascher -
Greg Ward -
john coppola -
Kevin Jacobs -
martin＠v.loewis.de -
Samuele Pedroni