Composition over Inheritance
As recent threads indicate, composition may sometimes be better than inheritance. And so I'd like to propose composition as a built-in feature. My idea is syntax of the form o.[c].m(), where o is an object, c is a component, m is a method. I am not sure how you'd set components, or test for components, but I don't think it makes sense to be able to do o.[c][x] or x=o.[c], because those break the idea of automatically passing the object as an argument (unless we create whole wrapper objects every time the syntax is used, and that's a bit ew. also how would you handle o.[c1].[c2] ?). Thus I think o.[c].m() should be syntax sugar for o[c].m(o), with o being evaluated only once, as that solves a lot of current issues relating to inheritance while introducing very few issues relating to python's "everything is separate" (e.g. __get__ vs __getattr__) policy.This also makes setting components and testing for components fairly trivial, and completely avoids the issues mentioned above by making their syntax illegal. (Disclaimer: This was inspired by my own programming language, Cratera[1], so I'm a bit biased here. Cratera was, in turn, inspired by Rust[2] traits. Note however that the original plans for Cratera were far more flexible, including allowing the "problematic" o.[c1].[c2].m() and o.[c][x].m(). I can go into more detail on how those should work, if wanted, but implementation-wise it's not looking good.) [1] https://bitbucket.org/TeamSoni/cratera [2] https://www.rust-lang.org/
On Sat, Oct 28, 2017 at 09:09:30AM -0200, Soni L. wrote:
How is that different from o.c.m() which works today? My understanding of composition is simply setting an attribute of your instance to the component, then calling methods on the attribute. How does that differ from what you are describing? Instead of the classic multiple-inheritence: class Car(Engine, AcceleratorPedal, GasTank, ...): pass which requires each superclass to be designed to work with each other (e.g. you can't have both EntertainmentSystem.start() and Ignition.start(), unless you want the ignition to automatically turn on when the entertainment system does) we can instead use composition and delegation: class Car: def __init__(self): self.engine = Engine() self.accelerator = AcceleratorPedal() ... def start(self): # Delegate to the ignition component. self.ignition.start() etc. Obviously this is just a very loose sketch, don't take it too literally. Is this the sort of thing you are talking about?
I am not sure how you'd set components, or test for components,
If you don't know how to set components, or test for them, what do you know how to do with components? And how are components different from attributes?
I'm afraid I do not understand what you are talking about here. If might help if you give a concrete example, with meaningful names. It would help even better if you can contrast the way we do composition now with the way you think we should do it. I'm afraid that at the moment I'm parsing your post as: "composition is cool, we should use it; and o.[c].m() is cool syntax, we should use it for composition; I'll leave the details to others".
Thus I think o.[c].m() should be syntax sugar for o[c].m(o), with o being evaluated only once,
I don't see why you're using __getitem__ instead of attribute access; nor do I understand why m gets o as argument instead of c. Wait... is this something to do with Lieberman-style delegation? http://web.media.mit.edu/~lieber/Lieberary/OOP/Delegation/Delegation.html http://code.activestate.com/recipes/519639-true-lieberman-style-delegation-i...
Above you said that you don't know how to set and test for components, now you say that doing so is trivial. Which is it?
Traits are normally considered to be a more restricted, safer form of multiple inheritence, similar to mixins but even more restrictive. http://www.artima.com/weblogs/viewpost.jsp?thread=246488
[1] https://bitbucket.org/TeamSoni/cratera [2] https://www.rust-lang.org/
-- Steve
On 2017-10-28 09:51 AM, Steven D'Aprano wrote:
So how do you call car.ignition.start() from car.key.turn()?
They're more like conflict-free interfaces, and in this specific case they're designed with duck typing in mind. (You can dynamically add and remove components, and use whatever you want as the component. You cannot do that with inheritance.)
Again, how do you call car.ignition.start() from car.key.turn()?
TL;DR. But no, it's not some form of delegation. It still gets `self` (which is whatever is in o[c] - which may be c itself, or an arbitrary object that fulfills the contract defined by c), but also gets `o` in addition to `self`. (Unless it's a plain function, in which case it gets no `self`.)
If you pay closer attention, you'll notice the two different paragraphs talk about two different syntaxes. - o.[c] as a standalone syntax element, allowing things like x=o.[c1].[c2]; and x=o.[c1][c2];. - o.[c].m() as a standalone syntax element, *disallowing* the above.
What do you mean more restricted? They let you have the same method in multiple components/traits and not have them conflict, among other things. My variant also makes them dynamic and ducky, making them even more relaxed. Definitely (still) safer tho.
On 28/10/17 14:19, Soni L. wrote:
self.car.ignition.start() of course. If the key has to do something involving the car, it has to know about the car, so tell it about the car: class Car: def __init__(self): self.engine = Engine() self.accelerator = AcceleratorPedal(self.engine) self.ignition = Ignition(self) self.key = Key(self) # and so on. FWIW I haven't the faintest idea what you're talking about. Please provide an example that shows how you might create a "component" and use it. Ideally, comparing it with an example of how you would currently so the same thing in Python.
On Sat, Oct 28, 2017 at 10:19:09AM -0200, Soni L. wrote:
You don't -- the key is not a component of the car, its an argument of ignition.start. If the key doesn't fit, ignition.start() raises an exception and the car doesn't start. I'm not really interested in getting into a tedious debate over the best way to design a Car object. As I said, the above is just a loose sketch illustrating composition, not a carefully planned and debugged object. The aim here is to understand your proposal: what exactly do you mean for Python to support composition over inheritence, and how does it differ from Python's existing support for composition? You ignored my question: Is that the sort of thing you mean by composition? If not, then what do you mean by it? This is not a rhetorical question: I'm having difficulty understanding your proposal. It is too vague, and you are using terminology in ways I don't understand. Maybe that's my ignorance, or maybe you're using non-standard terminology. Either way, if I'm having trouble, probably others are too. Help us understand your proposal.
What do you mean by "conflict-free interfaces"? I can only repeat my earlier request:
Maybe you can't. Maybe this is a crippling example of why composition isn't as good as inheritence and the OOP community is right that inheritence is the best thing since sliced bread. Maybe my design of the Car object sucks. But who cares? None of this comes any closer to explaining your proposal.
One of us is using non-standard terminology, and I don't think it is me. (Happy to be corrected if I'm wrong.) I understand that composition and delegation go hand in hand: you can't have one without the other. Composition refers to the arrangement of an object that is composed of other objects. Delegation refers to the way that the compound object calls methods on the component objects. The point (as I understand it) of composition is that a Car doesn't just have an Engine, it delegates functionality to the Engine: the Car object derives functionality by calling Engine methods directly, rather than inheriting them. Car.forward() delegates to Engine.forward(). The point is that the implementation of Car.forward is found in the self.engine object, rather than being inherited from an Engine class. Without delegation, the components aren't components at all, merely data attributes: Car.colour = 'red'. Does this match your proposal? If not, how is your proposal different?
That sounds like a hybrid of Lieberman-style delegation and the more common form. At first glance, that seems to add complexity without giving the advantages of either form of delegation.
I don't care about syntax yet. I'm still trying to understand the semantics of your proposal. Whether you spell this thing instance.[component] get_component(instance, 'component') instance!component is less important than understand what it *does*.
That makes no sense to me. I cannot make head or tail of what that is supposed to mean.
I mean that if you have two traits with the same method: class SpamTrait: def foo(self): ... class EggTrait: def foo(self): ... then you cannot use them both in a single class: class MyClass(SpamTrait, EggTrait): ... since the foo method clashes, unless MyClass explicitly specifies which foo method to use. Mixins and regular multiple inheritence do not have that restriction. If you expect both foo methods to be called, that's just regular multiple inheritence, with all its complexity and disadvantages. (See Michele Simionato numerous posts on Artima about super, multiple inheritence, mixins and his own traits implementation.) The point of traits is to prevent the existence of such conflicts: either by prohibiting the use of both SpamTrait and EggTrait at the same time, or by forcing MyClass to explicitly choose which foo method gets used. That's safer than unrestricted mixins and multiple inheritence, since it reduces the complexity of the inheritence heirarchy.
They let you have the same method in multiple components/traits and not have them conflict, among other things.
I think we are in agreement here. But in any case... traits are a form of inheritence, not composition. You said this proposal is inspired by Rust traits. Can you explain the connection between inheritence of traits and composition? -- Steve
On 2017-10-28 02:51 PM, Steven D'Aprano wrote:
With composition, you can have car.key.turn() call car.ignition.start(), without having to add car to key or ignition to key. You just have to put both in a car and they can then see eachother!
In Rust, you can put as many "conflicting" traits as you want on the same object, and it'll still work! It'll compile, it'll run, you'll be able to use the object in existing code that expects a specific trait, and code that operates on the object itself is fairly easy to write.
Composition works fine, if you do it like Rust.
Meet ECS: https://en.wikipedia.org/wiki/Entity_component_system Now, keep in mind, the usual ECS is crap. Why's it crap? Because there's no reasonable call convention that's also performant! System.action(entity) # crap. can't override. entity.action() # crap. conflicts easily. either not very dynamic or awful semantics. entity.get(System).action(entity) # crap. while you can override this one, you get to evaluate entity twice! entity.get(System).action() # crap. creates an object every time it's used (or hogs some RAM to cache the object). I could keep going but the point is that Rust traits solve all these problems if you let them.
It's not delegation.
Semantics is Rust traits at runtime, without the delegation (this is by design - with delegation, libraries could make assumptions, and adding new traits to an object could break them).
It means whether your parser looks for "o.[c]" and emits an opcode for it, or it looks for "o.[c].m()" and emits an opcode for that instead.
Rust traits are the best traits. You can have a Rust struct implement 2 traits with the same method. And it just works. You can call each trait method separately - disambiguation is done at the call site. This doesn't translate well to a dynamic language, so, for python, it's best to always specify the trait (no delegation - o.m() is always o.m() and if you want o.[c].m() you need to be explicit about it). Otherwise, it's the same as always: called method gets main object, etc. (Disclaimer: I wrote this message starting with the last question. It should still make sense tho. PS: You should go play with some Rust. As far as compiled languages go, Rust is pretty much the best.)
On Sat, Oct 28, 2017 at 11:24 PM, Soni L. <fakedme+py@gmail.com> wrote:
I have to say I'm almost impressed by the constructiveness of the discussion, even though I still don't understand the point of all the square brackets in the proposal.
Here it's a bit confusing that the key is thought of as part of the car. It's easy to imagine that an owner of two cars would want the same key to work for both of them. Or that one car could have multiple users with non-identical keys. I'm not sure if these things already exist in real life, but if not, it's probably just a matter of time. But let's ignore this confusion for a moment, and imagine that the example makes perfect sense. Now, it sounds like you want something like namespacing for methods and attributes within a complicated class. Maybe you could implement it using nested classes and decorators to make sure 'self' is passed to to the methods in the way you want. The usage might look roughly like: @namespacedclass class Car: @classnamespace class ignition: def start(self): ... @classnamespace class key: def turn(self): self.ignition.start() Another concern regarding the example, however, is that this seems to make it unclear what the public API of the car is. It looks like you can just as easily drive the car without having the key: just call car.ignition.start(). -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven +
On 2017-10-28 11:31 PM, Koos Zevenhoven wrote:
It's a crap example, yes. But you should be able to do things like car = object() car[Engine] = SimpleEngine() car.[Engine].kickstart() # calls kickstart method with an instance of SimpleEngine as `self`/first argument and `car` as second argument. # etc Which your decorator-based approach quite obviously doesn't let you.
On Sun, Oct 29, 2017 at 12:46 PM, Soni L. <fakedme+py@gmail.com> wrote:
I think I follow what you're trying to do here. You want to have a way to refer to a subobject while letting it know about the parent. We already have something like that: when you call a function that was attached to a class, it gets to know which instance of that class you used to locate that function. Maybe there's a way to use descriptor protocol for this too? class SimpleEngine: def kickstart(self, caller): """*boot* Engine starts""" class Car: Engine = magic(SimpleEngine) car = Car() When you look up car.Engine, it remembers a reference to car. Then you look up any callable from there, and it automatically provides an additional parameter. I'm not sure how the details would work, but in theory, this should be possible, right? ChrisA
On 2017-10-28 11:57 PM, Chris Angelico wrote:
And how do you make the object creation so cheap to the point where it's actually practical? (quick question: does Python use a single opcode and an optimized codepath for method calls, or does it always create a method wrapper, even for immediate o.m() calls? If it's the latter then yeah I guess there's no reason for new syntax because it's not gonna be significantly slower than what we currently have...)
On 2017-10-29 12:05 AM, Soni L. wrote:
Hmm thinking about it some more, this whole "magic()" thing is still bad. Replace class Car with: class Car: pass # or something like that and use it as: car = Car() car[Engine] = GasEngine() # please use the actual type instead of a stringy type for this. car[Engine].kickstart() # kickstart gets the car as second argument. And to have all cars have engines, you'd do: class Car: def __init__(self, ???): self[Engine] = GasEngine() car = Car() car[Engine].kickstart() # kickstart gets the car as second argument. And if you can't do that, then you can't yet do what I'm proposing, and thus the proposal makes sense, even if it still needs some refining...
On 2017-10-28 19:13, Soni L. wrote:
As near as I can tell you can indeed do that, although it's still not clear to me why you'd want to. You can give Car a __getitem__ that on-the-fly generates an Engine object that knows which Car it is attached to, and then you can make Engine.kickstart a descriptor that knows which Engine it is attached to, and from that can figure out which Car it is attached to. But why? Why do you want to use this particular syntax to do these things? You haven't explained what semantics you're attaching to the [brackets] as opposed to the .attribute notation. Also, why do you want the car to be the second argument? What if the components are more deeply nested? Are you going to pass every parent component as a separate argument? Why not just have the call be made on an object that has the car available as an attribute (so that inside kickstart() you can access self.car or something)? The pandas library, for instance, makes heavy use of descriptors along these lines. A DataFrame object has attributes called .loc and .iloc that you use to do different kinds of indexing with things like my_data_frame.loc['this':'that'] . This appears conceptually similar to what you're describing here. But it's hard to tell, because you still haven't given a clear, direct example of how you would use what you're proposed to actually accomplish some task. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 29 October 2017 at 12:25, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Right, I think a few different things are getting confused here related to how different folks use composition. For most data modeling use cases, the composition model you want is either a tree or an acyclic graph, where the subcomponents don't know anything about the whole that they're a part of. This gives you good component isolation, and avoids circular dependencies. However, for other cases, you *do* want the child object to be aware of the parent - XML etrees are a classic example of this, where we want to allow navigation back up the tree, so each node gains a reference to its parent node. This often takes the form of a combination of delegation (parent->child references) and dependency inversion (child->parent reference). For the car/engine example, this relates to explicitly modeling the relationship whereby a car can have one or more engines (but the engine may not currently be installed), while an engine can be installed in at most one car at any given point in time. You don't even need the descriptor protocol for that though, you just need the subcomponent to accept the parent reference as a constructor parameter: class Car: def __init__(self, engine_type): self.engine = engine_type(self) However, this form of explicit dependency inversion wouldn't work as well if you want to be able to explicitly create an "uninstalled engine" instance, and then pass the engine in as a parameter to the class constructor: class Car: def __init__(self, engine): self.engine = engine # How would we ensure the engine is marked as installed here? As it turns out, Python doesn't need new syntax for this either, as it's all already baked into the regular attribute access syntax, whereby descriptor methods get passed a reference not only to the descriptor, but *also* to the object being accessed: https://docs.python.org/3/howto/descriptor.html#descriptor-protocol And then the property builtin lets you ignore the existence of the descriptor object entirely, and only care about the original object, allowing the above example to be written as: class Car: def __init__(self, engine): self.engine = engine # This implicitly marks the engine as installed @property def engine(self): return self._engine @engine.setter def engine(self, engine): if engine is not None: if self._engine is not None: raise RuntimeError("Car already has an engine installed") if engine._car is not None: raise RuntimeError("Engine is already installed in another car") engine._car = self self._engine = engine car = Car(GasEngine()) ORMs use this kind of descriptor based composition management extensively in order to reliably model database foreign key relationships in a way that's mostly transparent to users of the ORM classes. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2017-10-29 02:28 AM, Nick Coghlan wrote:
And this is how you miss the whole point of being able to dynamically add/remove arbitrary components on objects you didn't create, at runtime. Someone gave me this code and told me it explains what I'm trying to do: https://repl.it/NYCF/3 class T: pass class C: pass c = C() #c.[T] = 1 c.__dict__[T] = 1 I'd also like to add: def someone_elses_lib_function(arbitrary_object): #arbitrary_object.[T] = object() arbitrary_object.__dict__[T] = object() and def another_ones_lib_function(arbitrary_object): #if arbitrary_object.[T]: if arbitrary_object.__dict__[T]: #arbitrary_object.[T].thing() arbitrary_object.__dict__[T].thing(arbitrary_object)
On 2017-10-29 04:44, Soni L. wrote:
Again, can you please explain why you want to write c.[T]? What do you intend that to *do*? Your commented line seems to indicate you want it to do what `c.__dict__[T]` does, but you can already do that with `setattr(c, T, 1)`. Or you can just give c an attribute that's a dict, but has an easier-to-type name than __dict__, so you can do `c.mydict[T]`. What is the specific advantage of `c.[T]` over these existing solutions? -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 2017-10-29 02:57 PM, Brendan Barnwell wrote:
Hmm... Why can't we just allow empty identifiers, and set a default handler for empty identifiers that implements the proposed ECS? But the basic idea is to indicate something at the call site, namely that T is a contract and the object returned should respect that contract and any function calls should pass the original object as an argument. (I personally don't like how Python treats o.m() (has self) the same as o.f() (no self) syntax-wise, either.)
On 29/10/17 19:25, Soni L. wrote:
It sounds to me like the general shape of what you're proposing is already entirely possible, just without the idiosyncratic syntax. You could write a library that adds support for your components using some other syntax without any additional language support. I don't know, something like this should be doable: @componentlib.has_components class Car: # ... class Engine(componentlib.Component): # might have to be a metaclass? def stall(self, car): raise UnpleasantNoise() # ... car = Car() car.engine = Engine() car.engine.stall() # and so on. If you prefer square brackets, you can implement it with __getitem__ syntax instead of __getattr__ syntax. The point is: not only does your proposal not *need* additional language support; I'm not at all convinced that it would benefit from additional language support.
On 29 October 2017 at 21:44, Soni L. <fakedme+py@gmail.com> wrote:
You can already do that by adding new properties to classes post-definition, or by changing __class__ to refer to a different type, or by wrapping objects in transparent proxy types the way wrapt does. We *allow* that kind of thing, because it's sometimes beneficial in order to get two libraries to play nicely together at runtime without having to patch one or the other. However, it's a last resort option that you use when you've exhausted the other more maintainable alternatives, not something we actually want to encourage. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sunday, October 29, 2017, Nick Coghlan <ncoghlan@gmail.com> wrote:
This is Java-y and maybe not opcode optimizable, but maybe there's a case for defining __setattribute__ so that square brackets denote Rust-like traits: https://docs.spring.io/spring-python/1.2.x/sphinx/html/objects-pythonconfig.... @Object(parent="request") def request_dev(self, req=None):
Like the factory, dependency injection is just a design pattern. The core
It's testable, but confusing to Java programmers who aren't familiar with why Guice forces the patterns that it does: https://docs.spring.io/spring-python/1.2.x/sphinx/html/objects-more.html#tes... https://github.com/google/guice/wiki/Motivation#dependency-injection principle is to separate behaviour from dependency resolution. In our example, the RealBillingService is not responsible for looking up the TransactionLog and CreditCardProcessor. Instead, they're passed in as constructor parameters: When these are constructor parameters, we don't need to monkeypatch attrs in order to write tests; which, IIUC, is also partly why you'd want traits/mixins with the proposed special Rust-like syntax: https://docs.pytest.org/en/latest/monkeypatch.html https://docs.pytest.org/en/latest/fixture.html#modularity-using-fixtures-fro... (this is too magic(), too) But you want dynamic mixins that have an upward reference and Rust-like syntax (and no factories).
For the car/engine example, this relates to explicitly modeling the relationship whereby a car can have one or more engines
class MultiEngine(): zope.interface.implements(IEngine) https://zopeinterface.readthedocs.io/en/latest/README.html#declaring-impleme... But interfaces aren't yet justified because it's only a few lines and those are just documentation or a too-complex adapter registry dict, anyway.
(but the engine may not currently be installed),
So it should default to a MockEngine which also implements(IEngine) and raises NotImplementedError
while an engine can be installed in at most one car at any given point in time.
But the refcounts would be too difficult This:
This could be less verbose. And less likely to raise a RuntimeError.
So there's a 'factory' which passes the ref as a constructor parameter for such ORM instances; but they generally don't need to be dynamically modified at runtime because traits.
... But interfaces are clunky and traits are lightweight, and this isn't Go, so we can't just create a class as a namespace full of @staticmethods which accept the relevant object references. * __setattribute__ -> __getitem__, __setitem__ On Monday, October 30, 2017, Wes Turner <wes.turner@gmail.com> wrote:
On Monday, October 30, 2017, Wes Turner <wes.turner@gmail.com> wrote:
https://docs.python.org/3/howto/descriptor.html ... http://python-reference.readthedocs.io/en/latest/docs/dunderdsc/#descriptor-... ""' Instance BindingIf binding to a new-style object instance, a.x is transformed into the call: type(a).__dict__[‘x’].__get__(a, type(a)). """
My take on all this is that it's much simpler just to give the engine an attribute that refers back to the car. The only downside is that it creates a circular reference, but in these days of cyclic gc, that's not much of an issue. -- Greg
On Sun, Oct 29, 2017 at 1:05 PM, Soni L. <fakedme+py@gmail.com> wrote:
Python-the-language simply specifies the semantics. Different implementations do different things. AIUI CPython 3.7 always creates a method wrapper (using a free-list to minimize memory allocations), but a future version might not; and PyPy's current versions have a special opcode. ChrisA
On Sat, Oct 28, 2017 at 09:09:30AM -0200, Soni L. wrote:
How is that different from o.c.m() which works today? My understanding of composition is simply setting an attribute of your instance to the component, then calling methods on the attribute. How does that differ from what you are describing? Instead of the classic multiple-inheritence: class Car(Engine, AcceleratorPedal, GasTank, ...): pass which requires each superclass to be designed to work with each other (e.g. you can't have both EntertainmentSystem.start() and Ignition.start(), unless you want the ignition to automatically turn on when the entertainment system does) we can instead use composition and delegation: class Car: def __init__(self): self.engine = Engine() self.accelerator = AcceleratorPedal() ... def start(self): # Delegate to the ignition component. self.ignition.start() etc. Obviously this is just a very loose sketch, don't take it too literally. Is this the sort of thing you are talking about?
I am not sure how you'd set components, or test for components,
If you don't know how to set components, or test for them, what do you know how to do with components? And how are components different from attributes?
I'm afraid I do not understand what you are talking about here. If might help if you give a concrete example, with meaningful names. It would help even better if you can contrast the way we do composition now with the way you think we should do it. I'm afraid that at the moment I'm parsing your post as: "composition is cool, we should use it; and o.[c].m() is cool syntax, we should use it for composition; I'll leave the details to others".
Thus I think o.[c].m() should be syntax sugar for o[c].m(o), with o being evaluated only once,
I don't see why you're using __getitem__ instead of attribute access; nor do I understand why m gets o as argument instead of c. Wait... is this something to do with Lieberman-style delegation? http://web.media.mit.edu/~lieber/Lieberary/OOP/Delegation/Delegation.html http://code.activestate.com/recipes/519639-true-lieberman-style-delegation-i...
Above you said that you don't know how to set and test for components, now you say that doing so is trivial. Which is it?
Traits are normally considered to be a more restricted, safer form of multiple inheritence, similar to mixins but even more restrictive. http://www.artima.com/weblogs/viewpost.jsp?thread=246488
[1] https://bitbucket.org/TeamSoni/cratera [2] https://www.rust-lang.org/
-- Steve
On 2017-10-28 09:51 AM, Steven D'Aprano wrote:
So how do you call car.ignition.start() from car.key.turn()?
They're more like conflict-free interfaces, and in this specific case they're designed with duck typing in mind. (You can dynamically add and remove components, and use whatever you want as the component. You cannot do that with inheritance.)
Again, how do you call car.ignition.start() from car.key.turn()?
TL;DR. But no, it's not some form of delegation. It still gets `self` (which is whatever is in o[c] - which may be c itself, or an arbitrary object that fulfills the contract defined by c), but also gets `o` in addition to `self`. (Unless it's a plain function, in which case it gets no `self`.)
If you pay closer attention, you'll notice the two different paragraphs talk about two different syntaxes. - o.[c] as a standalone syntax element, allowing things like x=o.[c1].[c2]; and x=o.[c1][c2];. - o.[c].m() as a standalone syntax element, *disallowing* the above.
What do you mean more restricted? They let you have the same method in multiple components/traits and not have them conflict, among other things. My variant also makes them dynamic and ducky, making them even more relaxed. Definitely (still) safer tho.
On 28/10/17 14:19, Soni L. wrote:
self.car.ignition.start() of course. If the key has to do something involving the car, it has to know about the car, so tell it about the car: class Car: def __init__(self): self.engine = Engine() self.accelerator = AcceleratorPedal(self.engine) self.ignition = Ignition(self) self.key = Key(self) # and so on. FWIW I haven't the faintest idea what you're talking about. Please provide an example that shows how you might create a "component" and use it. Ideally, comparing it with an example of how you would currently so the same thing in Python.
On Sat, Oct 28, 2017 at 10:19:09AM -0200, Soni L. wrote:
You don't -- the key is not a component of the car, its an argument of ignition.start. If the key doesn't fit, ignition.start() raises an exception and the car doesn't start. I'm not really interested in getting into a tedious debate over the best way to design a Car object. As I said, the above is just a loose sketch illustrating composition, not a carefully planned and debugged object. The aim here is to understand your proposal: what exactly do you mean for Python to support composition over inheritence, and how does it differ from Python's existing support for composition? You ignored my question: Is that the sort of thing you mean by composition? If not, then what do you mean by it? This is not a rhetorical question: I'm having difficulty understanding your proposal. It is too vague, and you are using terminology in ways I don't understand. Maybe that's my ignorance, or maybe you're using non-standard terminology. Either way, if I'm having trouble, probably others are too. Help us understand your proposal.
What do you mean by "conflict-free interfaces"? I can only repeat my earlier request:
Maybe you can't. Maybe this is a crippling example of why composition isn't as good as inheritence and the OOP community is right that inheritence is the best thing since sliced bread. Maybe my design of the Car object sucks. But who cares? None of this comes any closer to explaining your proposal.
One of us is using non-standard terminology, and I don't think it is me. (Happy to be corrected if I'm wrong.) I understand that composition and delegation go hand in hand: you can't have one without the other. Composition refers to the arrangement of an object that is composed of other objects. Delegation refers to the way that the compound object calls methods on the component objects. The point (as I understand it) of composition is that a Car doesn't just have an Engine, it delegates functionality to the Engine: the Car object derives functionality by calling Engine methods directly, rather than inheriting them. Car.forward() delegates to Engine.forward(). The point is that the implementation of Car.forward is found in the self.engine object, rather than being inherited from an Engine class. Without delegation, the components aren't components at all, merely data attributes: Car.colour = 'red'. Does this match your proposal? If not, how is your proposal different?
That sounds like a hybrid of Lieberman-style delegation and the more common form. At first glance, that seems to add complexity without giving the advantages of either form of delegation.
I don't care about syntax yet. I'm still trying to understand the semantics of your proposal. Whether you spell this thing instance.[component] get_component(instance, 'component') instance!component is less important than understand what it *does*.
That makes no sense to me. I cannot make head or tail of what that is supposed to mean.
I mean that if you have two traits with the same method: class SpamTrait: def foo(self): ... class EggTrait: def foo(self): ... then you cannot use them both in a single class: class MyClass(SpamTrait, EggTrait): ... since the foo method clashes, unless MyClass explicitly specifies which foo method to use. Mixins and regular multiple inheritence do not have that restriction. If you expect both foo methods to be called, that's just regular multiple inheritence, with all its complexity and disadvantages. (See Michele Simionato numerous posts on Artima about super, multiple inheritence, mixins and his own traits implementation.) The point of traits is to prevent the existence of such conflicts: either by prohibiting the use of both SpamTrait and EggTrait at the same time, or by forcing MyClass to explicitly choose which foo method gets used. That's safer than unrestricted mixins and multiple inheritence, since it reduces the complexity of the inheritence heirarchy.
They let you have the same method in multiple components/traits and not have them conflict, among other things.
I think we are in agreement here. But in any case... traits are a form of inheritence, not composition. You said this proposal is inspired by Rust traits. Can you explain the connection between inheritence of traits and composition? -- Steve
On 2017-10-28 02:51 PM, Steven D'Aprano wrote:
With composition, you can have car.key.turn() call car.ignition.start(), without having to add car to key or ignition to key. You just have to put both in a car and they can then see eachother!
In Rust, you can put as many "conflicting" traits as you want on the same object, and it'll still work! It'll compile, it'll run, you'll be able to use the object in existing code that expects a specific trait, and code that operates on the object itself is fairly easy to write.
Composition works fine, if you do it like Rust.
Meet ECS: https://en.wikipedia.org/wiki/Entity_component_system Now, keep in mind, the usual ECS is crap. Why's it crap? Because there's no reasonable call convention that's also performant! System.action(entity) # crap. can't override. entity.action() # crap. conflicts easily. either not very dynamic or awful semantics. entity.get(System).action(entity) # crap. while you can override this one, you get to evaluate entity twice! entity.get(System).action() # crap. creates an object every time it's used (or hogs some RAM to cache the object). I could keep going but the point is that Rust traits solve all these problems if you let them.
It's not delegation.
Semantics is Rust traits at runtime, without the delegation (this is by design - with delegation, libraries could make assumptions, and adding new traits to an object could break them).
It means whether your parser looks for "o.[c]" and emits an opcode for it, or it looks for "o.[c].m()" and emits an opcode for that instead.
Rust traits are the best traits. You can have a Rust struct implement 2 traits with the same method. And it just works. You can call each trait method separately - disambiguation is done at the call site. This doesn't translate well to a dynamic language, so, for python, it's best to always specify the trait (no delegation - o.m() is always o.m() and if you want o.[c].m() you need to be explicit about it). Otherwise, it's the same as always: called method gets main object, etc. (Disclaimer: I wrote this message starting with the last question. It should still make sense tho. PS: You should go play with some Rust. As far as compiled languages go, Rust is pretty much the best.)
On Sat, Oct 28, 2017 at 11:24 PM, Soni L. <fakedme+py@gmail.com> wrote:
I have to say I'm almost impressed by the constructiveness of the discussion, even though I still don't understand the point of all the square brackets in the proposal.
Here it's a bit confusing that the key is thought of as part of the car. It's easy to imagine that an owner of two cars would want the same key to work for both of them. Or that one car could have multiple users with non-identical keys. I'm not sure if these things already exist in real life, but if not, it's probably just a matter of time. But let's ignore this confusion for a moment, and imagine that the example makes perfect sense. Now, it sounds like you want something like namespacing for methods and attributes within a complicated class. Maybe you could implement it using nested classes and decorators to make sure 'self' is passed to to the methods in the way you want. The usage might look roughly like: @namespacedclass class Car: @classnamespace class ignition: def start(self): ... @classnamespace class key: def turn(self): self.ignition.start() Another concern regarding the example, however, is that this seems to make it unclear what the public API of the car is. It looks like you can just as easily drive the car without having the key: just call car.ignition.start(). -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven +
On 2017-10-28 11:31 PM, Koos Zevenhoven wrote:
It's a crap example, yes. But you should be able to do things like car = object() car[Engine] = SimpleEngine() car.[Engine].kickstart() # calls kickstart method with an instance of SimpleEngine as `self`/first argument and `car` as second argument. # etc Which your decorator-based approach quite obviously doesn't let you.
On Sun, Oct 29, 2017 at 12:46 PM, Soni L. <fakedme+py@gmail.com> wrote:
I think I follow what you're trying to do here. You want to have a way to refer to a subobject while letting it know about the parent. We already have something like that: when you call a function that was attached to a class, it gets to know which instance of that class you used to locate that function. Maybe there's a way to use descriptor protocol for this too? class SimpleEngine: def kickstart(self, caller): """*boot* Engine starts""" class Car: Engine = magic(SimpleEngine) car = Car() When you look up car.Engine, it remembers a reference to car. Then you look up any callable from there, and it automatically provides an additional parameter. I'm not sure how the details would work, but in theory, this should be possible, right? ChrisA
On 2017-10-28 11:57 PM, Chris Angelico wrote:
And how do you make the object creation so cheap to the point where it's actually practical? (quick question: does Python use a single opcode and an optimized codepath for method calls, or does it always create a method wrapper, even for immediate o.m() calls? If it's the latter then yeah I guess there's no reason for new syntax because it's not gonna be significantly slower than what we currently have...)
On 2017-10-29 12:05 AM, Soni L. wrote:
Hmm thinking about it some more, this whole "magic()" thing is still bad. Replace class Car with: class Car: pass # or something like that and use it as: car = Car() car[Engine] = GasEngine() # please use the actual type instead of a stringy type for this. car[Engine].kickstart() # kickstart gets the car as second argument. And to have all cars have engines, you'd do: class Car: def __init__(self, ???): self[Engine] = GasEngine() car = Car() car[Engine].kickstart() # kickstart gets the car as second argument. And if you can't do that, then you can't yet do what I'm proposing, and thus the proposal makes sense, even if it still needs some refining...
On 2017-10-28 19:13, Soni L. wrote:
As near as I can tell you can indeed do that, although it's still not clear to me why you'd want to. You can give Car a __getitem__ that on-the-fly generates an Engine object that knows which Car it is attached to, and then you can make Engine.kickstart a descriptor that knows which Engine it is attached to, and from that can figure out which Car it is attached to. But why? Why do you want to use this particular syntax to do these things? You haven't explained what semantics you're attaching to the [brackets] as opposed to the .attribute notation. Also, why do you want the car to be the second argument? What if the components are more deeply nested? Are you going to pass every parent component as a separate argument? Why not just have the call be made on an object that has the car available as an attribute (so that inside kickstart() you can access self.car or something)? The pandas library, for instance, makes heavy use of descriptors along these lines. A DataFrame object has attributes called .loc and .iloc that you use to do different kinds of indexing with things like my_data_frame.loc['this':'that'] . This appears conceptually similar to what you're describing here. But it's hard to tell, because you still haven't given a clear, direct example of how you would use what you're proposed to actually accomplish some task. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 29 October 2017 at 12:25, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Right, I think a few different things are getting confused here related to how different folks use composition. For most data modeling use cases, the composition model you want is either a tree or an acyclic graph, where the subcomponents don't know anything about the whole that they're a part of. This gives you good component isolation, and avoids circular dependencies. However, for other cases, you *do* want the child object to be aware of the parent - XML etrees are a classic example of this, where we want to allow navigation back up the tree, so each node gains a reference to its parent node. This often takes the form of a combination of delegation (parent->child references) and dependency inversion (child->parent reference). For the car/engine example, this relates to explicitly modeling the relationship whereby a car can have one or more engines (but the engine may not currently be installed), while an engine can be installed in at most one car at any given point in time. You don't even need the descriptor protocol for that though, you just need the subcomponent to accept the parent reference as a constructor parameter: class Car: def __init__(self, engine_type): self.engine = engine_type(self) However, this form of explicit dependency inversion wouldn't work as well if you want to be able to explicitly create an "uninstalled engine" instance, and then pass the engine in as a parameter to the class constructor: class Car: def __init__(self, engine): self.engine = engine # How would we ensure the engine is marked as installed here? As it turns out, Python doesn't need new syntax for this either, as it's all already baked into the regular attribute access syntax, whereby descriptor methods get passed a reference not only to the descriptor, but *also* to the object being accessed: https://docs.python.org/3/howto/descriptor.html#descriptor-protocol And then the property builtin lets you ignore the existence of the descriptor object entirely, and only care about the original object, allowing the above example to be written as: class Car: def __init__(self, engine): self.engine = engine # This implicitly marks the engine as installed @property def engine(self): return self._engine @engine.setter def engine(self, engine): if engine is not None: if self._engine is not None: raise RuntimeError("Car already has an engine installed") if engine._car is not None: raise RuntimeError("Engine is already installed in another car") engine._car = self self._engine = engine car = Car(GasEngine()) ORMs use this kind of descriptor based composition management extensively in order to reliably model database foreign key relationships in a way that's mostly transparent to users of the ORM classes. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2017-10-29 02:28 AM, Nick Coghlan wrote:
And this is how you miss the whole point of being able to dynamically add/remove arbitrary components on objects you didn't create, at runtime. Someone gave me this code and told me it explains what I'm trying to do: https://repl.it/NYCF/3 class T: pass class C: pass c = C() #c.[T] = 1 c.__dict__[T] = 1 I'd also like to add: def someone_elses_lib_function(arbitrary_object): #arbitrary_object.[T] = object() arbitrary_object.__dict__[T] = object() and def another_ones_lib_function(arbitrary_object): #if arbitrary_object.[T]: if arbitrary_object.__dict__[T]: #arbitrary_object.[T].thing() arbitrary_object.__dict__[T].thing(arbitrary_object)
On 2017-10-29 04:44, Soni L. wrote:
Again, can you please explain why you want to write c.[T]? What do you intend that to *do*? Your commented line seems to indicate you want it to do what `c.__dict__[T]` does, but you can already do that with `setattr(c, T, 1)`. Or you can just give c an attribute that's a dict, but has an easier-to-type name than __dict__, so you can do `c.mydict[T]`. What is the specific advantage of `c.[T]` over these existing solutions? -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 2017-10-29 02:57 PM, Brendan Barnwell wrote:
Hmm... Why can't we just allow empty identifiers, and set a default handler for empty identifiers that implements the proposed ECS? But the basic idea is to indicate something at the call site, namely that T is a contract and the object returned should respect that contract and any function calls should pass the original object as an argument. (I personally don't like how Python treats o.m() (has self) the same as o.f() (no self) syntax-wise, either.)
On 29/10/17 19:25, Soni L. wrote:
It sounds to me like the general shape of what you're proposing is already entirely possible, just without the idiosyncratic syntax. You could write a library that adds support for your components using some other syntax without any additional language support. I don't know, something like this should be doable: @componentlib.has_components class Car: # ... class Engine(componentlib.Component): # might have to be a metaclass? def stall(self, car): raise UnpleasantNoise() # ... car = Car() car.engine = Engine() car.engine.stall() # and so on. If you prefer square brackets, you can implement it with __getitem__ syntax instead of __getattr__ syntax. The point is: not only does your proposal not *need* additional language support; I'm not at all convinced that it would benefit from additional language support.
On 29 October 2017 at 21:44, Soni L. <fakedme+py@gmail.com> wrote:
You can already do that by adding new properties to classes post-definition, or by changing __class__ to refer to a different type, or by wrapping objects in transparent proxy types the way wrapt does. We *allow* that kind of thing, because it's sometimes beneficial in order to get two libraries to play nicely together at runtime without having to patch one or the other. However, it's a last resort option that you use when you've exhausted the other more maintainable alternatives, not something we actually want to encourage. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sunday, October 29, 2017, Nick Coghlan <ncoghlan@gmail.com> wrote:
This is Java-y and maybe not opcode optimizable, but maybe there's a case for defining __setattribute__ so that square brackets denote Rust-like traits: https://docs.spring.io/spring-python/1.2.x/sphinx/html/objects-pythonconfig.... @Object(parent="request") def request_dev(self, req=None):
Like the factory, dependency injection is just a design pattern. The core
It's testable, but confusing to Java programmers who aren't familiar with why Guice forces the patterns that it does: https://docs.spring.io/spring-python/1.2.x/sphinx/html/objects-more.html#tes... https://github.com/google/guice/wiki/Motivation#dependency-injection principle is to separate behaviour from dependency resolution. In our example, the RealBillingService is not responsible for looking up the TransactionLog and CreditCardProcessor. Instead, they're passed in as constructor parameters: When these are constructor parameters, we don't need to monkeypatch attrs in order to write tests; which, IIUC, is also partly why you'd want traits/mixins with the proposed special Rust-like syntax: https://docs.pytest.org/en/latest/monkeypatch.html https://docs.pytest.org/en/latest/fixture.html#modularity-using-fixtures-fro... (this is too magic(), too) But you want dynamic mixins that have an upward reference and Rust-like syntax (and no factories).
For the car/engine example, this relates to explicitly modeling the relationship whereby a car can have one or more engines
class MultiEngine(): zope.interface.implements(IEngine) https://zopeinterface.readthedocs.io/en/latest/README.html#declaring-impleme... But interfaces aren't yet justified because it's only a few lines and those are just documentation or a too-complex adapter registry dict, anyway.
(but the engine may not currently be installed),
So it should default to a MockEngine which also implements(IEngine) and raises NotImplementedError
while an engine can be installed in at most one car at any given point in time.
But the refcounts would be too difficult This:
This could be less verbose. And less likely to raise a RuntimeError.
So there's a 'factory' which passes the ref as a constructor parameter for such ORM instances; but they generally don't need to be dynamically modified at runtime because traits.
... But interfaces are clunky and traits are lightweight, and this isn't Go, so we can't just create a class as a namespace full of @staticmethods which accept the relevant object references. * __setattribute__ -> __getitem__, __setitem__ On Monday, October 30, 2017, Wes Turner <wes.turner@gmail.com> wrote:
On Monday, October 30, 2017, Wes Turner <wes.turner@gmail.com> wrote:
https://docs.python.org/3/howto/descriptor.html ... http://python-reference.readthedocs.io/en/latest/docs/dunderdsc/#descriptor-... ""' Instance BindingIf binding to a new-style object instance, a.x is transformed into the call: type(a).__dict__[‘x’].__get__(a, type(a)). """
My take on all this is that it's much simpler just to give the engine an attribute that refers back to the car. The only downside is that it creates a circular reference, but in these days of cyclic gc, that's not much of an issue. -- Greg
On Sun, Oct 29, 2017 at 1:05 PM, Soni L. <fakedme+py@gmail.com> wrote:
Python-the-language simply specifies the semantics. Different implementations do different things. AIUI CPython 3.7 always creates a method wrapper (using a free-list to minimize memory allocations), but a future version might not; and PyPy's current versions have a special opcode. ChrisA
participants (9)
-
Brendan Barnwell -
Chris Angelico -
Greg Ewing -
Koos Zevenhoven -
Nick Coghlan -
Soni L. -
Steven D'Aprano -
Thomas Jollans -
Wes Turner