On Wed, Jun 23, 2021 at 6:25 PM Steven D'Aprano
Soni, and Chris, you seem to be responding as if extension methods are clearly, obviously and self-evidently a stupid idea. Let me remind you that at least ten languages (C#, Java, Typescript, Oxygene, Ruby, Smalltalk, Kotlin, Dart, VB.NET and Swift) support it. Whatever the pros and cons of the technique, it is not self-evidently wrong or stupid.
It's not self-evidently wrong or stupid. But the semantics, as given, don't make sense. Before this could ever become part of the language, it will need some VERY well-defined semantics. (Preferably, semantics that don't depend on definitions involving CPython bytecode, although I'm fine with it being described like that for the time being. But ultimately, other Pythons will have to be able to match the semantics.) You have been saying certain things as if they are self-evidently right, without any justification or explanation.
Let me ask you this:
- should proxy objects really include `__slots__` by default?
- should they really include dynamic attributes generated by `__getattr__`?
- should they include attributes in the inheritence hierarchy?
- why should extension methods be any different?
They are different because they are *context-sensitive*. Every other example you have given is attached to the object itself. The object defines whether it has __slots__, __dict__, a __getattr__ method, a __getattribute__ method, and superclasses. The object defines, in those ways, which attributes can be looked up, and it doesn't matter how you ask the question, you'll get the same answer. Calling getattr(obj, "thing") is the same as obj.thing is the same as PyObject_GetAttr(ptr_to_obj, ptr_to_string_thing) is the same as any other way you would look it up. Extension methods change that. Now it depends on which module you are in. That means you're either going to have to forfeit these consistencies, or they are going to need to figure out WHICH module you are working with. The simplest definition is this: Extension methods apply *only* to dot notation here in the current module. Every piece of code compiled in this module will look up dotted attributes using extensions active in this module. (In CPython terms, that affects the behaviour of LOAD_ATTR only, and would mean that the code object retains a reference to that module.) That's pretty reasonable. But to accept this simple definition, you *must* forfeit the parallel with getattr(), since getattr() is defined in the builtins, NOT in your module. Yet you assert that, self-evidently, getattr(obj, "thing") MUST be the same as obj.thing, no matter what.
Let's step back from extension methods and consider a similar technique, the dreaded monkey-patch. If I extend a class by monkey-patching it with a new method:
import library library.Klass.method = patch_method
would you expect that (by default) the patched method are invisible to proxies of Klass? Would you expect there to be a way to "opt-out" of proxying that method?
I hope that your answers are "No, and no", because if either answer is "yes", you will be very disappointed in Python.
For my part, I absolutely agree with you - the proxy should see it. But that's because the *object*, not the module, is making that decision.
Why should extension methods be different from any other method? Let's go through the list of methods which are all treated the same:
- methods defined on the class; - methods defined on a superclass or mixin; - methods added onto the instance; - methods created dynamically by `__getattr__`.
(Did I miss any?)
Yes, all things that are defined by the object, regardless of its context.
And the list of those which are handled differently, with ways to opt-out of seeing them:
- ... um... er...
Have I missed any?
Nope. Python currently has a grand total of zero ways to have attributes whose existence depends on the module of the caller. (Barring shenanigans with __getattr__ and sys._getframe. Or ctypes. I think we can all agree that that sort of thing doesn't count.)
The origin of obj.method should not make any difference. I'm sorry to have to keep harping on this, but it doesn't matter to the proxy whether the method exists in the instance `__dict__`, or the class `__dict__`, or `__slots__`, or a superclass, or is dynamically generated by `__getattr__`. A method is a method.
Extension methods are methods.
By definition, extension methods are methods in one module, and not methods in another module.
Also note that they use instance method syntax, but no other. That is they apply to LOAD_ATTR opcodes but should not apply to getattr! (Indeed, reflection in C#/Kotlin doesn't see the extension methods!)
Okay, that's a good data point. The question is, why doesn't reflection see the extension methods? That will help us decide whether that's a limitation of reflection in those languages, or a deliberate design feature we should follow.
Yes, I'd definitely like to know this too.
A brief search suggests that people using C# do want to access extension methods via reflection, and that there are ways to do so:
https://duckduckgo.com/?q=c%23+invoke+extension+method+via+reflection
The answers appear to be bypassing the extension method and going for the concrete function that underlies it. That seems perfectly reasonable, but it's basically an acknowledgement that extension methods don't show up in these kinds of ways. So... is it really so self-evident that getattr(obj, "thing") HAS to be the same as obj.thing ? ChrisA