On Sat, 25 Jun 2022 at 10:37, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
On Fri, Jun 24, 2022 at 5:38 AM Chris Angelico <rosuav@gmail.com> wrote:
On Fri, 24 Jun 2022 at 16:34, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
On Fri, Jun 24, 2022 at 1:06 AM Chris Angelico <rosuav@gmail.com> wrote:
How much benefit would this be? You're proposing a syntactic construct for something that isn't used all that often, so it needs to be a fairly dramatic improvement in the cases where it _is_ used.
Excuse-me Who is the "you" you are referring to in the last paragraphs? (honest question)
I am not proposing this - the proto-pep is David Mertz' .
You, because you're the one who devised the version that I was responding to. His version is a much more in-depth change, although it has other issues.
ok.
I just pointed out that the language, as it is today,can handle the inner part of the deferred object, as it is.
Yes, but with the limitations that I described.
Indeed - I don't want to argue about that, just point out that the natural way things work in Python as is, some of those limitations do not apply.
(if one just adds all possible dunder methods to your proxy example above, for example)
I still don't understand why you treat dunder methods as special here. Are you, or are you not, relying on __getattribute__? Have you taken tp_* slots into account? I had not thought about tp_*slots - I am just considering pure Python code: any slot which does not alias to a visible dunder method would map to the proxy instead, in a straightforward way for one looking only at the Python code. Maybe some of the not mapped slots might cause some undesired effects, and should trigger the resolve as well.
. The reason I am treating dunder attributes as special is simply because it is what cPython does when resolving any operator with an object - any other attribute access, from Python code, goes through __getattribute__, but the code path triggered by operators (+, -, ..., not, len, str) does not.
Hmmm, I think possibly you're misunderstanding the nature of class slots, then. The most important part is that they are looked up on the *class*, not the instance; but there are some other quirks too:
class Meta(type): ... def __getattribute__(self, attr): ... print("Fetching %s from the metaclass" % attr) ... return super().__getattribute__(attr) ... class Demo(metaclass=Meta): ... def __getattribute__(self, attr): ... print("Fetching %s from the class" % attr) ... return super().__getattribute__(attr) ... x = Demo() x * 2 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for *: 'Demo' and 'int'
Neither the metaclass nor the class itself had __getattribute__ called, because __mul__ goes into the corresponding slot. HOWEVER:
Demo().__mul__ Fetching __mul__ from the class Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 4, in __getattribute__ Fetching __dict__ from the class Fetching __class__ from the class Fetching __dict__ from the metaclass Fetching __bases__ from the metaclass AttributeError: 'Demo' object has no attribute '__mul__'. Did you mean: '__module__'?
If you explicitly ask for the dunder method, it does go through __getattribute__. In other words, even though methods are able to customize the behaviour of operators, the behaviour of operators is not defined in terms of method lookups. (This is particularly obvious with function objects, which have __call__ methods; obviously the effect of calling a function cannot be to look up its __call__ method and call that, as it would lead to infinite recursion.)
Moreover, there could be an attribute namespace to deal/modify the object so - retrieving the "real" object could be trivial. (the original would actually be retrieved in _any_ operation with with the object that would make use of its dunder attributes - think "str", or "myobj + 3", since the proxy dunder would forward the operation to the wrapped object corresponding method.
Okay, here's an exercise for you. Given any function f(), ascertain whether these two calls returned the same object:
x = f() y = later f()
You do not know what kind of object it is. You just have to write the code that will answer the question of whether the second call to f() returned the exact same object as the first call. Calling str() on the two objects is insufficient, for instance. Calling id(y) is not going to touch any of y's dunder methods - it's just going to return the ID of the proxy, so it'll always show as different.
It won't work, indeed. unless there are reserved attributes that would cause the explicit resolve. Even if it is not given, and there is no way for a "is" comparison, this derives from the natural usage of the proxy, with no exceptional behaviors needed. The proxy is not the underlying object, after all. And not even a convention such as a ".__deferred_resolve__" call could solve it: the simpler path I pointed out does not involve "in place attribute substitution". But such a method could return resolve and return the wrapped object, and then: `(z := resolve(y)) is x`, would work , as well as id(resolve(y)) == id(x), but "y"would still be the proxy <- no magic needed, and that is the point I wanted to bring.
That's a consequence of it being a proxy, though. You're assuming that a proxy is the only option. Proxies are never fully transparent, and that's a fundamental difficulty with working with them; you can't treat them like the underlying object, you have to think of them as proxies forever. The original proposal, if I'm not mistaken, was that the "deferred thing" really truly would become the resulting object. That requires compiler support, but it makes everything behave sanely: basic identity checks function as you'd expect, there are no bizarre traps with weak references, C-implemented functions don't have to be rewritten to cope with them, etc, etc, etc.
A similar proxy that is used in day to day coding is a super() instance, and I never saw one needing `super(cls, instance) is instance` to be true.
That's partly because super() deliberately does NOT return a transparent, or even nearly-transparent, proxy. The point of it is to have different behaviour from the underlying instance. So, obviously, the super object itself has to be a distinct thing. Usually, a proxy offers some kind of special value that makes it distinct from the original object (otherwise why have it?), so it'll often have some special attributes that tie in with that (for instance, a proxy for objects stored in a database might have an "is_unsaved" attribute/method to show whether it's been assigned a unique ID yet). This is one of very few places where there's no value whatsoever in keeping the proxy around; you just want to go straight to the real object with minimal fuss.
Then you are not talking about the same thing at all. You're talking about a completely different concept, and you *are* the "you" from my last paragraphs.
I see. I've stepped in because that approach worked _really_ well, and I don't think it is _all_ that different from the proposal on the thread, and is instead a middleground not involving "inplace object mutation", that could make something very close to that proposal feasible.
This seems to be a bit of a theme: a proposal is made, someone else says "but you could do it in this completely different way", and because code is so flexible, that's always technically true. But it's not the same proposal, and when you describe it as a different implementation of the same proposal, you confuse the issue quite a bit. Your proposal is basically just a memoized lambda function with proxying capabilities. The OP in this thread was talking about deferred expressions. And my proposal was about a different way to do argument defaults. All of these are *different* proposals, they are not just implementations of each other. Trying to force one proposal to be another just doesn't work.
Maybe I'd be more happy to see a generic way to implement "super proxys" like these in a less hacky way, and then those could be used to build the deferred objects as in this proposal, than this specific implementation. In the example project itself, Lelo, the proxys are used to calculate the object in a subprocess, rather than just delaying their resolve in-thread.
IMO that's a terrible idea. A proxy usually has some other purpose for existing; purely transparent proxies are usually useless. Making it easier to make transparent proxies in a generic way isn't going to be any value to anything that doesn't want to be fully transparent. Calculating in a subprocess means that everything needed for that calculation has to be able to be serialized (probably pickled) and sent to the subprocess, and the result likewise. That's very limiting, and where you're okay with that, you probably _aren't_ okay with that sort of thing magically happening with all attribute lookups.
I just wrote because it is something I made work before - and if there are indeed uses for it, the language might not even need changes to support it beyond an operator keyword.
Yes, you've done something that is broadly similar to this proposal, but like every idea, has its own set of limitations. It's easy to say "I did something different from what you did, and it doesn't require language support", but your version of the proposal introduces new problems, which is why I responded to them.
Alright - but the only outstanding problem is the "is" and "id"comparison - I am replying still because I have the impression you had not grokked the main point: at some point, sooner or later, for any object in Python, one of the dunder methods _will_ be called (except for identity comparison, if one has it as an "end in itself"). Be it for printing, serializing, or being the target of a unary or binary operator. This path can be hooked to trigger the deferred resolve in the proposal in this thread.
As shown above, not true; dunder methods are not always called if they don't exist, so __getattribute__ cannot always proxy them.
That said, I am not super in favor of it being in the language, and I will leave that for other people to discuss.
So, thank you for your time. Really!
No probs. Always happy to discuss ideas; there's nothing wrong with throwing thoughts out there, as long as you don't mind people disagreeing with you :) ChrisA