On Mon, Dec 6, 2021 at 2:56 AM Steven D'Aprano <steve@pearwood.info> wrote:
What sort of "behave differently" do you think would prevent us from introspecting the function object? "Differently" from what?
Wrapping it in a function means the walrus would assign in that function's context, not the outer function.
Unless the variable was a cell variable of the outer function. Which I think that they need to be. Didn't you already decide that walrus assignment has to be in the context of the function?
That's not a rhetorical question, I genuinely don't remember.
Yes, it does, and that won't be the case if you create an invisible nested function.
But this raises a new problem: The function object, when created, MUST know its context. A code object says "this is a nonlocal", and a function object says "when I'm called, this is my context". Which means you can't have a function object that gets called externally, because it's the code, not the function, that is what you need here. And that means it's not directly executable, but it needs a context.
Sorry Chris, I don't understand what you are trying to say here. If I take what you are saying literally, I would take you as trying to say that closures can never be executed. But they clearly can be, and I know that you know that. So obviously I am misunderstanding you.
Closures cannot be executed without a context. Consider: def f(x=lambda: (a:=[])): if isinstance(x, FunctionType): x = x() print(a) Here's the problem: The name 'a' should be in the context of f, but that context *does not exist* until f starts executing. As an early-bound default, like this, it's not going to work, because the function object has to be fully constructed at function definition time. If you try to do this sort of thing with a magical function that is synthesized as part of late-bound default handling, you'll still need to figure out when f's context gets formed. Normally, it won't get formed until the stack frame for f gets constructed - which is when arguments get assigned to parameters, etc, etc. Trying to call that inner function without first giving it a context won't work. And if all you have is a code object without a function, what's the use? How is it better than just having code in the function itself?
I don't understand why you think that we can't take one of those Late-Bound (LB) functions, which will be a closure, and call it like we can call any other closure.
By the time we have access to the LB functions, the owner function will exist, so there shouldn't be any problem with the context not existing.
The function will exist. The instance of it won't. Consider this basic demo of closures: def counter(): n = 1 def incr(): nonlocal n n += 1 return n return incr Prior to calling counter(), you can disassemble incr's bytecode and see what it does. But you can't call incr. Its context does not yet exist. Until you call counter, there is actually no function for incr, only the code object. And while it is possible to eval() a code object...
eval(counter.__code__.co_consts[2]) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: code object passed to eval() may not contain free variables
... you can't if it has nonlocals, because it needs a context. So if there's any possibility of nonlocals in latebound argument defaults, you won't be able to call this externally, which leaves me wondering: why all this hassle if it isn't even going to work?
So, once again, we come right back around to what I have already: code that you can't lift out and call externally. The difference is that, by your proposal, there's a lot more overhead, for the benefit of maybe under some very specific circumstances being able to synthesize the result.
Surely it is the other way around? If you are correct, there are some extremely specific cicumstances involving the use of walrus operator in two different default expressions such that you cannot call one of the two functions, but the 99.99% of straight-forward non-walrus, non-tricky default expressions will work perfectly fine as independently callable functions.
To be honest, 99.99% of late-bound defaults would probably work just fine if you simply eval their texts. But that doesn't justify having functions either.
I'm still not convinced that it's as useful as you say. Compare these append-and-return functions:
def build1(value, lst=None): if lst is None: lst = [] [...] In which of them can you introspect the []?
"Here are lots of ways to solve a problem that don't let you test or introspect part of your code. Therefore we shouldn't add a feature that will let us more easily test and introspect that part of the code."
Yes Chris, I know that emulated late-bound defaults are currently effectively invisible at runtime (short of byte-code hacking). That weakness of the status quo is why you have written this PEP.
That invisibility is not a feature we should duplicate in late-bound defaults, but an argument for introducing defaults that overcome that weakness.
Okay, but you're asking for a LOT of complexity for a very small part of the feature.
Right now, your PEP adds a new feature that gives us *zero* new functionality. There is nothing your PEP allows us to do that we cannot already do. It adds no new functionality. It's just syntactic sugar for "if arg is missing: return expression". You compile that code (or something very close to it) into the body of the function.
MOST new features give us zero new functionality. That's what it means for a language to be Turing-complete. But "if arg is missing:" doesn't currently exist, nor does "this argument is optional and has no default", so you can't quite write this currently, and that's why we need sentinels.
The only wins are (1) when people stumble into the "mutable default" gotcha, it is easy to tell them how to fix it (but not much easier than the status quo) and (2) a small increase in clarity, since we will be able to write the default expression directly in the signature instead of hiding behind a sentinel.
I consider those to be LARGE wins.
That's not nothing, but we've seen on this list a fair number of people who think that the increase in clarity is not enough to justify the feature. Do you think the Steering Council will consider those two small wins sufficient?
I'm hoping so. That's the main thrust of what I'm doing. Clarity, and an easier way to make mutable defaults working. And more intuitive behaviour in some situations. Clarity, mutable defaults, and intuitive behaviour. And nice red uniforms.
But with a different design, we have not just the increase in clarity, but also open the door to a ton of new possibilities that come about thanks to the separation of those default expressions into their own objects. I don't know whether that will be enough to sway the Steering Council. They might hate my idea even more. But feedback on this list suggests that your design's lack of introspectability is putting people off the idea.
Or it's suggesting that people on this list love to argue. I don't think we're surprised by that. :) (And I don't have a problem with it.)
It may be that this feature will never be used for anything more complex than `param=>[]`. In which case, okay, I concede, moving the default expression into its own code unit (a function? something else?) is overkill.
But I don't think that people will only use this for such trivial expressions. Maybe 60% of the cases will be just a literal list display or dict display, and maybe another 30% will be a simple call to an existing function, like `time.ctime()`.
But its the last 10% of non-trivial, interesting expressions where people will want to do more than just inspect the string. And having the expressions as first-class code units will open up those new opportunities that we currently lack.
If you want to write up a competing reference implementation, go ahead. I don't think it will be as easy as you claim. And I'm not going to mention the possibility in the PEP *without* a reference implementation, nor do I intend to write one myself (since I dispute its usefulness). Will the benefits outweigh the costs? I don't know. But I think not, and that the much simpler proposal will be better. ChrisA