On Wed, Dec 01, 2021 at 07:47:49AM -0000, Jeremiah Vivian wrote:

...

2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden? No, it isn't much of a cognitive burden.

You say that now, but if you read function definitions that looked like this: def process(func:List->int=>xs=>expression)->int: ... would you still agree? -- Steve

On Wed, Dec 1, 2021 at 7:07 PM Chris Angelico <rosuav@gmail.com> wrote:

On Wed, Dec 1, 2021 at 6:58 PM Steven D'Aprano <steve@pearwood.info> wrote:

...

You say that now, but if you read function definitions that looked like this:

def process(func:List->int=>xs=>expression)->int: ...

would you still agree?

I'm not sure what that's supposed to mean.

(To clarify: by "that", I mean "the hypothetical line of code", not the question you're asking. I'm not sure what the code example is supposed to mean.) ChrisA

On Wed, Dec 1, 2021 at 9:09 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Dec 01, 2021 at 07:07:20PM +1100, Chris Angelico wrote:

...

...

def process(func:List->int=>xs=>expression)->int: ...

...

I'm not sure what that's supposed to mean.

You did a pretty good job of working it out :-)

Okay, cool, thanks.

Exactly my point. Beyond your arrow syntax for default parameters, and the existing return annotation use, there are two other hypothetical proposals for arrow syntax on the table:

- using `->` as an alias for typing.Callable;

- using `=>` as a more compact lambda;

That confuses me. Why would they use different arrows? If we have a compact lambda syntax, why can't actual functions be used as annotations to represent functions? I believe there's plans to have [str] mean List[str], which makes a lot of sense. Why not have "lambda str: int" or "str=>int" be an annotation too? But okay. Supposing that annotations have to use one arrow and lambda functions use another, then yes, this is a good reason to use "=:" for late-bound defaults. It's not that big a change.

...

func:ann=>dflt # late-bound default, completely unnecessary here

Come on Chris, how can you say that it is "completely unnecessary"? Unless that's an admission that late-bound defaults are all unnecessary... *wink*

There is a difference between these two:

def func(arg=lambda a: expression): ...

and this:

def func(arg=None): if arg is None: arg = lambda a: expression ...

therefore there will be a difference between:

def func(arg=lambda a: expression): def func(arg=>lambda a: expression):

If nothing else, in the first case (early binding) you get the same function object every time. In the second, you get a freshly made function object each time. Since function objects are mutable (they have a writable `__dict__` that's a visible difference even if the bodies are identical. And they may not be.

Yyyyyes. Okay. There is a technical way in which you might want this alternate behaviour. Is that REALLY something you're planning on doing - having a function which takes another function as an argument, and which behaves differently based on whether it's given the same function every time or multiple different functions with the same behaviour?

But even if it is unnecessary, it will still be permitted, just as we will be able to write:

# Now this *actually is* totally unnecessary use of late-binding def func(arg=>None):

Yes. That one is absolutely unnecessary, since there is no way you'll ever get back a different result. Except that you could then mutate func's dunders and change the behaviour. So in a purely technical sense, nothing can be called "unnecessary". Now, in real terms: late-binding a lambda function with no default arguments seems like a pretty pointless thing to do. I stand by my original statement :) And I also stand by my original statement that proper use of the space bar can not only improve readability, it can also help you find a date with space girls and space guys... or maybe that last part only works on Mars.

...

xs=>expression # presumably a lambda function def process(args)->int # return value annotation

Why should List->int and xs=>expression use different arrows?

Because they do different things. To avoid confusion.

Chris, you wrote the PEP for the walrus operator. Why should the assignment operator use a different symbol from the assignment statement? Same reason that typing.Callable and lambda will likely use different arrows.

Anyway, if you disagree, take it up with Guido, it was his suggestion to use different arrows :-P

Fair enough, but typing also started out with List[str] and is now looking at using list[str] and, I think, [str], because it's more consistent :)

...

Wouldn't it be much more reasonable for them to use the same one, whichever that be? And if that does turn out to be "=>", then yes, I would be looking at changing PEP 671 to recommend := or =: or something, for clarity (but still an equals sign with one other symbol next to it).

Oh great, now you're going to conflict with walrus...

def process(obj:Union[T:=something, List[T]]:=func(x:=expression)+x)->T:

We ought to at least try to avoid clear and obvious conflicts between new and existing syntax.

Using `:=` is even worse than `=>`, and `=:` is just *begging* to confuse newcomers "why do we have THREE different assignment symbols?"

If you look at every way the equals sign is used in Python grammar, there are a lot of subtle differences. The difference between "f(x=1)" and "x=1" doesn't bother people, and those are exactly the same symbol. There are only so many symbols that we can type on everyone's keyboards. But you're right, and that's why I would very much like to reuse an existing symbol rather than create new ones.

...

It's always possible to come up with pathological code. But this is only really as bad as you describe if it has zero spaces in it. Otherwise, it's pretty easy to clarify which parts go where:

def process(func: List->int => xs=>expression) -> int: ...

"My linter complains about spaces around operators! Take them out!"

Yes, take your linter out the back and execute it :)

Or maybe we should put more in? Spaces are optional.

def process(func : List -> int => xs => expression) -> int: ...

I'm not saying that an experienced Pythonista who is a careful reader can't work out what the meaning is. It's not *ambiguous* syntax to a careful reader. But it's *confusing* syntax to somebody who may not be as careful and experienced as you, or is coding late at night, or in a hurry, or coding while tired and emotional or distracted.

We should prefer to avoid confusable syntax when we can. The interpreter can always disambiguate assignment as an expression from assignment as a statement, but we still chose to make it easy on the human reader by using distinct symbols.

def process(arg>=expression)

would be totally unambiguous to the intepreter, but I trust you would reject that because it reads like "greater than" to the human reader.

Actually that, to me, is no different from the other options - it's still an arrow (but now more like a funnel than an arrow, which is a bit weird). It's no more a greater-than than "=>" is. ChrisA

I'll reply to this comment once I've done so. I just realized that this message replied on a comment instead of the thread. Anyways, it's a really impressive implementation (hope this statement doesn't spark some sort of debate).

To be honest, I like the `@param=value` syntax. It is sort of easier to read than `param=>value`, though I do not have problems with distinguishing the arrow `=>` from other things.

On Wed, Dec 1, 2021 at 6:43 PM Abdulla Al Kathiri <alkathiri.abdulla@gmail.com> wrote:

...

On 1 Dec 2021, at 10:16 AM, Chris Angelico <rosuav@gmail.com> wrote: 3) If "yes" to question 1, would you use it for any/all of (a) mutable defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else? I will definitely use it for default mutable collections like list, set, dictionary etc. I will also use it to reference things that might have changed. For example, when making callbacks to GUI push buttons, I find myself at the start of the function/callback to be fetching the values from other widgets so we can do something with them. Now those values can be directly passed as late-bound defaults from their respective widgets (e.g., def callback(self, text1 => self.line_edit.text()): …).

Very interesting. That doesn't normally seem like a function default - is the callback ever going to be passed two arguments (self and actual text) such that the default would be ignored? But, hey, if it makes sense in your code to make it a parameter, sure!

...

5) Do you know how to compile CPython from source, and would you be willing to try this out? Please? :) I haven’t done it from source. I might try to learn how to do it in the next weekend and give it a try.

...

I'd love to hear, also, from anyone's friends/family who know a bit of Python but haven't been involved in this discussion. If late-bound defaults "just make sense" to people, that would be highly informative. I will show this to some of my coworkers who are python experts and I will report back.

Thank you! All feedback greatly appreciated. Building CPython from source can be done by following these instructions: https://devguide.python.org/setup/ Instead of creating your own clone from the pristine master copy, instead clone my repository at https://github.com/rosuav/cpython and checkout the pep-671 branch. That'll give you my existing reference implementation (it's pretty crummy but it mostly works). ChrisA

On Wed, Dec 1, 2021 at 8:18 PM Abdulla Al Kathiri <alkathiri.abdulla@gmail.com> wrote:

...

On 1 Dec 2021, at 12:01 PM, Chris Angelico <rosuav@gmail.com> wrote:

On Wed, Dec 1, 2021 at 6:43 PM Abdulla Al Kathiri <alkathiri.abdulla@gmail.com> wrote:

...

...

On 1 Dec 2021, at 10:16 AM, Chris Angelico <rosuav@gmail.com> wrote: 3) If "yes" to question 1, would you use it for any/all of (a) mutable defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else? I will definitely use it for default mutable collections like list, set, dictionary etc. I will also use it to reference things that might have changed. For example, when making callbacks to GUI push buttons, I find myself at the start of the function/callback to be fetching the values from other widgets so we can do something with them. Now those values can be directly passed as late-bound defaults from their respective widgets (e.g., def callback(self, text1 => self.line_edit.text()): …).

Very interesting. That doesn't normally seem like a function default - is the callback ever going to be passed two arguments (self and actual text) such that the default would be ignored? Yeah. Let’s say the callback prints the text on the main window console. I could use the same function to print something on the console not related to the default widget changing text. Maybe another push button that prints literal “WOW”. If I made the second argument a default widget (def callback(self, line_edit=self.line_edit):...) and then call line_edit.text() inside the function, I would be able to pass any other widget that has the method text() but I wouldn’t be able to pass a literal text.

Ahh gotcha. Then yes, you have a function that takes up to two parameters (or one if you don't count self), and yes, omitting the second argument will have the same effect as fetching the text from that control. So, sounds like a great use for it! ChrisA

On Wed, Dec 1, 2021 at 8:57 PM Neil Girdhar <mistersheik@gmail.com> wrote:

On Wednesday, December 1, 2021 at 1:18:33 AM UTC-5 Chris Angelico wrote:

...

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ with some additional information about the reference implementation, and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would you use it?

No, I would avoid for the sake of readers. Anyway, it can't be used in any public projects until thy drop support for 3.10, which is many years off.

Also, I think this question is quite the biased sample on python-ideas. Please consider asking this to less advanced python users, e.g., reddit.com/r/python or learnpython.

I know it is :) I never expected to get an unbiased sample (from anywhere, least of all here).

This PEP has a lot of interesting ideas. I still think that none-aware operators (PEP 505) are an easier, more readable general solution to binding calculated default values to arguments succinctly. I think the problems with this idea include: * The caller cannot explicitly ask for default behaviour except by omitting the parameter. This can be very annoying to set up when the parameter values are provided from other places, e.g., if need_x: # do lots of stuff x = whatever else: # do more x = None f(x=x, ...) # easy, but with this PEP, you would have to find a way to remove x from the parameter list. Typically, removing a parameter from a dynamically-created parameter list is hard.

You can always do that with *args or **kwargs if that's what you need, and that actually DOES represent the concept of "don't pass this argument". Passing None is a hack that doesn't actually mean "don't pass this argument", and if you're depending on that behaviour, it's been locked in as the function's API, which causes problems if None ever becomes a meaningful non-default value. None-aware operators are solving a different problem. They allow you to accept None, but then simplify the replacement of it with some other value. This proposal allows you to leave None out of the equation altogether. And it only works if None isn't a valid value.

* The function code becomes unreadable if the parameter-setting code is long. Having a long piece of code inside the function after "if parameter is None" is just fine. Having none-aware operators would make such code more succinct.

Again, that only works if None is not a valid parameter value, and PEP 505 doesn't generalize to other sentinels at all. If the code is too long, don't put it into the parameter default. But "if x is None: x = []" can easily be replaced with "x=>[]" and the parameters will actually become shorter. ANY feature can be used badly. You can daisy-chain everything into a gigantic expression with horrendous abuses of lambda functions, but that doesn't mean you should. It also doesn't mean that lambda functions are bad :)

* People nearly always avoid writing code in the parameter defaults themselves, and this new practice adds a lot of cognitive load. E.g., people rarely write: def f(x: int = 1+g()) -> None: ... Parameter lists are already busy enough with parameter names, annotations, and defaults. We don't need to encourage this practice.

They don't become any more noisy by being able to use defaults that aren't constant. The main goal is to put the function header in the function header, and the function body in the function body, instead of having the function body doing part of the work of the header :)

In short, I think this is a creative idea, a great exploration. While optional parameters are common, and some of them have defaults that are calculated inside the function, my feeling is that people will continue to set their values inside the function.

In some cases, they certainly will. And that's not a problem. But for the cases where it's better to put it in the header, it's better to have that option than to restrict it for an arbitrary technical limitation. Some languages don't have ANY function defaults. They simply have "optional arguments", where an omitted optional argument will always and only have a specific null value. Python allows you to show what the default REALLY is, and that's an improvement. I want to be able to show what the default is, even if that is defined in a non-constant way. ChrisA

1) If this feature existed in Python 3.11 exactly as described, would you use it?

Absolutely!

2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden?

Nope, I don't think this is an undue cognitive burden. If anything I think the symmetry between the proposed '=>' syntax and the arrow syntax for lambdas in other languages (potentially even in python in the future) reduces the cognitive burden significantly, given the there is an equivalent symmetry with their semantics (in both cases the code is being evaluated later when something is called). Steven gave the following example of a function signature that would be difficult to visually parse if this proposal and arrow lambdas were accepted: def process(func:List->int=>xs=>expression)->int: And while I agree that it does sort of stop you in your tracks when you see this, I think there are a couple of reasons why this is not as big of a problem as it appears. Firstly, I think if you're writing this sort of code, you can improve the legibility a lot by using appropriate spacing and adding parentheses around the lambda arguments (even if they're unnecessary for a lambda with only a single argument, like in Steven's example): def process(func: (List) -> int => (xs) => expression) -> int: Personally I find this much easier to visually parse since I'm used to seeing lambdas in the form: (*args) => expr So my brain just kind of groups `(List) -> int` and `(xs) => expression` into atomic chunks fairly effortlessly. And this is before even mentioning the *massive* cognitive assistance provided by syntax highlighting :) But at this point I would also like to point out that code like this is probably going to be vanishingly rare in the wild. The majority of use-cases for late-bound defaults as far as I can tell are going to be for mutable defaults (and maybe some references to early-bound arguments). I don't like the idea of compromising on what is a very clean and intuitive syntax in 99% of cases to cater to the hypothetical 1% of cases where it admittedly takes a bit more cognitive effort to parse. I agree with the statement Chris made above where he mentioned that the same symbols sometimes have different meanings based on context, and that's okay. For example: def some_func(some_arg: int) -> int: ... def another_func(another_arg: int = some_func(some_arg=3)) -> int: ... In the second function definition above you've got the '=' symbol pulling double-duty both as declaring an argument for a function def and as a keyword arg for a function call in the same statement. I think this is perfectly fine, and I think the same thing is true of the => symbol being used for both lambdas and late-bound arguments, especially given they are semantically related. 3) If "yes" to question 1, would you use it for any/all of (a) mutable

defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else?

(a) is the primary use-case I can see myself using this feature for. I'm not sure what you mean by (b). I can also definitely see some situations where (c) would be dead useful, though I don't think this would come up in the sort of code I write as much as (a). Still, when the use-case for (c) did present itself, I would be *extremely* grateful to have late-bound defaults to reach for.

4) If "no" to question 1, is there some other spelling or other small change that WOULD mean you would use it? (Some examples in the PEP.)

Not relevant, since my strong preference for late-bound default syntax is the '=>' symbol. On Wed, Dec 1, 2021 at 11:50 AM Chris Angelico <rosuav@gmail.com> wrote:

On Wed, Dec 1, 2021 at 10:30 PM André Roberge <andre.roberge@gmail.com> wrote:

...

On Wed, Dec 1, 2021 at 2:17 AM Chris Angelico <rosuav@gmail.com> wrote:

...

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ with some additional information about the reference implementation, and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would you use it?

Currently, I'm thinking "absolutely not".

However, I thought the same about the walrus operator and I now miss not being able to use it in a program that includes support for Python 3.6 and where I have literally dozens of places where I would use it if I could.

Very fair :)

...

...

2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden?

Yes. I really think that using a keyword like defer, or from_calling_scope ;-), would significantly reduce the cognitive burden.

Also fair. I'm not a fan of keywords for this sort of thing, since it implies that you could do this:

def f(x=defer []): ...

dflt = defer [] def f(x=dflt): ...

which is a completely different proposal (eg it would be evaluated only when you "touch" that, rather than being guaranteed to be evaluated before the first line of the function body). That's why I want to adorn the equals sign and nothing else.

...

...

4) If "no" to question 1, is there some other spelling or other small change that WOULD mean you would use it? (Some examples in the PEP.)

*Perhaps* if a keyword would be used instead of symbols, I might reconsider.

I find the emphasis of trying to cram too much information in single lines of code to be really a burden. Many years ago, I argued very unsuccessfully for using a 'where:' code block for annotations. (To this day, I still believe it would make the code much more readable, at the cost of a slight duplication.) Using what is at first glance a cryptic operator like => for late binding is not helping readability, especially when type annotations are thrown in the mix.

Aside: at the same time, I can see how using => instead of lambda as a potential win in readability, including for beginners.

It's interesting how different people's views go on that sort of thing. It depends a lot on how much people expect to use something. Features you use a lot want to have short notations, features you seldom use are allowed to have longer words.

...

...

5) Do you know how to compile CPython from source, and would you be willing to try this out? Please? :)

Sorry, I'm not interested enough at this point but, given the amount of work you put into this, I decided that the least I could do is provide feedback rather than be a passive reader.

That's absolutely fine. I don't by any means expect everyone to be able or willing to compile CPython. Feedback is absolutely appreciated, and I asked the question with full expectation of getting a few "No" responses :)

Thank you for taking the time to respond. Thoughtful feedback is incredibly helpful.

ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WTU355... Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Dec 2, 2021 at 1:30 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Wed, Dec 01, 2021 at 12:26:33PM +0000, Matt del Valle wrote:

...

If anything I think the symmetry between the proposed '=>' syntax and the arrow syntax for lambdas in other languages (potentially even in python in the future) reduces the cognitive burden significantly, given the there is an equivalent symmetry with their semantics (in both cases the code is being evaluated later when something is called).

It may be that behind the scenes the default expression is stored as a callable function which the interpreter calls. (I believe that list comprehensions do something similar.) But that's an implementation detail that can change: it might just as well store the source code as a string, and pass it to eval().

In my reference implementation, there is no object that stores it; it's simply part of the function. A good parallel is the if/else expression: x = float("inf") if z == 0 else y/z Is there an object that represents the if/else expression (or the deferred "y/z" part)? No, although it could be implemented that way if you chose: x = iff(z == 0, lambda: y/z, lambda: float("inf")) But just because you CAN use a function to simulate this behaviour, that doesn't mean that it's inherently a function. So I agree with Steve that the parallel is quite weak. That said, though, I still like the arrow notation, not because of any parallel with a lambda function, but because of the parallel with assignment. (And hey. You're allowed to think of things in any way you like. I'm not forcing you to interpret everything using the same justifications I do.)

Or use some other mechanism that I'm not clever enough to think of, so I shall just call "deepest black magic".

Remind me some time to use some slightly shallower black magic.

There is no *neccessity* for the late-bound default to be a hidden function, and it is certainly not part of the semantics of late-bound defaults. Just the implementation.

If you disagree, and still think that the symmetry is powerful enough to use the same syntax for both lambdas and default arguments, well, how about if we *literally* do that?

def function(spam=expression, # regular default lambda eggs: expression, # late-bound default )

Heh, perhaps the symmetry is not that strong after all :-)

That..... is hilariously confusing. I like it. Just not EVER in the language :) ChrisA

On Wed, Dec 01, 2021 at 09:58:11PM -0600, Abe Dillon wrote:

My favorite alternative is ?= if people think => and -> are getting overly loaded. What I really don't like is @param=[] because it puts the emphasis on the parameter name rather than the act of binding. Not only does it make it look like @param is a special kind of variable, it also mimics the *args and **kwargs syntax which makes them seem related.

But it *is* a special kind of parameter: it is a parameter that uses late binding for the default value, instead of early binding. Putting the emphasis on the parameter name is entirely appropriate. Late bound parameters don't have a different sort of binding to other parameters. There aren't two kinds of binding: 1. Parameters with no default, and parameters with early bound default, use the same old type of name binding that other local variables use (local slots in CPython, maybe a dict for other implementations); 2. and parameters with late bound defaults use a different sort of name binding, and we need a different syntax to reflect that. That is wrong: there is only one sort of binding in Python (although it can have a few different storage mechanisms: slots, cells and dict namespaces, maybe even others). The storage mechanism is irrelevant. It's all just name binding, and the implementation details are handled by the interpreter. By the time the function body is entered, all the parameters have been bound to a value. (If there are parameters that don't have a value, the interpreter raises an exception and you never enter the function body.) It doesn't matter where those values came from, whether they were passed in by the caller, or early bound defaults loaded from the cache, or late bound defaults freshly evaluated. The value is bound to the parameter in exactly the same way, and you cannot determine where that value came from. -- Steve

Steven D'Aprano "> My favorite alternative is ?= if people think => and -> are getting

overly loaded. What I really don't like is @param=[] because it puts the emphasis on the parameter name rather than the act of binding. Not only does it make it look like @param is a special kind of variable, it also mimics the *args and **kwargs syntax which makes them seem related.

But it *is* a special kind of parameter: it is a parameter that uses late binding for the default value, instead of early binding. Putting the emphasis on the parameter name is entirely appropriate. Late bound parameters don't have a different sort of binding to other parameters. There aren't two kinds of binding:" Again: the important part is the *binding*, not the name of the parameter. Right now, the order of reading something like `def load_tweets(... hash_tags: List[str] = None, ...)` goes like this: 1) there's a parameter named 'hash_tags' 2) it's a list of strings 3) it has a default value (bound at function declaration) 4) the default value is None With `def load_tweets(... @hash_tags: List[str] = [], ...)` it goes like: 1) There's some parameter bound late 2) It's a parameter named 'hash_tags' 3) It's a list of strings 4) It has a default value (of course because of information we learned 3 steps ago) 5) The default value is an empty list With `def load_tweets(... hash_tags: List[str] => [], ...)` it goes like: 1) there's a parameter named 'hash_tags' 2) it's a list of strings 3) it has a default value (bound at call-time) 4) the default value is None Notice how it makes way more sense to put related parts closer together rather than saying "here's a little information. Now put a pin in that: we'll get back to it later on". To drive the point home: The @ and = signs are connected, but not visually, which is confusing. You couldn't write @param without an '=' somewhere down the line: `def load_tweets(... @hash_tags, ...) # Error!` I probably should have been more clear about what I meant when I said "it make[s] it look like @param is a special kind of variable, it also mimics the *args and **kwargs syntax which makes them seem related." Of course the way the parameter is bound *is* special making it, in the ways you pointed out, a "special kind of variable", however, again: the specialness is all about the binding of the variable, and: unlike *args and **kwargs, @param doesn't imply any specific type (in the 'type(kwargs)' sense) for the variable. We know that args and kwargs are an iterable and mapping, respectively. The other question is: how important is the information that a parameter is late-binding. I've always been against the coding standard of making enums and constants ALL_CAPS because it makes code shouty and obnoxious just as it makes any other communication shouty and obnoxious. It doesn't seem like the fact that something is an enum or constant is important enough to warrant anything more than some syntax coloring from my IDE (if that). People always point out that the fact that something is an enum or constant is, in fact, information, and how else would they possibly communicate that information? The answer is: you communicate it by declaring an enum or constant and stop shouting at me! There's other information I you could put in your variable names that most sane people don't. You don't always append the name of the class of an object to the variable's name. You say 'hash_tags' not 'hash_tags_list_of_strings'. In the same vein, I also don't see the need here to make a big fuss about the binding behavior of a parameter such that it *must* be up-front ahead of the name, annotation, and other binding info. It may seem like a big deal now because it's new and scary so we must prefix it with "HERE_BE_DRAGONS!!!", but years down the line when late-binding params are common place, it'll be a wart. On Thursday, December 2, 2021 at 9:16:33 AM UTC-6 Steven D'Aprano wrote:

On Wed, Dec 01, 2021 at 09:58:11PM -0600, Abe Dillon wrote:

...

My favorite alternative is ?= if people think => and -> are getting overly loaded. What I really don't like is @param=[] because it puts the emphasis on the parameter name rather than the act of binding. Not only does it make it look like @param is a special kind of variable, it also mimics the *args and **kwargs syntax which makes them seem related.

But it *is* a special kind of parameter: it is a parameter that uses late binding for the default value, instead of early binding.

Putting the emphasis on the parameter name is entirely appropriate.

Late bound parameters don't have a different sort of binding to other parameters. There aren't two kinds of binding:

1. Parameters with no default, and parameters with early bound default, use the same old type of name binding that other local variables use (local slots in CPython, maybe a dict for other implementations);

2. and parameters with late bound defaults use a different sort of name binding, and we need a different syntax to reflect that.

That is wrong: there is only one sort of binding in Python (although it can have a few different storage mechanisms: slots, cells and dict namespaces, maybe even others). The storage mechanism is irrelevant. It's all just name binding, and the implementation details are handled by the interpreter.

By the time the function body is entered, all the parameters have been bound to a value. (If there are parameters that don't have a value, the interpreter raises an exception and you never enter the function body.) It doesn't matter where those values came from, whether they were passed in by the caller, or early bound defaults loaded from the cache, or late bound defaults freshly evaluated. The value is bound to the parameter in exactly the same way, and you cannot determine where that value came from.

-- Steve _______________________________________________ Python-ideas mailing list -- python...@python.org To unsubscribe send an email to python-id...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python...@python.org/message/LYRLA3TH4... <https://mail.python.org/archives/list/python-ideas@python.org/message/LYRLA3TH47H4EKOR3NCIUZKXJOICJ5R7/> Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Dec 2, 2021 at 6:27 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

On 2021-12-01 18:35, Chris Angelico wrote:

...

In my reference implementation, there is no object that stores it; it's simply part of the function. A good parallel is the if/else expression:

x = float("inf") if z == 0 else y/z

Is there an object that represents the if/else expression (or the deferred "y/z" part)? No, although it could be implemented that way if you chose:

This is not a good parallel. There is nothing deferred there. The entire line is evaluated when it is encountered and you get a result and no part of the if/else expression can ever impact anything else again unless, via some external control flow, execution returns and re-executes the entire line. That is not comparable to a function default, which is STORED and evaluated later independently of the context in which it was originally written (i.e., the function default is re-executed but the function definition itself is not re-executed).

The ternary expression vanishes without a trace by the next line, leaving only its evaluated result. There would be no use in being able to access some part of it, since the whole (i.e., the ternary expression) is completely finished by the time you would be able to access it. This is not the case with a function definition. The function definition leaves behind a function object, and that function object needs to "know" about the late-bound default as an independent entity so that it can be evaluated later. It is bad for the function to store that late-bound default only in some private format for its exclusive future use without providing any means for other code to access it as a first-class value.

That's exactly why it's such a close parallel. The late-evaluated default is just code, nothing else. It's not "stored" in any way - it is evaluated as part of the function beginning execution. There is no "first class object" for a late-evaluated default any more than there is one for the "y/z" part of the ternary. ChrisA

On Thu, Dec 2, 2021 at 6:59 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

On 2021-12-01 23:36, Chris Angelico wrote:

...

That's exactly why it's such a close parallel. The late-evaluated default is just code, nothing else. It's not "stored" in any way - it is evaluated as part of the function beginning execution.

But it IS stored! There is no way for it to be evaluated without it being stored!

I know we're talking past each other here but it is quite obvious that something has to be stored if it is going to be evaluated later. You can say that it is "just code" but that doesn't change the fact that that code has to be stored. You can say that it is just prepended to the function body but that's still storing it. That is still not parallel to a ternary operator in which no part of the expression is EVER re-executed unless control flow causes execution to return to that same source code line and re-execute it as a whole.

I'm not sure I understand you here. How is the late-bound default "stored" when one side of a ternary is "not stored"?

Actually this raises a question that maybe was answered in the earlier thread but if so I forgot: if a function has a late-bound default, will the code to evaluate it be stored as part of the function's code object?

Yes. To be precise, it is part of the code object's co_code attribute - the bytecode (or wordcode if you prefer) of the function. Here's how a ternary if looks:

...

...

def f(n): ... return 0 if n == 0 else 42/n ... dis.dis(f) 2 0 LOAD_FAST 0 (n) 2 LOAD_CONST 1 (0) 4 COMPARE_OP 2 (==) 6 POP_JUMP_IF_FALSE 6 (to 12) 8 LOAD_CONST 1 (0) 10 RETURN_VALUE >> 12 LOAD_CONST 2 (42) 14 LOAD_FAST 0 (n) 16 BINARY_TRUE_DIVIDE 18 RETURN_VALUE

The "42/n" part is stored in f.__code__.co_code as the part that says "LOAD_CONST 42, LOAD_FAST n, BINARY_TRUE_DIVIDE". It's not an object. It's just code - three instructions. Here's how (in the reference implementation - everything is subject to change) a late-bound default looks:

...

...

def f(x=>[]): print(x) ... dis.dis(f) 1 0 QUERY_FAST 0 (x) 2 POP_JUMP_IF_TRUE 4 (to 8) 4 BUILD_LIST 0 6 STORE_FAST 0 (x) >> 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 CALL_FUNCTION 1 14 POP_TOP 16 LOAD_CONST 0 (None) 18 RETURN_VALUE

The "=>[]" part is stored in f.__code__.co_code as the part that says "QUERY_FAST x, and if false, BUILD_LIST, STORE_FAST x". It's not an object. It's four instructions in the bytecode. In both cases, no part of the expression is ever re-executed. I'm not understanding the distinction here. Can you explain further please? ChrisA

On Fri, Dec 3, 2021 at 12:43 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

On Thu, Dec 2, 2021 at 3:33 AM Chris Angelico <rosuav@gmail.com> wrote:

...

...

But it IS stored! There is no way for it to be evaluated without it being stored!

I'm not sure I understand you here. How is the late-bound default "stored" when one side of a ternary is "not stored"?

This seems like dishonest argument. I'm not even sure what point you think it is making.

My point was a response to a claim that a late-bound default fundamentally has to be stored somewhere, yet half of a ternary conditional isn't. My point was that there is no difference, and they are both code. Please examine the surrounding context and respond in more detail, rather than calling my argument "dishonest". ChrisA

On Fri, Dec 3, 2021 at 9:26 AM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

On 2021-12-02 00:31, Chris Angelico wrote:

...

Here's how a ternary if looks:

...

...

...

...

>>def f(n): ... return 0 if n == 0 else 42/n ... >>dis.dis(f) 2 0 LOAD_FAST 0 (n) 2 LOAD_CONST 1 (0) 4 COMPARE_OP 2 (==) 6 POP_JUMP_IF_FALSE 6 (to 12) 8 LOAD_CONST 1 (0) 10 RETURN_VALUE >> 12 LOAD_CONST 2 (42) 14 LOAD_FAST 0 (n) 16 BINARY_TRUE_DIVIDE 18 RETURN_VALUE

The "42/n" part is stored in f.__code__.co_code as the part that says "LOAD_CONST 42, LOAD_FAST n, BINARY_TRUE_DIVIDE". It's not an object. It's just code - three instructions.

Here's how (in the reference implementation - everything is subject to change) a late-bound default looks:

...

...

...

...

>>def f(x=>[]): print(x) ... >>dis.dis(f) 1 0 QUERY_FAST 0 (x) 2 POP_JUMP_IF_TRUE 4 (to 8) 4 BUILD_LIST 0 6 STORE_FAST 0 (x) >> 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 CALL_FUNCTION 1 14 POP_TOP 16 LOAD_CONST 0 (None) 18 RETURN_VALUE

The "=>[]" part is stored in f.__code__.co_code as the part that says "QUERY_FAST x, and if false, BUILD_LIST, STORE_FAST x". It's not an object. It's four instructions in the bytecode.

In both cases, no part of the expression is ever re-executed. I'm not understanding the distinction here. Can you explain further please?

Your explanation exactly shows how it IS re-executed. I'm not totally clear on this disassembly since this is new behavior, but if I understand right, BUILD_LIST is re-executing the expression `[]` and STORE_FAST is re-assigning it to x. The expression `[]` is syntactically present in the function definition but its execution has been shoved into the function body where it may be re-executed many times (any time the function is called without passing a value).

Ah, I think I get you. The problem is that code is in the def line but is only executed when the function is called, is that correct? Because the code would be "re-executed" just as much if it were written in the function body. It's executed (at most) once for each call to the function, just like the ternary's side is. I suppose that's a consideration, but it's not nearly as strong in practice as you might think. A lot of people aren't even aware of the difference between compilation time and definition time (even people on this list have made that mistake). Function default args are executed when the function is defined, not when it's called, and that's something that changes with this proposal; but there are many other subtleties to execution order and timing that don't really matter in practice. Perhaps the key point here is to consider function decorators. We could avoid them altogether: def f(): @deco def g(x): ... def h(x): ... h = deco(h) But as well as having the name replication problem, this buries important information down in the body of the surrounding code, rather than putting it at the signature of g/h where it belongs. Even though, semantically, this is actually part of the body of f, we want to be able to read it as part of the signature of g. Logically and conceptually, it is part of the signature. Now compare these two: def f2(): _SENTINEL = object() def g(x=_SENTINEL): if x is _SENTINEL: x = [] ... def h(x=>[]): ... Which one has its signature where its signature belongs? Yes, semantically, the construction of the empty list happens at function call time, not at definition time. But what you're saying is: if there are no args passed, behave as if a new empty list was passed. That's part of the signature. In neither case will you find an object representing the expression [] in the function's signature, because that's not an object, it's an instruction to build an empty list. In the case of g, you can find a meaningless and useless object stashed away in __defaults__, but that doesn't tell you anything about the true behaviour of the function. At least in the case of h, you can find the descriptive string "[]" stashed there, which can tell a human what's happening. ChrisA

On 02/12/2021 07:05, Brendan Barnwell wrote:

On 2021-12-01 18:35, Chris Angelico wrote:

...

In my reference implementation, there is no object that stores it; it's simply part of the function. A good parallel is the if/else expression:

x = float("inf") if z == 0 else y/z

Is there an object that represents the if/else expression (or the deferred "y/z" part)? No, although it could be implemented that way if you chose:

    This is not a good parallel.  There is nothing deferred there.  The entire line is evaluated when it is encountered and you get a result and no part of the if/else expression can ever impact anything else again unless, via some external control flow, execution returns and re-executes the entire line.  That is not comparable to a function default, which is STORED and evaluated later independently of the context in which it was originally written (i.e., the function default is re-executed but the function definition itself is not re-executed).

    The ternary expression vanishes without a trace by the next line, leaving only its evaluated result.  There would be no use in being able to access some part of it, It could be useful for debugging. Best wishes Rob Cliffe since the whole (i.e., the ternary expression) is completely finished by the time you would be able to access it.  This is not the case with a function definition.  The function definition leaves behind a function object, and that function object needs to "know" about the late-bound default as an independent entity so that it can be evaluated later.  It is bad for the function to store that late-bound default only in some private format for its exclusive future use without providing any means for other code to access it as a first-class value.

On 05/12/2021 15:16, Steven D'Aprano wrote:

On Wed, Dec 01, 2021 at 12:26:33PM +0000, Matt del Valle wrote:

I'm sure that people will learn the many uses of the arrow symbols. After all, people learn Perl and APL :-) Closer to home, we also learn all the many different uses of the star symbol `*`.

Not to mention the half-dozen or so uses of colon (introduce a suite, slicing, dictionary displays, annotations, lambdas).  Anybody still have a problem with them?  Raise your hand and I'll give you your money back.😁 Best wishes Rob Cliffe

On Thu, Dec 2, 2021 at 1:33 AM André Roberge <andre.roberge@gmail.com> wrote:

On Wed, Dec 1, 2021 at 7:51 AM Chris Angelico <rosuav@gmail.com> wrote:

...

On Wed, Dec 1, 2021 at 10:30 PM André Roberge <andre.roberge@gmail.com> wrote:

...

...

2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden?

Yes. I really think that using a keyword like defer, or from_calling_scope ;-), would significantly reduce the cognitive burden.

Also fair. I'm not a fan of keywords for this sort of thing, since it implies that you could do this:

def f(x=defer []): ...

dflt = defer [] def f(x=dflt): ...

which is a completely different proposal (eg it would be evaluated only when you "touch" that, rather than being guaranteed to be evaluated before the first line of the function body). That's why I want to adorn the equals sign and nothing else.

Shouldn't the PEP contain a rejected idea section where this could be mentioned?

Hmm, maybe. It's such a completely different proposal, but it does get asked a few times. If someone could actually put it forward as a full proposal, I'd gladly mention it as an interaction with another PEP. Otherwise, I'll write up a very brief thing on deferred expressions and how they're not what this is about. ChrisA

On Wed, Dec 1, 2021 at 10:12 AM Chris Angelico <rosuav@gmail.com> wrote:

On Thu, Dec 2, 2021 at 12:42 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

...

...

4) If "no" to question 1, is there some other spelling or other small change that WOULD mean you would use it? (Some examples in the PEP.)

Yes, the delay/later/defer keyword approach is not confusing, and does not preempt a later feature that would actually be worth having.

Do you mean changing the spelling of the existing proposal, or a completely different proposal for deferred objects that are actually objects? Because that is NOT what I mean by a "small change". :)

The spelling of the existing proposal. I.e. if the proposal were: def fun(things: list[int] = defer []) -> int: # ... some implementation I'd be -0 on the idea rather than -100. Yes, my change in attitude is largely because I want *some future PEP* to address the more general situation like: result = defer really_expensive_calculation() if predicate: doubled = result * 2 But I do not think your PEP does (nor even should) include that potential future behavior/syntax. Such a hypothetical future PEP would have a continuity with the syntax of your feature, albeit DEFINITELY need to address many independent concerns/issues that yours does not create. However, even if I assume the mythical future PEP never happens, in terms of readability, a WORD is vastly less confusing than a combination of punctuation that has no obvious or natural interpretation like '=>'. Or rather, I think that spelling is kinda-sorta obvious for the lambda meaning, and the use you want is kinda-sorta similar to a lambda. So I *do* understand how you get there... but it still seems like much too much line noise for a very minimal need. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 01/12/2021 15:39, David Mertz, Ph.D. wrote:

However, even if I assume the mythical future PEP never happens, in terms of readability, a WORD is vastly less confusing than a combination of punctuation that has no obvious or natural interpretation like '=>'.  Or rather, I think that spelling is kinda-sorta obvious for the lambda meaning, and the use you want is kinda-sorta similar to a lambda.  So I *do* understand how you get there... but it still seems like much too much line noise for a very minimal need.

Hm.  A word is "vastly less confusing".  OK.  Should Python have been designed with     x assigned y+1 rather than     x = y +1 (note that '=' here does not have its "obvious or natural interpretation", viz. "is equal to"). Should we have     (x becomes y+1) rather than     (x := y+1) Conciseness *is* a virtue, even if it is often trumped by other considerations. Best wishes Rob Cliffe

On Thu, Dec 2, 2021 at 1:21 AM Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, 1 Dec 2021 at 06:19, Chris Angelico <rosuav@gmail.com> wrote:

...

3) If "yes" to question 1, would you use it for any/all of (a) mutable defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else?

N/A, except to say that when you enumerate the use cases like this, none of them even tempt me to use this feature.

I'm just listing some of the common use-cases, not all of them.

I think that the only thing I might use it for is to make it easier to annotate defaults (as f(a: list[int] => []) rather than as f(a: list[int] | None = None).

That's exactly the goal! It's hugely simpler to say "this is always a list, and defaults to a new empty list" than "this is a list or None, defaults to None, and hey, if it's None, I'll make a new empty list".

So I'll revise my answer to (1) and say that I *might* use this, but only in a way it wasn't intended to be used in, and mostly because I hate how verbose it is to express optional arguments in type annotations. (And the fact that the annotation exposes the sentinel value, even when you want it to be opaque). I hope I don't succumb and do that, though ;-)

That's definitely an intended use-case ("mutable defaults" above); you're focusing on the fact that it improves the annotations, I'm focusing on the fact that it improves documentation and introspection, but it's all the same improvement :)

...

5) Do you know how to compile CPython from source, and would you be willing to try this out? Please? :)

Sorry, I really don't have time to, in the foreseeable future. If I did have time, one thing I would experiment with is how this interacts with typing and tools like pyright and mypy (yes, I know type checkers would need updating for the new syntax, so that would mostly be a thought experiment) - as I say, I'd expect to annotate a function with an optional list argument defaulting to an empty list as f(a: list[int] => []), which means that __annotations__ needs to distinguish between this case and f(a: list[int]) with no default.

No problem, yup.

Sorry, I don't have any feedback like that. What I can say, though, is I'd find it quite hard to express the question, in the sense that I'd struggle to explain the difference between early and late bound parameters to a non-expert, much less explain why we need both. I'd probably just say "it's short for a default of None and a check" which doesn't really capture the point...

The point is that it's a shortcut for "omitted" rather than "a default of None", but there are many ways to explain it, and I only have one brother who knows enough Python to be able to talk to about this (and I got feedback from him VERY early).

...

Any and all comments welcomed. I mean, this is python-ideas after all... bikeshedding is what we do best!

I hope this was useful feedback.

It was. Thank you. ChrisA

On Thu, Dec 2, 2021 at 2:24 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

On Wed, Dec 1, 2021 at 9:24 AM Paul Moore <p.f.moore@gmail.com> wrote:

...

I think that the only thing I might use it for is to make it easier to annotate defaults (as f(a: list[int] => []) rather than as f(a: list[int] | None = None).

Why not `f(a: Optional[list[int]] = None)`?

I'm not counting characters, but that form seems to express the intention better than either of the others IMHO.

"a: list[int] => []" fully expresses the intention IMO, but yes, Optional is usually better than "| None". That's just a minor spelling problem though. ChrisA

On 2/12/21 4:40 am, Paul Moore wrote:

the intended use is that people must supply a list[int] or not supply the argument *at all*.

I don't think this is a style of API that we should be encouraging people to create, because it results in things that are very awkward to wrap. -- Greg

On Thu, 2 Dec 2021 at 08:29, Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, 1 Dec 2021 at 22:27, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

On 2/12/21 4:40 am, Paul Moore wrote:

...

the intended use is that people must supply a list[int] or not supply the argument *at all*.

I don't think this is a style of API that we should be encouraging people to create, because it results in things that are very awkward to wrap.

Hmm, interesting point, I agree with you. It's particularly telling that I got sucked into designing that sort of API, even though I know it's got this problem. I guess that counts as an argument against the late bound defaults proposal - or maybe even two:

1. It's hard (if not impossible) to wrap functions that use late-bound defaults. 2. The feature encourages people to write such unwrappable functions when an alternative formulation that is wrappable is just as good.

(That may actually only be one point - obviously a feature encourages people to use it, and any feature can be over-used. But the point about wrappability stands).

Actually, Chris - does functools.wraps work properly in your implementation when wrapping functions with late-bound defaults?

...

...

def dec(f): ... @wraps(f) ... def inner(*args, **kw): ... print("Calling") ... return f(*args, **kw) ... return inner ... @dec ... def g(a => []): ... return len(a)

Paul

On Thu, Dec 2, 2021 at 7:31 PM Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, 1 Dec 2021 at 22:27, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

On 2/12/21 4:40 am, Paul Moore wrote:

...

the intended use is that people must supply a list[int] or not supply the argument *at all*.

I don't think this is a style of API that we should be encouraging people to create, because it results in things that are very awkward to wrap.

Hmm, interesting point, I agree with you. It's particularly telling that I got sucked into designing that sort of API, even though I know it's got this problem. I guess that counts as an argument against the late bound defaults proposal - or maybe even two:

1. It's hard (if not impossible) to wrap functions that use late-bound defaults. 2. The feature encourages people to write such unwrappable functions when an alternative formulation that is wrappable is just as good.

(That may actually only be one point - obviously a feature encourages people to use it, and any feature can be over-used. But the point about wrappability stands).

Wrappability when using an arbitrary sentinel looks like this: _SENTINEL = object() def f(value=_SENTINEL): if value is _SENTINEL: value = ... # in another module def wrap_f(): othermodule.f(othermodule._SENTINEL) Or: def wrap_f(): othermodule.f(othermodule.f.__defaults__[0]) I'm not sure either of these is any better than the alternatives. You're reaching into a module to access its internal implementation details. Is that really better than messing with *args? def wrap_f(): args = [42] or [] # pass or don't pass? othermodule.f(*args) It only looks easy when None is used, and even then, it's largely an implementation detail. ChrisA

On 1 Dec 2021, at 17:59, Chris Angelico <rosuav@gmail.com> wrote:

On Thu, Dec 2, 2021 at 4:40 AM Barry Scott <barry@barrys-emacs.org> wrote:

...

On 1 Dec 2021, at 06:16, Chris Angelico <rosuav@gmail.com> wrote: 3) If "yes" to question 1, would you use it for any/all of (a) mutable defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else?

yes (a) What does (b) mean? example please. yes (c)

global_default = 500 def do_thing(timeout=>global_default): ...

If the global_default timeout changes between function definition and call, omitting timeout will use the updated global.

Similarly, you could say "file=>sys.stdout" and if code elsewhere changes sys.stdout, you'll use that.

On a case-by-case basis I might still put defaulting into the body of the function if that made the intent clearer. I could see me using @file=sys.stdout. Barry

ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KVLO3C... Code of Conduct: http://python.org/psf/codeofconduct/

2% of functions is a lot of functions. We're talking about a language that's been around 30 years. The feature set is pretty mature. If it were lacking features that would improve a much larger percent of code for so long, I don't think it would be as popular. It's not like the next PIP is going to be as big as the for-loop. On Wednesday, December 1, 2021 at 9:56:40 PM UTC-6 David Mertz, Ph.D. wrote:

On Wed, Dec 1, 2021, 10:38 PM Steven D'Aprano

...

"If param is missing **or None**, the default if blah..." I reject Chris' characterisation of this as a hack. There are function parameters where None will *never* in any conceivable circumstances become a valid argument value, and it is safe to use it as a sentinel.

In particular, the cases where None will never, in any conceivable circumstances, be a non-sentinel value are at least 98% of all the functions (that have named parameters) I've ever written in Python.

All of this discussion of a syntax change is for, at most, 2% of functions that need a different sentinel.

On Wed, Dec 01, 2021 at 09:00:50PM -0800, abed...@gmail.com wrote:

Steven D'Aprano ""If param is missing **or None**, the default if blah..."

The bottom line is:

you *don't actually* want the parameter to default to the value of a sentinel.

Yes I do. I *do* want to be able to give a convenient sentinel value in order to explicitly tell the function "give me the default value".

you *have* to use that hack because you can't express what you want the default to actually be.

The point of having default values is so that the caller doesn't have to express what the default will actually be. If the caller has to express that value, it's not a default, it's a passed-in argument.

You're doing something misleading to work around a shortcoming of the language.

How is it misleading? The parameter is explicitly documented as taking None to have a certain effect. None is behaving here as a convenient, common special constant to trigger a certain behaviour, no different than passing (for example) buffering=-1 to open() to trigger a very complex set of behaviour. (With buffering=-1, the buffer depends on the platform details, and whether the file is binary, text, and whether or not it is a tty.) For historical reasons, probably related to C, the default value for buffering is -1. But it could have just as easily be None, or the string "default", or an enumeration, or some builtin constant. -- Steve

On Thu, Dec 02, 2021 at 01:20:27PM -0800, abed...@gmail.com wrote:

If you want the parameter to default to an empty list, it's categorically more explicit to have the parameter default to an empty list.

You've gone from talking about *my* intentions, which you got wrong, to an argument about which is "more explicit". In the simple example of the default being a new empty list: # Using Chris' preferred syntax, not mine def func(arg=>[]): then I agree: once you know what the syntax means, then it is more explicit at telling the reader that the default value is an empty list. Great! This sort of thing is the primary use-case of the feature. But let's talk about more complex examples: def open(filename, mode='r', buffering=-1): ... How would you make that "more explicit"? def open(filename, mode='r', buffering=>( 1 if 'b' not in mode and os.fdopen(os.open(filename)).isatty() else _guess_system_blocksize() or io.DEFAULT_BUFFER_SIZE ) ): ... Too much information! We're now drowning in explicitness. Now how do I, the caller, *explicitly* indicate that I want to use that system-dependent value? I just want a nice, simple value I can explicitly pass as buffering to say "just use the default". with open('myfile.txt', buffering= what? ) as f: I can't. So in this case, your agonisingly explicit default expression forces the caller to be *less* explicit when they call the function. What the right hand giveth, the left hand taketh away. Can we agree that, like salt, sugar, and bike-shedding, sometimes you can have *too much* explicitness in code? Sometimes the right level of explicitness is to have an agreed upon symbol that acts as a short, easy to read and write, documented and explicit signal to the function "give me the default value". And that symbol can be -1, or an enumeration, or a string, or None. This is not a hack, and its not "implicit".

None of my comment had anything to do with the caller's perspective.

The caller's perspective is important. The caller has to read the function signature (either in the documentation, or if they use `help(function)` in the interpreter, or when their IDE offers up tooltips or argument completion). The caller has to actually use the function. We should consider their perspective.

If my intent is that I want a parameter to default to an empty list, I wouldn't assign it to an object called "monkey_pants" by default then add code to the body of the function to check if it's "monkey_pants".

Funny you should say that, but *totally by coincidence*, the Bulgarian word for "default" happens to be transliterated into English as *monkeypants*, so if I were Bulgarian that is exactly what I might do! How weird is that? Nah, I'm kidding of course. But the point I am making is that you are mocking the concept by using a totally silly example. "monkey_pants" indeed, ha ha how very droll. But if you used a more meaningful symbol, like "default", or some enumeration with a meaningful name, or a value which is a standard idiom such as -1, then the idea isn't so ridiculous. And None is one such standard idiom. *Especially* when the default value is not some trivially simple expression such as an empty list, but a more complicated expression such as the default buffering used by open files, or in my earlier example of the default collation (a sequence of letters which depends on the other input arguments).

Steven D'Aprano "How is it misleading?"

Your way of having a parameter default to an empty list (or whatever) is by having the parameter default to None. That's how it's misleading.

All you have done is repeat that it is misleading. I'm still no clearer why you think it is misleading to document "if the argument is missing or None, the default behaviour is ...". Is the documentation false? Does the function not do exactly what it says it will do if you pass that symbol? Let's be more concrete. Do you feel that using -1 for the buffer size in open() is "misleading"? Do you think that there is ever a possibility that someday Python's file I/O will use a buffer with *literally* a size of -1? -- Steve

On Fri, Dec 3, 2021 at 3:47 PM Abe Dillon <abedillon@gmail.com> wrote:

This seems like an exceedingly minor complaint. The obvious answer is: you invoke the default behaviour by not passing the argument. And yes, I know that's less explicit, but that's kinda the whole point of defaults to begin with. It's behavior when the user decides not to specify their own. Nobody's asking "what do I pass to logging.config.fileConfig to explicitly invoke the logging module's default behavior?!" The answer is simply: don't configure logging. It's not rocket science.

I'd say it isn't "less explicit". It's actually the ultimate definition of not doing something: you just... don't do it. The same line of explanation is important when working with callback functions: def callback(*a): print("Doing stuff") button.signal_connect("clicked", callback()) Oops, you called the callback instead of passing it to the signal_connect function. What's the solution? DON'T call the callback at this point. How do you NOT call something? Well, how do you call it? You put parentheses after it. So how do you not call it? You don't put the parentheses after it. Python is fairly good at giving you syntactic features that do things, such that you can not do them by omitting them. So I don't think anyone is truly concerned about the pure "don't pass the argument" case. The problem is the "maybe pass the argument" case. That ends up looking like this: c = 42 if cond: func(a, b) else: func(a, b, c) # or args = {} if not cond: args["c"] = 42 func(a, b, **args) # or if you know what the default is: c = some_default if not cond: c = 42 func(a, b, c) The first two will, of course, continue to work. The second one is a fairly clear way to indicate whether or not a parameter should get a value. The problem with the third is that it requires a documented and future-guaranteed default value, published and promised to work, and that simply isn't always possible or practical. So, yes, some forms of default might be a little unclear. And if you're converting an existing codebase, it may be necessary to maintain "if c is None: c = some_default" in the body forever, or until such time as you can remove the support for None as a parameter. But None isn't truly the default value.

"Can we agree that, like salt, sugar, and bike-shedding, sometimes you can have *too much* explicitness in code?"

Surely *you* would never agree that there's such a thing as *too much* bike-shedding! Isn't that your raison d'être?!

I never said one had to be a Nazi about stamping out every use of the sentinel... er... idiom (if you prefer it to hack).

Idiom. Sentinels are extremely important, and PEP 671 isn't proposing to eliminate them. The "hack" part comes when, for technical reasons, a sentinel is used when one isn't logically part of the API. I agree with Steve here: sometimes, the true default is extremely clunky, and it's better to use a sentinel. That's part of why they're never going away (and None will continue to be a popular sentinel). PEP 671 doesn't propose changing every function ever written, because the vast majority of them are perfectly fine as they are.

"The caller's perspective is important. The caller has to read the function signature (either in the documentation, or if they use `help(function)` in the interpreter, or when their IDE offers up tooltips or argument completion). The caller has to actually use the function. We should consider their perspective."

I never said we shouldn't. That's just not what I was talking about at that time and you were misinterpreting my argument.

Yes, absolutely. And that's why the function should be able to put the information in the signature when it's appropriate to, and in the body when it isn't.

Just because it's an idiom, doesn't mean it's actually good. I think the `if __name__ == "__main__":` idiom is a horrible, round-about way of saying "if this file is being run as a script". You might look at that and say "What do you mean? That's exactly what it says!" because you've seen the idiom so many times, just like how you think "x=None" explicitly says "give x the default value", but if you look closely `if __name__ == "__main__":` *doesn't* actually say anything like "if this file is being run as a script", it says a bunch of esoteric and confusing garbage that you've learned to read as "if this file is being run as a script".

TBH I think that the name-is-main idiom isn't too bad, but mainly because I've seen far worse idioms. If someone has a better proposal, I'd be happy to hear it, but the existing idiom isn't horrible. (But you're right. Well-known idioms can indeed be really REALLY bad. Just look at the sorts of things that pre-C99 C code would do.) ChrisA

"# or if you know what the default is: c = some_default if not cond: c = 42 func(a, b, c)" I would argue that, of the three examples; this one is the most problematic because it overly couples your code to an implementation detail of the function and makes it unnecessarily brittle to change. On top of that, in the case of a sentinel; it might even encourage people to circumvent the "guarantees" that your API relies upon. For example, I might make a function where the optional parameter could be anything even a None is valid, so I make my own secret sentinel that, as Steven D'Aprano put it "will *never* in any conceivable circumstances become a valid argument value": _default = object() def func(a, b, c=_default): ... Now the pattern above violates the "nobody will *ever* pass me this" condition because it imports _default. Then some intern sees your code and starts importing _default and using it elsewhere because it's such a neat pattern... "The "hack" part comes when, for technical reasons, a sentinel is used when one isn't logically part of the API." That's my entire point. "TBH I think that the name-is-main idiom isn't too bad" The bad part is a lot of Python newbies see it on day one or two because it's so common and try explaining it without talking about the interpreter and watching their eyes glaze over. You just showed them how to assign values to variables and that unbound variables cause errors and now you're pulling __name__ out of thin air? Where the heck did that come from? Anyway, this is a tangent I promised myself I wouldn't go on... On Thu, Dec 2, 2021 at 11:16 PM Chris Angelico <rosuav@gmail.com> wrote:

On Fri, Dec 3, 2021 at 3:47 PM Abe Dillon <abedillon@gmail.com> wrote:

...

This seems like an exceedingly minor complaint. The obvious answer is: you invoke the default behaviour by not passing the argument. And yes, I know that's less explicit, but that's kinda the whole point of defaults to begin with. It's behavior when the user decides not to specify their own. Nobody's asking "what do I pass to logging.config.fileConfig to explicitly invoke the logging module's default behavior?!" The answer is simply: don't configure logging. It's not rocket science.

I'd say it isn't "less explicit". It's actually the ultimate definition of not doing something: you just... don't do it. The same line of explanation is important when working with callback functions:

def callback(*a): print("Doing stuff")

button.signal_connect("clicked", callback())

Oops, you called the callback instead of passing it to the signal_connect function. What's the solution? DON'T call the callback at this point. How do you NOT call something? Well, how do you call it? You put parentheses after it. So how do you not call it? You don't put the parentheses after it.

Python is fairly good at giving you syntactic features that do things, such that you can not do them by omitting them.

So I don't think anyone is truly concerned about the pure "don't pass the argument" case. The problem is the "maybe pass the argument" case. That ends up looking like this:

c = 42 if cond: func(a, b) else: func(a, b, c)

# or args = {} if not cond: args["c"] = 42 func(a, b, **args)

# or if you know what the default is: c = some_default if not cond: c = 42 func(a, b, c)

The first two will, of course, continue to work. The second one is a fairly clear way to indicate whether or not a parameter should get a value. The problem with the third is that it requires a documented and future-guaranteed default value, published and promised to work, and that simply isn't always possible or practical.

So, yes, some forms of default might be a little unclear. And if you're converting an existing codebase, it may be necessary to maintain "if c is None: c = some_default" in the body forever, or until such time as you can remove the support for None as a parameter. But None isn't truly the default value.

...

"Can we agree that, like salt, sugar, and bike-shedding, sometimes you can have *too much* explicitness in code?"

Surely *you* would never agree that there's such a thing as *too much* bike-shedding! Isn't that your raison d'être?!

I never said one had to be a Nazi about stamping out every use of the sentinel... er... idiom (if you prefer it to hack).

Idiom. Sentinels are extremely important, and PEP 671 isn't proposing to eliminate them. The "hack" part comes when, for technical reasons, a sentinel is used when one isn't logically part of the API.

I agree with Steve here: sometimes, the true default is extremely clunky, and it's better to use a sentinel. That's part of why they're never going away (and None will continue to be a popular sentinel). PEP 671 doesn't propose changing every function ever written, because the vast majority of them are perfectly fine as they are.

...

"The caller's perspective is important. The caller has to read the function signature (either in the documentation, or if they use `help(function)` in the interpreter, or when their IDE offers up tooltips or argument completion). The caller has to actually use the function. We should consider their perspective."

I never said we shouldn't. That's just not what I was talking about at that time and you were misinterpreting my argument.

Yes, absolutely. And that's why the function should be able to put the information in the signature when it's appropriate to, and in the body when it isn't.

...

Just because it's an idiom, doesn't mean it's actually good. I think the `if __name__ == "__main__":` idiom is a horrible, round-about way of saying "if this file is being run as a script". You might look at that and say "What do you mean? That's exactly what it says!" because you've seen the idiom so many times, just like how you think "x=None" explicitly says "give x the default value", but if you look closely `if __name__ == "__main__":` *doesn't* actually say anything like "if this file is being run as a script", it says a bunch of esoteric and confusing garbage that you've learned to read as "if this file is being run as a script".

TBH I think that the name-is-main idiom isn't too bad, but mainly because I've seen far worse idioms. If someone has a better proposal, I'd be happy to hear it, but the existing idiom isn't horrible.

(But you're right. Well-known idioms can indeed be really REALLY bad. Just look at the sorts of things that pre-C99 C code would do.)

ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QM2ALI... Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Dec 2, 2021 at 6:12 PM <role.pythonorg-readers@jlassocs.com> wrote:

Brendan Barnwell wrote:

...

...

No. As I mentioned in the earlier thread, I don't support any proposal in which an argument can "have a default" but that default is not a first-class Python object of some sort.

What if a default is a function?

I was inspired by learning Django and saw in models that fields can have a default which is either a regular (early-bound) default such as a first-class Python object as one would expect, *or* a function -- which will be called 'later' when needed.

That prompted me to contemplate a syntax for late-bound defaults, albeit a bit clunky, but I did think it suited a special-case requirement met by late-bound defaults. I still think that littering function arguments throughout all code with large numbers of arrows would make things less readable. Special requirements need special treatment, I'm thinking.

The problem is passing arguments to such a function without it looking like it's being called at definition time.

Also has the same problem of other deferreds, which is: when exactly is it evaluated? def func(stuff, n=>len(stuff)): stuff.append("spam") print(n) func(["spam", "ham"]) What will this print? If it's a function default, it MUST print 2, since len(stuff) is 2 as the function starts. But if it's a deferred object of some sort, then it should probably print 3, since len(stuff) is 3 at the time that n gets printed. Which is it to be? That's why PEP 671 is *not* about generic deferred evaluation. It is specifically about function default arguments, and guarantees that they will be evaluated prior to the first line of code in the function body. ChrisA

On Thu, Dec 2, 2021 at 8:40 PM Steven D'Aprano <steve@pearwood.info> wrote:

Depending on the implementation, you *might* be able to inspect the function and see the default expression as some sort of callable function, or evaluatable code object. (That would be nice.)

Unfortunately not, since the default expression could refer to other parameters, or closure variables, or anything else from the context of the called function. So you won't be able to externally evaluate it.

Or even as a plain old string. All of which are first-class objects.

For documentation purposes, it is indeed available as a plain old string. In colloquial terms, it is the source code for the expression, although technically it's reconstructed from the AST. ( This is approximately as accurate as saying that the repr of an object is its source code. Possibly more accurate, actually.)

Or it might be that the default expression will be compiled into the body of the function, where is it effectively invisible. So I guess that's a third change: when, how often, and the difference to introspection.

Among our changes are when, how often, the difference to introspection, and the ability to externally manipulate the defaults. And nice red uniforms. Although the ability to manipulate them could be considered part of introspection, not sure. I'm still unsure whether this is a cool feature or an utter abomination:

...

...

def f(x=...): ... try: print("You passed x as", x) ... except UnboundLocalError: print("You didn't pass x") ... f.__defaults_extra__ = ("n/a",) f(42) You passed x as 42 f() You didn't pass x

(It's an implementation detail and not part of the specification, but if CPython adopts this behaviour, it will become de facto part of the expected behaviour, and someone somewhere will use this deliberately.) ChrisA

On 02/12/2021 14:47, Steven D'Aprano wrote:

...

...

...

...

def f(x=...): ... try: print("You passed x as", x) ... except UnboundLocalError: print("You didn't pass x") ... f.__defaults_extra__ = ("n/a",) f(42) You passed x as 42 f() You didn't pass x That is absolutely an abomination. If your implementation has the side-effect that setting a regular early-bound default to Ellipsis makes

On Thu, Dec 02, 2021 at 11:00:33PM +1100, Chris Angelico wrote: I'm still unsure whether this is a cool feature or an utter abomination: the parameter unable to retrieve the default, then the implementation is fatally broken.

It absolutely is not a feature.

It's backward incompatible: 15:03:04 R:\>python Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:20:19) [MSC v.1925 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.

...

...

def f(x=...): ...     try: print("You passed x as", x) ...     except UnboundLocalError: print("You didn't pass x") ... f() You passed x as Ellipsis

So I must agree with Steven that this should not be a feature. Best wishes Rob Cliffe

On Fri, Dec 3, 2021 at 1:53 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, Dec 02, 2021 at 11:00:33PM +1100, Chris Angelico wrote:

...

On Thu, Dec 2, 2021 at 8:40 PM Steven D'Aprano <steve@pearwood.info> wrote:

...

Depending on the implementation, you *might* be able to inspect the function and see the default expression as some sort of callable function, or evaluatable code object. (That would be nice.)

Unfortunately not, since the default expression could refer to other parameters, or closure variables, or anything else from the context of the called function. So you won't be able to externally evaluate it.

Why not? Functions can do all those things: refer to other variables, or closures, or anything else. You can call functions. Are you sure that this limitation of the default expression is not just a limitation of your implementation?

def f(): a = 1 def f(b, c=>a+b): return c a = 2 return f If there were a function to represent the late-bound default value for c, what parameters should it accept? How would you externally evaluate this? And also: what do you gain by it being a function, other than a lot of unnecessary overhead? And it is potentially a LOT of unnecessary overhead. Consider this edge case: def f(a, b=>c:=len(a)): ... In what context should the name c be bound? If there's a function for the evaluation of b, then that implies making c a closure cell, just for the sake of that. Every reference to c anywhere in the function (not just where it's set to len(a), but anywhere in f()) has to dereference that. It's a massive amount of completely unnecessary overhead AND a difficult question of which parts belong in the closure and which parts belong as parameters, which means that this is nearly impossible to define usefully.

...

I'm still unsure whether this is a cool feature or an utter abomination:

...

...

...

def f(x=...): ... try: print("You passed x as", x) ... except UnboundLocalError: print("You didn't pass x") ... f.__defaults_extra__ = ("n/a",) f(42) You passed x as 42 f() You didn't pass x

That is absolutely an abomination. If your implementation has the side-effect that setting a regular early-bound default to Ellipsis makes the parameter unable to retrieve the default, then the implementation is fatally broken.

It absolutely is not a feature.

That's not what the example shows. It shows that changing dunder attributes can do this. I'm not sure why you think that the implementation is as restricted as you imply. The assignment to __defaults_extra__ is kinda significant here :) ChrisA

On Fri, Dec 3, 2021 at 12:48 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Fri, Dec 03, 2021 at 02:10:12AM +1100, Chris Angelico wrote:

...

...

...

Unfortunately not, since the default expression could refer to other parameters, or closure variables, or anything else from the context of the called function. So you won't be able to externally evaluate it.

Why not? Functions can do all those things: refer to other variables, or closures, or anything else. You can call functions. Are you sure that this limitation of the default expression is not just a limitation of your implementation?

def f(): a = 1 def f(b, c=>a+b): return c a = 2 return f

If there were a function to represent the late-bound default value for c, what parameters should it accept?

I'm not saying that it *must* be a function. It could be a bare code object, that is `eval()`ed. Or something completely new. Dunno.

But you're saying something is impossible, and that seems implausible to me, because things that seems *very similar* are totally possible.

...

How would you externally evaluate this?

inner = f() # Get the inner function. default = inner.__code__.__late_defaults__.wibble[1] # whatever try: value = default() except NameError: # Well, what did you expect to happen? value = eval(default.__code__, globals(), {'a': 101, 'b': 202})

Or something. The point is, rather than dismissing the possibility outright, this should be something we discuss, and carefully consider, before the PEP is complete.

How, with external calling, are you going to know which name references to look up, and where to get their values from? This is already a virtually impossible task in Python, which is why f-strings cannot be implemented as str.format(**something) for any known meaning of "something". You cannot get your *current* variable set, much less the available variables in some other context.

...

And it is potentially a LOT of unnecessary overhead. Consider this edge case:

def f(a, b=>c:=len(a)): ...

In what context should the name c be bound?

With late-bound defaults, the expression is evaluated at function call time, in the scope of f()'s locals. So c would be a local.

In other words, if you're trying to evaluate b's default externally, you have to set c in a context that doesn't even exist yet. Is that correct?

The third obvious answer is that if either the decision or the implementation is really too hard, then make it a syntax error for now, and revisit it in the future.

Everything's perfectly well defined, save that you can't externally evaluate the defaults. You can quite happily use the walrus in a late-bound default, and it'll bind to the function's locals. (Though I don't recommend it. I think that that's going to make for less-readable code than alternatives. Still, it's perfectly legal and well-defined.)

...

It's a massive amount of completely unnecessary overhead AND a difficult question of which parts belong in the closure and which parts belong as parameters, which means that this is nearly impossible to define usefully.

I've given you two useful definitions.

Its not clear what overhead you are worried about.

Accessing variables in cells is almost as fast as accessing locals, but even if they were as slow as globals, premature optimization is the root of all evil. Globals are fast enough.

Or are you worried about the memory overhead of the closures? The extra cost of fetching and calling the functions when evaluating the defaults? None of these things seem to be good reasons to dismiss the idea that default expressions should be independent of the function body.

The problem is that you're trying to reference cell variables in a closure that might not even exist yet, so *just in case* you might have nonlocals, you have to construct a closure... or find an existing one. Ill-defined and inefficient.

...

That's not what the example shows. It shows that changing dunder attributes can do this. I'm not sure why you think that the implementation is as restricted as you imply. The assignment to __defaults_extra__ is kinda significant here :)

Ah, well that is not so clear to people who aren't as immersed in the implementation as you :-)

Maybe, but I did think that the fact that I was assigning to a dunder should be obvious to people who are reading here :)

Messing about with function dunders can do weird shit:

...

...

...

def func(a=1, b=2): ... return a+b ... func.__defaults__ = (1, 2, 3, 4, 5) func() 9

I wouldn't worry about it.

Good, so, not an abomination. (In your opinion. Others are, of course, free to abominate as they please.) ChrisA

On Fri, Dec 3, 2021 at 2:24 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Fri, Dec 03, 2021 at 01:08:50PM +1100, Chris Angelico wrote:

...

How, with external calling, are you going to know which name references to look up, and where to get their values from?

Isn't the source code available as a string? I will see the names in the expression.

Or I will disassemble the code object and inspect its byte-code and/or AST. Or read the documentation. Or maybe the code object will expose those names in a dunder and I can just look at that.

...

This is already a virtually impossible task in Python, which is why f-strings cannot be implemented as str.format(**something) for any known meaning of "something".

f-strings cannot be implemented as str.format because str.format doesn't evaluate arbitrary expressions.

...

...

...

f'{1000+len(repr(None))}' '1004'

...

You cannot get your *current* variable set, much less the available variables in some other context.

Why not? The default expression object can record nonlocal variables in a closure, and you can provide your own globals and/or locals.

You're saying "can't, impossible" but I could take your function object, extract the code object, disassemble the code to AST or byte-code, extract the portions that represent the default expression, and evaluate that. So not *impossible*, just bloody inconvenient and annoying.

(Well, when I say *I could*, I mean *somebody with mad leet AST or byte- code hacking skillz. Not actually me.)

When f-strings were first proposed, one of the counter-arguments was basically "just use globals()/locals()". https://www.python.org/dev/peps/pep-0498/#no-use-of-globals-or-locals But you cannot, in a fully general way, solve the problem that way. It is *impossible*. Not just difficult. Impossible. You cannot, in Python as it currently is, look up arbitrary names in any reliable way. This is especially true now that we have assignment expressions, such that evaluating the function default could create a new local or nonlocal name. Not just inconvenient and annoying. So impossible that a str method could not be written to do it. ChrisA

On Fri, Dec 3, 2021 at 6:24 AM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

What I'm saying is that I want that "thing" to exist. At the time the function is defined, I want there to be a Python object which represents the behavior to be activated at call time if the argument is not passed. In the current proposal there is no such "thing". The function just has behavior melded with its body that does stuff, but there is no addressable "thing" where you can say "if you call the function and the argument isn't passed were are going to take this default-object-whatchamacallit and 'use' it (in some defined way) to get the default value". This is what we already have for early-bound defaults in the function's `__defaults__` attribute.

What would you do with this object? Suppose you have this function: def f(lst=>[], n=>len(lst)): ... What can you usefully do with this hypothetical object representing the default for n? There is no "thing" and no way you could "use" that thing to predict the value of n without first knowing the full context of the function. With early-bound defaults, you can meaningfully inspect them and see what the value is going to be. With late-bound defaults, should the default for lst be an empty list? Should the default for n be 0? Neither is truly correct (but they're wrong in different ways). Do you actually need that object for anything, or is it simply for the sake of arbitrary consistency ("we have objects for early-bound defaults, why can't we have objects for late-bound defaults")? The value that will be assigned to the parameter does not exist until the function is actually called, and may depend on all kinds of things about the function and its context. Until then, there is no value, no object, that can represent it. I put to you the same question I put to David Mertz: how is a late-bound default different from half of a conditional expression? def f(*args): lst = args[0] if len(args) > 0 else [] n = args[1] if len(args) > 1 else len(lst) ... Do the expressions "[]" and "len(lst)" have to have object representations in this form? If not, why should they need object representations when written in the more natural way as "lst=>[]"? ChrisA

On Fri, Dec 3, 2021 at 8:07 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

On Thu, Dec 2, 2021 at 2:40 PM Chris Angelico <rosuav@gmail.com> wrote:

...

How is a late-bound default different from half of a conditional expression?

...

def f(lst=>[], n=>len(lst)):

def f(*args): lst = args[0] if len(args) > 0 else [] n = args[1] if len(args) > 1 else len(lst)

Although such is obviously not your intention, I think you have provided a stronger argument against this feature/PEP than any other I've seen raised so far. Well, maybe just against the specific implementation you have in mind.

Fascinating.

You are correct, of course, that the second form does not provide inspectability for `lst` and `n`. Well, it does, but only in a fairly contorted way of disassembling f.__code__.co_code. Or maybe with an equally indirect look at the parse tree or something. The body of a function is very specifically A BODY.

What your proposal/implementation does is put things into the function signature that are simultaneously excluded from direct inspectability as function attributes. Python, like almost all programming languages, makes a pretty clear distinction between function signatures and function bodies. You propose to remove that useful distinction, or at least weaken it.

Actually, no. I want to put the default arguments into the signature, and the body in the body. The distinction currently has a technical restriction that means that, in certain circumstances, what belongs in the signature has to be hacked into the body. I'm trying to make it so that those can be put where they belong.

For the reasons Eric Smith and others have pointed out, I really WANT to keep inspectability of function signatures.

Here's what you get:

...

...

def f(lst=>[], n=>len(lst)): ... ... f.__defaults_extra__ ('[]', 'len(lst)')

String representation, but exactly what the default is. Any tool that inspects a signature (notably, help() etc) will be able to access this. Now write the function the other way, and show me how easy it is to determine the behaviour when arguments are omitted. What can you access? ChrisA

On Fri, Dec 3, 2021 at 2:30 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

On 2021-12-02 15:40, Chris Angelico wrote:

...

Actually, no. I want to put the default arguments into the signature, and the body in the body. The distinction currently has a technical restriction that means that, in certain circumstances, what belongs in the signature has to be hacked into the body. I'm trying to make it so that those can be put where they belong.

Chris, I know this is probably not your intention, but I feel the discussion is continually muddle by you just saying "default arguments" as if everyone agrees on what those are and the issue is just where to put them. But clearly that is not the case. You seem to take it for granted that "len(x) evaluated when the function is called" "is" a "default argument" in the same sense that an early-bound default like the number 3 is. I do not agree, and it's pretty clear David Mertz does not agree, and I think there are others here who also do not agree.

It's not as is there is some pre-existing notion of "default argument" in Python and you are just proposing to add an implementation of it that was left out due to some kind of oversight. Your proposal is CHANGING the idea of what a default argument is. I get that it's natural to refer to your new proposal as "default arguments" but I've now seen a number of messages here where you say "but no we should do X because this is a default argument", taking for granted that what you're talking about is already agreed to be a default argument. (No doubt I and many others are also guilty of similar missteps by saying "default argument" as a shorthand for something or other, and I'll try to be careful about it myself.)

Some functions most assuredly DO have a pre-existing notion of "default argument". Some do not. Allow me to give a few examples: def getattr(object, name, default): Omitting the third argument does not have a default. It will cause different behaviour (getattr will raise an exception). def dict.get(key, default=None): Omitting the last argument behaves exactly as if you passed None. This has a default; the function behaves identically whether you pass an argument or not, and you can determine what the behaviour would be if you do. (These two are a little confusing in that they use the name "default", but the fact is that default arguments are often used for defaults, surprise surprise. So I'm going to be reusing that word a lot.) def open(file, mode='r', encoding=????): Omitting mode is exactly the same as passing 'r'. Omitting encoding is exactly the same as passing locale.getpreferredencoding(False). Both of these have real defaults. def bisect.bisect(a, x, lo=0, hi=len(a)): Omitting lo is exactly the same as passing 0. Omitting hi is exactly the same as passing len(a). Both of these have real defaults. For parameters that do not have defaults, PEP 671 has nothing to say. Continue doing what you're already doing (whether that's a sentinel, or *args, or whatever), as there's nothing needing to be changed. For parameters whose defaults are simple constants, PEP 671 also has nothing to say. Continue defining them as early-bound defaults, and the behaviour will not change. The difference comes with those arguments whose true default is calculated in some way, or is dynamic. These really truly do have default values, and there is absolutely no behavioural difference between hi=len(a) and omitting the hi parameter to bisect(). Due to a technical limitation, though, the documentation for bisect() has to explain this in the docstring rather than the function signature. Would this code pass review? def spaminate(msg, times=None): if times is None: times = 50 ... Unless there's a very VERY good reason for using None as the default, shouldn't it use 50 in the signature? It would be a clearer declaration of intent: omitting this argument is the same as passing 50.

By my definition as of now, a default argument has to be an object. Thus what your proposal envisions are not default arguments. You can't just say "I want to do this with default arguments". You need to provide an argument for why we should even consider these to be default arguments at all.

Yes, that is a current, and technical, limitation. Consider this: https://en.wikipedia.org/wiki/Default_argument#Evaluation Python is *notable* for always evaluating default arguments just once. The concept of default args, as described there and as understood by anyone that I have asked (mainly students, which I admit isn't an unbiased sample, but nothing is), is that they should be evaluated at call time. My argument for why they should be considered default arguments is that, aside from Python being unable to represent them, they *are* precisely what default arguments are. Suppose you're on the ECMAScript committee, and someone proposes that the language support bignums. (That's not truly hypothetical, incidentally - I saw references to such a proposal.) Would you argue that 9007199254740993 is not really a number, on the basis that current ECMAScript cannot represent it? Would you force someone to prove that it is really a number? It's only special because of implementation restrictions.

Perhaps a way of stating this without reference to arguments is this: you want to put code in the function signature but not have it executed until the function is called. I do not agree with that choice. The function signature and the function body are different syntactic environments with different semantics. Everything in the function signature should be executed when the function is defined (although of course that execution can result in an object which contains deferred behavior itself, like if we pass a function as an argument to another).

That's reasonable, but I disagree from a perspective of practicality: logically and usefully, it is extremely helpful to be able to describe arguments that way. Passing positional parameters is approximately equivalent to: def func(*args): """func(a, b, c=1, d=a+b)""" a, args = args b, args = args if args: c, args = args else: c = 1 if args: d, args = args else: d = a + b If you try to explain it to someone who's learning about default argument values, do you first have to explain that they get evaluated and recorded somewhere, and these indescribable values are what's actually assigned? Or would you describe it mostly like this (or as "if c is omitted: c = 1", which comes to the same thing)? Argument default snapshotting is an incredibly helpful performance advantage, but it isn't inherent to the very concept of default arguments.

Only the function body should be executed when the function is called. If we want to provide some way to augment the function body to assign values to missing arguments, I wouldn't rule that out completely, but in my view the function signature is not an available space for that. The function signature is ONLY for things that happen when the function is defined. It is too confusing to have the function signature mix definition-time and call-time behavior.

The trouble is that the function signature MUST be the one and only place where you figure out whether the argument is optional or mandatory. Otherwise there is an endless sea of debugging nightmares - just ask anyone who works regularly in JavaScript, where all arguments are optional and will default to the special value 'undefined' if omitted. So you'd need to separate "this is optional" from "if omitted, do this". That might be a possibility, but it would need two separate compiler features: def func(a, b, c=1, d=?): if unset d: d = a+b where "unset N" is a unary operator which returns True if the name would raise, False if not. I'm not proposing that, but if someone wants to, I would be happy to share my reference implementation, since 90% of the code is actually there :) ChrisA

On Sat, Dec 4, 2021 at 2:34 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Fri, Dec 03, 2021 at 10:40:42AM +1100, Chris Angelico wrote:

...

Here's what you get:

...

...

...

def f(lst=>[], n=>len(lst)): ... ... f.__defaults_extra__ ('[]', 'len(lst)')

String representation, but exactly what the default is.

Excellent. And you've just proven that we can evaluate the defaults.

>>> a = '[]' # f.__defaults_extra__[0] >>> b = 'len(lst)' # f.__defaults_extra__[1] >>> lst = eval(a, globals()) >>> n = eval(b, globals(), dict(lst=lst)) >>> print(lst, n) [] 0

Worst case scenario, we can eval the string representation, and that should work for the great majority of cases that don't involve nonlocals.

Yes, those awkward cases that involve nonlocals. A pity about those. But hey, if that's not a problem to you, then sure, go ahead, just eval it. The string is there for you to use.

But if the defaults are proper code objects, complete with a closure to capture their nonlocal environment, then we should be able to do even better and capture nonlocals as well.

Could there be odd corner cases that don't quite work? Say, like a comprehension inside a class body?

https://github.com/satwikkansal/wtfpython#-name-resolution-ignoring-class-sc...

Oh well. Let's not make the perfect the enemy of the good.

Lots and lots and lots of potential problems. Consider: def f(): a = 1 def f(b, x=>a+b): def g(): return x, a, b Both a and b are closure variables - one because it comes from an outer scope, one because it's used in an inner scope. So to evaluate a+b, you have to look up an existing closure cell, AND construct a new closure cell. The only way to do that is for the compiled code of a+b to exist entirely within the context of f's code object. Which means that there isn't really anything to usefully pull out and evaluate, since it can't be executed until you've set up a stack frame to call f, and once you've done that, you're basically just... calling f anyway. If you still dispute that it's impossible, you're absolutely welcome to write your own reference implementation. I've argued this point with as much detail as I can, and short of pointing stuff out in the code, I can't do anything more. If you want to keep on saying "but of COURSE it's possible", then go ahead, prove it to me. ChrisA

On Sat, Dec 4, 2021 at 8:48 PM Steven D'Aprano <steve@pearwood.info> wrote:

On Sat, Dec 04, 2021 at 03:14:46PM +1100, Chris Angelico wrote:

...

Lots and lots and lots of potential problems. Consider:

def f(): a = 1 def f(b, x=>a+b): def g(): return x, a, b

Both a and b are closure variables - one because it comes from an outer scope, one because it's used in an inner scope. So to evaluate a+b, you have to look up an existing closure cell, AND construct a new closure cell.

The only way to do that is for the compiled code of a+b to exist entirely within the context of f's code object.

I dispute that is the only way. Let's do a thought experiment.

First, we add a new flag to the co_flags field on code objects. Call it the "LB" flag, for late-binding.

Second, we make this:

def f(b, x=>a+b): ...

syntactic sugar for this:

def f(b, x=lambda b: a+b): ...

except that the lambda has the LB flag set.

Okay. So the references to 'a' and 'b' here are one more level of function inside the actual function we're defining, which means you're paying the price of nonlocals just to be able to late-evaluate defaults. Not a deal-breaker, but that is a notable cost (every reference to them inside the function will be slower).

And third, when the interpreter fetches a default from func.__defaults__, if it is a LB function, it automatically calls that function with the parameters to the left of x (which in this case would be just b).

Plausible. Okay. What this does mean, though, is that there are "magic objects" that cannot be used like other objects. Consider: def make_printer(dflt): def func(x=dflt): print("x is", x) return func Will make_printer behave the same way for all objects? Clearly the expectation is that it will display the repr of whichever object is passed to func, or if none is, whichever object is passed to make_printer. But if you pass it a function with the magic LB flag set, it will *execute* that function. I don't like the idea that some objects will be invisibly different like that.

Here's your function, with a couple of returns to make it actually do something:

def f(): a = 1 def f(b, x=>a+b): def g(): return x, a, b return g return f

We can test that right now (well, almost all of it) with this:

def func(): # change of name to distinguish inner and outer f a = 1 def f(b, x=lambda b: a+b): def g(): return x, a, b return g return f

and just pretend that x is automatically evaluated by the interpreter. But as a proof of concept, it's enough that we can demonstrate that *we* can manually evaluate it, by calling the lambda.

Okay, sure. It's a bit hard to demo it (since it has to ONLY do that magic if the arg was omitted), but sure, we can pretend.

We can call func() to get the inner function f, and call f to get g:

>>> f = func() >>> print(f) <function func.<locals>.f at 0x7fc945c41f30>

>>> g = f(100) >>> print(g) <function func.<locals>.f.<locals>.g at 0x7fc945e1f520>

Calling g works:

>>> print(g()) (<function func.<locals>.<lambda> at 0x7fc945c40f70>, 1, 100)

with the understanding that the real implementation will have automatically called that lambda, so we would have got 101 instead of the lambda. That step requires interpreter support, so for now we just have to pretend that we get

(101, 1, 100)

instead of the lambda. But we can demonstrate that calling the lambda works, by manually calling it:

>>> x = g()[0] >>> print(x) <function func.<locals>.<lambda> at 0x7fc945c40f70> >>> print(x(100)) # the interpreter knows that b=100 101

Now let's see if we can extract the default and play around with it:

>>> default_expression = f.__defaults__[0] >>> print(default_expression) <function func.<locals>.<lambda> at 0x7fc945c40f70>

The default expression is just a function (with the new LB flag set). So we can inspect its name, its arguments, its cell variables, etc:

>>> default_expression.__closure__ (<cell at 0x7fc945de74f0: int object at 0x7fc94614c0f0>,)

We can do anything that we could do with any other other function object.

Yup. As long as it doesn't include any assignment expressions, or anything else that would behave differently.

Can we evaluate it? Of course we can. And we can test it with any value we like, we're not limited to the value of b that we originally passed to func().

>>> default_expression(3000) 3001

Of course, if we are in a state of *maximal ignorance* we might have no clue what information is needed to evaluate that default expression:

>>> default_expression() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: func.<locals>.<lambda>() missing 1 required positional argument: 'b'

Oh look, we get a useful diagnostic message for free!

What are we missing? The source code of the original expression, as text. That's pretty easy too: the compiler knows the source, it can cram it into the default expression object:

>>> default_expression.__expression__ = 'a+b'

Introspection tools like help() can learn to look for that.

What else are we missing? A cool repr.

>>> print(default_expression) # Simulated. <late bound default expression a+b>

We can probably come up with a better repr, and a better name than "late bound default expression". We already have other co_flags that change the repr:

32 GENERATOR 128 COROUTINE 256 ITERABLE_COROUTINE

so we need a name that is at least as cool as "generator" or "coroutine".

Those parts are trivial, no problem.

Summary of changes:

* add a new co_flag with a cool name better than "LB";

* add an `__expression__` dunder to hold the default expression; (possibly missing for regular functions -- we don't necessarily need *every* function to have this dunder)

* change the repr of LB functions to display the expression;

* teach the interpreter to compile late-bound defaults into one of these LB functions, including the source expression;

* teach the interpreter that when retrieving default values from the function's `__defaults__`, if they are a LB function, it must call the function and use its return result as the actual default value;

* update help() and other introspection tools to handle these LB functions; but if any tools don't get updated, you still get a useful result with an informative repr.

Great. So now we have some magnificently magical behaviour in the language, which will have some nice sharp edge cases, but which nobody will ever notice. Totally. I'm sure. Plus, we pay a performance price in any function that makes use of argument references, not just for the late-bound default, but in the rest of the code. We also need to have these special functions that get stored as separate code objects. All to buy what, exactly? The ability to manually synthesize an equivalent parameter value, as long as there's no assignment expressions, no mutation, no other interactions, etc, etc, etc? That's an awful lot of magic for not a lot of benefit. I *really* don't like the idea that some types of object will be executed instead of being used, just because they have a flag set. That strikes me as the sort of thing that should be incredibly scary, but since I can't think of any specific reasons, I just have to call it "extremely off-putting". But hey. Go ahead and build a reference implementation. I'll compile it and give it a whirl. ChrisA

On Sun, Dec 5, 2021 at 6:16 AM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

On 2021-12-04 03:50, Chris Angelico wrote:

...

On Sat, Dec 4, 2021 at 8:48 PM Steven D'Aprano <steve@pearwood.info> wrote:

...

And third, when the interpreter fetches a default from func.__defaults__, if it is a LB function, it automatically calls that function with the parameters to the left of x (which in this case would be just b).

Plausible. Okay.

What this does mean, though, is that there are "magic objects" that cannot be used like other objects. Consider:

Your proposal also has the same problem, since it involves "magic functions" that do not have usable values for their argument defaults, instead having some kind of Ellipsis two-step. It's all a matter of what you consider magic.

My proposal allows any object to be used as a function default argument. There's a minor technical difference that means that there's a second lookup if you use Ellipsis, but you can still use Ellipsis just fine.

...

...

def f(x=...): ... print(type(x), x) ... f() <class 'ellipsis'> Ellipsis f(None) <class 'NoneType'> None f("spam") <class 'str'> spam

There are no objects that will behave differently if used in this way. EVERY object can be a function default argument. Steve's proposal has some objects (functions with the LB flag set) actually behave differently - they *will not behave correctly* if used in this way. This is a restriction placed on the rest of the language.

...

Great. So now we have some magnificently magical behaviour in the language, which will have some nice sharp edge cases, but which nobody will ever notice. Totally. I'm sure. Plus, we pay a performance price in any function that makes use of argument references, not just for the late-bound default, but in the rest of the code. We also need to have these special functions that get stored as separate code objects.

All to buy what, exactly? The ability to manually synthesize an equivalent parameter value, as long as there's no assignment expressions, no mutation, no other interactions, etc, etc, etc? That's an awful lot of magic for not a lot of benefit.

I would consider most of what you say here an accurate description of your own proposal. :-)

That's highly unfair. No, I won't let that pass. Please retract or justify that statement. You are quoting the conclusion of a lengthy post in which I show significant magic in Steve's proposal, contrasting it with mine which has much clearer behaviour, and you then say that my proposal has the same magic. Frankly, that is not a reasonable assertion, and I take offense.

Now we have magnificently magical behavior in the language, which will take expressions in the function signature and behind the scenes "inline" them into the function body. We also need to have these special function arguments that do NOT get stored as separate objects, unlike ordinary function arguments. All to buy what, exactly? The ability to write something in the function signature that we can already write in the body, and that quite naturally belongs in the body, because it is executed when the function is called, not when it is defined.

You assert that it "belongs in the body", but only because Python currently doesn't allow it to be anywhere else. Other languages have this exact information in the function signature. This is a much larger distinction than what Steve shows, which is the exact same feature but with these magic callables.

...

I *really* don't like the idea that some types of object will be executed instead of being used, just because they have a flag set. That strikes me as the sort of thing that should be incredibly scary, but since I can't think of any specific reasons, I just have to call it "extremely off-putting".

I *really* don't like the idea that some types of argument will be inlined into the function body instead of being stored as first-class values like other `__defaults__`, just because there happens to be this one extra character next to the equals sign in the function signature. That strikes me as the sort of thing that should be incredibly scary.

You're still being highly offensive here. There's a HUGE difference between these two assertions. Steve's proposal makes some objects *behave differently when used in existing features*. It would be like creating a new type of string which, when printed out, would eval itself. That proposal wouldn't fly, and it's why f-strings are most assuredly NOT first-class objects. Why is it such a big deal for these function default expressions to not be first-class objects? None of these are first-class either: print(f"An f-string's {x+y} subexpressions") print(x/y if y else "An if/else expression's sides") assign(x.y[42], "An assignment target") We don't have a problem with these being unable to be externally referenced, manipulated, etc, as first-class objects. Why is it a problem to be unable to refer to "new empty list" as some sort of object when used like this? def f(x=>[]): ... Can you explain why it is necessary? And then, after that, explain why you claim that Steve's proposal, which makes some objects *not even work in early-bound defaults*, is just as magical? ChrisA

On Mon, Dec 6, 2021 at 10:02 PM Steven D'Aprano <steve@pearwood.info> wrote:

...

You assert that it "belongs in the body", but only because Python currently doesn't allow it to be anywhere else. Other languages have this exact information in the function signature.

This argument about where the evaluation of the default expression is surreal.

Brendan argues in favour of the status quo, because he thinks that the evaluation of the default expression "belongs in the body".

You (Chris) argue in favour of your PEP, where the bytecode of the default expression is inlined into the function's body, because you insist that it belongs in the body. You justify that claim by making spurious arguments that it is "impossible" to do otherwise (your term, not mine). But it isn't impossible, and I shall justify that claim in another post. A bold claim for somebody who knows nothing about the C implementation :-)

I am arguing that it belongs in the *header*. I think I once mistyped that and said "body" by mistake, but I have never intentionally advocated for that (and you called me out on it promptly for a correction). Default expressions, whether evaluated at definition time or call time, belong in the header. With definition-time expressions, their code is inlined into the body of the surrounding context. With call-time expressions, their code is inlined into the body of the function. Neither of these is part of the signature, but that's because the signature doesn't itself have any code.

The bottom line is, you both are arguing at each other because you both want the default expression to be executed in the body of the function o_O

You're conflating implementation with syntax. I want it to be implemented in the body, because that's clean and efficient *for implementation*. I want it to be written in the signature, because that's where function defaults belong. You're asking for it to be written in the signature, same as I am, but then to have it executed... somewhere else. But still in basically the same place.

...

You're still being highly offensive here.

Everyone please chill a bit.

Taking offence at vigourous but still respectful intellectual disagreement is not kind or welcoming and most of all it is not *helpful*. Brendan has not accused you of molesting children or betraying a position of trust. He hasn't even suggested you have bad breath. He has merely disagreed with your interpretation of a proposed programming language feature. Is it really worth taking offence over that?

He apologized privately and we have settled that matter. Let's drop it.

Chris, I know this has been a long, hard PEP for you. Here is a feature that people in the Python community say they want, you write a PEP proposing it, and you get nothing but negativity. Ouch :-(

Well, not nothing but negativity. There has been some support. And I didn't spend six hours researching and writing up evaluation strategies of nearly two dozen languages because I'm trying to sabotage your proposal. You and I may disagree with some details of this PEP, but I'm supporting it and want it to be the best late-bound feature it can be.

Thank you, I do appreciate that. Still, it does get somewhat tiresome when there is a dogpile of negativity about the proposal, much of it unbacked by any sort of code, and just declaring that what I've said doesn't work or isn't ideal or should be left off because of a hypothetical future feature. Very tiresome. ChrisA

Brendan Barnwell writes:

The ability to write something in the function signature that we can already write in the body, and that quite naturally belongs in the body, because it is executed when the function is called, not when it is defined.

I'm basically in sympathy with your conclusion, but I don't think it's useful to prejudice the argument by saying it *naturally belongs* in the body. Some languages quite naturally support thunks/blocks (Ruby) or even go so far as to equate code to data (Lisp), and execute that code in lieu of what appear to be variable references. But maybe it's *Pythonic* to necessarily place the source code in the body? I can't say that.

...

I *really* don't like the idea that some types of object will be executed instead of being used, just because they have a flag set.

From a syntactic point of view, that's how Ruby blocks work. Closer to home, that's how properties work. And in the end, *all* objects are accessed by executing code. This is a distinction without a difference, except in our heads. I wouldn't want to be asked to explain the dividing line between objects that were "just used" and objects that were "produced by code that was executed instead of being just used".

I *really* don't like the idea that some types of argument will be inlined into the function body instead of being stored as first-class values like other `__defaults__`, just because there happens to be this one extra character next to the equals sign in the function signature. That strikes me as the sort of thing that should be incredibly scary.

Properties have *no* visible syntax if they're imported from a module. Properties are extremely useful, and we all use them all the time without noticing or caring. I see no reason in principle why the same kind of feature wouldn't be useful and just as invisible and just as "natural" for local or global variables -- or callable parameters, as long as properly restricted. Chris's proposal is nothing if not restricted! :-) My issues with Chris's proposal are described elsewhere, but I don't really see a problem in principle.

On Sun, Dec 5, 2021 at 11:34 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Sat, Dec 04, 2021 at 10:50:14PM +1100, Chris Angelico wrote:

...

...

syntactic sugar for this:

def f(b, x=lambda b: a+b): ...

except that the lambda has the LB flag set.

Okay. So the references to 'a' and 'b' here are one more level of function inside the actual function we're defining, which means you're paying the price of nonlocals just to be able to late-evaluate defaults. Not a deal-breaker, but that is a notable cost (every reference to them inside the function will be slower).

How much slower? By my tests:

- access to globals is 25% more expensive than access to locals; - access to globals is 19% more expensive than nonlocals; - and nonlocals are 6% more expensive than locals.

Or if you do the calculation the other way (the percentages don't match because the denominators are different):

- locals are 20% faster than globals; - and 5% faster than nonlocals; - nonlocals are 16% faster than globals.

Premature optimization is the root of all evil.

We would be better off spending effort making nonlocals faster for everyone than throwing out desirable features and a cleaner design just to save 5% on a microbenchmark.

Fair, but the desirable feature can be achieved without this cost, and IMO your design isn't cleaner than the one I'm already using, and 5% is a lot for no benefit.

[...]

...

What this does mean, though, is that there are "magic objects" that cannot be used like other objects.

NotImplemented says hello :-)

Good point. Still, I don't think we want more magic like that.

And if you still think that we should care, we can come up with a more complex trigger condition:

- the parameter was flagged as using a late-default; - AND the default is a LB function.

Problem solved. Now you can use LB functions as early-bound defaults, and all it costs is to record and check a flag for each parameter. Is it worth it? Dunno.

Uhh.... so..... the parameter has to be flagged AND the value has to be flagged? My current proposal just flags the parameter. So I ask again: what are you gaining by this change? You've come right back to where you started, and added extra costs and requirements, all for.... what?

[...]

...

...

The default expression is just a function (with the new LB flag set). So we can inspect its name, its arguments, its cell variables, etc:

>>> default_expression.__closure__ (<cell at 0x7fc945de74f0: int object at 0x7fc94614c0f0>,)

We can do anything that we could do with any other other function object.

Yup. As long as it doesn't include any assignment expressions, or anything else that would behave differently.

I don't get what you mean here. Functions with the walrus operator are still just functions that we can introspect:

...

...

...

f = lambda a, b: (len(w:=str(a))+b)*w f('spam', 2) 'spamspamspamspamspamspam' f.__code__ <code object <lambda> at 0x7fc945e07c00, file "<stdin>", line 1>

What sort of "behave differently" do you think would prevent us from introspecting the function object? "Differently" from what?

Wrapping it in a function means the walrus would assign in that function's context, not the outer function. I think it'd be surprising if this works: def f(x=>(a:=1)+a): # default is 2 but this doesn't: def g(x=>(a:=1), y=>a): # default is UnboundLocalError It's not a showstopper, but it is most definitely surprising. The obvious solution is to say that, in this context, a is a nonlocal. But this raises a new problem: The function object, when created, MUST know its context. A code object says "this is a nonlocal", and a function object says "when I'm called, this is my context". Which means you can't have a function object that gets called externally, because it's the code, not the function, that is what you need here. And that means it's not directly executable, but it needs a context. So, once again, we come right back around to what I have already: code that you can't lift out and call externally. The difference is that, by your proposal, there's a lot more overhead, for the benefit of maybe under some very specific circumstances being able to synthesize the result.

...

We also need to have these special functions that get stored as separate code objects.

That's not a cost, that's a feature. Seriously. We're doing that so that we can introspect them individually, not just as the source string, but as actual callable objects that can be:

- introspected;

- tested;

- monkey-patched and modified in place (to the degree that any function can be modified, which is not a lot);

- copied or replaced with a new function.

Testing is probably the big one. Test frameworks will soon develop a way to let you write tests to confirm that your late bound defaults do what you expect them to do.

That's trivial for `arg=[]` expressions, but for complex expressions in complex functions, being able to isolate them for testing is a big plus.

I'm still not convinced that it's as useful as you say. Compare these append-and-return functions: def build1(value, lst=None): if lst is None: lst = [] lst.append(value) return lst _SENTINEL = object() def build2(value, lst=_SENTINEL): if lst is _SENTINEL: lst = [] lst.append(value) return lst def hidden_sentinel(): _SENTINEL = object() def build3(value, lst=_SENTINEL): if lst is _SENTINEL: lst = [] lst.append(value) return lst return build3 build3 = hidden_sentinel() def build4(value, *optional_lst): if len(optional_lst) > 1: raise TypeError("too many args") if not optional_lst: optional_lst = [[]] optional_lst[0].append(value) return optional_lst[0] def build5(value, lst=>[]): lst.append(value) return lst (Have I missed any other ways to do this?) In which of them can you introspect the []? Three of them have an externally-visible sentinel, but you can't usefully change it in any way. You can look at it, and you'll see None or "<object object at 0xYourCapitalCity>", but that doesn't tell you that you'll get a new empty list. In which of them do you have a thing you can call that will generate an equivalent empty list? In which can you monkey-patch or modify-in-place the []? Not a big deal though, you admit that there wouldn't be much monkey-patching in any form. But you get none whatsoever with other idioms. In which of these can you separately test the []? Do any current test frameworks have a way to let you write tests to make sure that these do what you expect? In which of these can you copy the [] into another context, or replace the [] with something else? Why are these such major problems for build5 if they're not for the other four? ChrisA

On Mon, Dec 6, 2021 at 2:56 AM Steven D'Aprano <steve@pearwood.info> wrote:

...

...

What sort of "behave differently" do you think would prevent us from introspecting the function object? "Differently" from what?

Wrapping it in a function means the walrus would assign in that function's context, not the outer function.

Unless the variable was a cell variable of the outer function. Which I think that they need to be. Didn't you already decide that walrus assignment has to be in the context of the function?

That's not a rhetorical question, I genuinely don't remember.

Yes, it does, and that won't be the case if you create an invisible nested function.

...

But this raises a new problem: The function object, when created, MUST know its context. A code object says "this is a nonlocal", and a function object says "when I'm called, this is my context". Which means you can't have a function object that gets called externally, because it's the code, not the function, that is what you need here. And that means it's not directly executable, but it needs a context.

Sorry Chris, I don't understand what you are trying to say here. If I take what you are saying literally, I would take you as trying to say that closures can never be executed. But they clearly can be, and I know that you know that. So obviously I am misunderstanding you.

Closures cannot be executed without a context. Consider: def f(x=lambda: (a:=[])): if isinstance(x, FunctionType): x = x() print(a) Here's the problem: The name 'a' should be in the context of f, but that context *does not exist* until f starts executing. As an early-bound default, like this, it's not going to work, because the function object has to be fully constructed at function definition time. If you try to do this sort of thing with a magical function that is synthesized as part of late-bound default handling, you'll still need to figure out when f's context gets formed. Normally, it won't get formed until the stack frame for f gets constructed - which is when arguments get assigned to parameters, etc, etc. Trying to call that inner function without first giving it a context won't work. And if all you have is a code object without a function, what's the use? How is it better than just having code in the function itself?

I don't understand why you think that we can't take one of those Late-Bound (LB) functions, which will be a closure, and call it like we can call any other closure.

By the time we have access to the LB functions, the owner function will exist, so there shouldn't be any problem with the context not existing.

The function will exist. The instance of it won't. Consider this basic demo of closures: def counter(): n = 1 def incr(): nonlocal n n += 1 return n return incr Prior to calling counter(), you can disassemble incr's bytecode and see what it does. But you can't call incr. Its context does not yet exist. Until you call counter, there is actually no function for incr, only the code object. And while it is possible to eval() a code object...

...

...

eval(counter.__code__.co_consts[2]) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: code object passed to eval() may not contain free variables

... you can't if it has nonlocals, because it needs a context. So if there's any possibility of nonlocals in latebound argument defaults, you won't be able to call this externally, which leaves me wondering: why all this hassle if it isn't even going to work?

...

So, once again, we come right back around to what I have already: code that you can't lift out and call externally. The difference is that, by your proposal, there's a lot more overhead, for the benefit of maybe under some very specific circumstances being able to synthesize the result.

Surely it is the other way around? If you are correct, there are some extremely specific cicumstances involving the use of walrus operator in two different default expressions such that you cannot call one of the two functions, but the 99.99% of straight-forward non-walrus, non-tricky default expressions will work perfectly fine as independently callable functions.

To be honest, 99.99% of late-bound defaults would probably work just fine if you simply eval their texts. But that doesn't justify having functions either.

...

I'm still not convinced that it's as useful as you say. Compare these append-and-return functions:

def build1(value, lst=None): if lst is None: lst = [] [...] In which of them can you introspect the []?

"Here are lots of ways to solve a problem that don't let you test or introspect part of your code. Therefore we shouldn't add a feature that will let us more easily test and introspect that part of the code."

Yes Chris, I know that emulated late-bound defaults are currently effectively invisible at runtime (short of byte-code hacking). That weakness of the status quo is why you have written this PEP.

That invisibility is not a feature we should duplicate in late-bound defaults, but an argument for introducing defaults that overcome that weakness.

Okay, but you're asking for a LOT of complexity for a very small part of the feature.

Right now, your PEP adds a new feature that gives us *zero* new functionality. There is nothing your PEP allows us to do that we cannot already do. It adds no new functionality. It's just syntactic sugar for "if arg is missing: return expression". You compile that code (or something very close to it) into the body of the function.

MOST new features give us zero new functionality. That's what it means for a language to be Turing-complete. But "if arg is missing:" doesn't currently exist, nor does "this argument is optional and has no default", so you can't quite write this currently, and that's why we need sentinels.

The only wins are (1) when people stumble into the "mutable default" gotcha, it is easy to tell them how to fix it (but not much easier than the status quo) and (2) a small increase in clarity, since we will be able to write the default expression directly in the signature instead of hiding behind a sentinel.

I consider those to be LARGE wins.

That's not nothing, but we've seen on this list a fair number of people who think that the increase in clarity is not enough to justify the feature. Do you think the Steering Council will consider those two small wins sufficient?

I'm hoping so. That's the main thrust of what I'm doing. Clarity, and an easier way to make mutable defaults working. And more intuitive behaviour in some situations. Clarity, mutable defaults, and intuitive behaviour. And nice red uniforms.

But with a different design, we have not just the increase in clarity, but also open the door to a ton of new possibilities that come about thanks to the separation of those default expressions into their own objects. I don't know whether that will be enough to sway the Steering Council. They might hate my idea even more. But feedback on this list suggests that your design's lack of introspectability is putting people off the idea.

Or it's suggesting that people on this list love to argue. I don't think we're surprised by that. :) (And I don't have a problem with it.)

It may be that this feature will never be used for anything more complex than `param=>[]`. In which case, okay, I concede, moving the default expression into its own code unit (a function? something else?) is overkill.

But I don't think that people will only use this for such trivial expressions. Maybe 60% of the cases will be just a literal list display or dict display, and maybe another 30% will be a simple call to an existing function, like `time.ctime()`.

But its the last 10% of non-trivial, interesting expressions where people will want to do more than just inspect the string. And having the expressions as first-class code units will open up those new opportunities that we currently lack.

If you want to write up a competing reference implementation, go ahead. I don't think it will be as easy as you claim. And I'm not going to mention the possibility in the PEP *without* a reference implementation, nor do I intend to write one myself (since I dispute its usefulness). Will the benefits outweigh the costs? I don't know. But I think not, and that the much simpler proposal will be better. ChrisA

On Tue, Dec 07, 2021 at 12:53:39AM +1100, Steven D'Aprano wrote:

If we can't execute the expression without the context existing, we make it exist. Details to follow in another post.

Here is some handwavy pseudo-code for setting up the context prior to calling a function "func": code = func.__code__ locals = [NIL pointer for _ in range(code.co_nlocals)] cells = [cell() for _ in range(len(code.co_cellvars))] assign args, kwargs, early defaults to locals # make a frame frame = Frame() frame.f_back = current frame frame.f_lasti = 0 frame.f_locals = locals + cells + list(func.__closure__) frame.f_code = code and then the interpreter does its funky thang with the frame and the function is executed. But here's the thing... there is nothing that says that the f_code has to have come from func.__code__. It just needs to be code that expects the same environment (locals, cell, etc) as the frame is set up for. So here's how we could run the late bound default expression alone: 1. Ensure that the default expression code object has the same environment (locals, cells etc) as the function that owns it; this could be a copy, or it could be an reference back to the owner. 2. Set up the frame, as above, for the function that owns the expression. 3. But instead of setting f_code to the owner's code object, we set it to the default expression's code object. They share the same environment (cells, etc) so that's safe. 4. And then just run the function. And to run the owner function: 1. Set up the frame. 2. Run each of the default expressions as above. 3. Then run the owner function. Obviously this is very handwavy. But I am confident that it demonstrates that the idea of factoring default expressions out into their own code objects is not "impossible". -- Steve

On 2021-12-05 08:14, Chris Angelico wrote:

Closures cannot be executed without a context. Consider:

def f(x=lambda: (a:=[])): if isinstance(x, FunctionType): x = x() print(a)

Here's the problem: The name 'a' should be in the context of f, but that context*does not exist* until f starts executing.

Frankly, I would consider this another disadvantage of late-bound arguments as defined under your proposal. I do not want argument defaults to be able to have the side effect of creating additional local variables in the function. (There is also the question of whether they could assign in this manner to names already used by other arguments, so that one argument's default could potentially override the default of another.) -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

On 2021-12-06 11:21, Chris Angelico wrote:

On Tue, Dec 7, 2021 at 6:16 AM Brendan Barnwell <brenbarn@brenbarn.net> wrote:

...

On 2021-12-05 08:14, Chris Angelico wrote:

...

Closures cannot be executed without a context. Consider:

def f(x=lambda: (a:=[])): if isinstance(x, FunctionType): x = x() print(a)

Here's the problem: The name 'a' should be in the context of f, but that context*does not exist* until f starts executing.

Frankly, I would consider this another disadvantage of late-bound arguments as defined under your proposal. I do not want argument defaults to be able to have the side effect of creating additional local variables in the function. (There is also the question of whether they could assign in this manner to names already used by other arguments, so that one argument's default could potentially override the default of another.)

Unless it's otherwise denied, all valid forms of expression should be valid in an argument default. They are already valid in early-bound defaults:

...

...

...

def f(): ... def g(a=(b:=42)): ... ... return locals() ... f() {'b': 42, 'g': <function f.<locals>.g at 0x7f9a7083ef00>}

That's true. I guess what I really mean is "I still think the walrus was a bad idea". :-) -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

Barry Scott writes:

There are many possible implementation of the late bound idea that could create an object/default expression. But is it reasonable to bother with that added complexity/maintenance burden for a first implementation.

Yes. If you don't do it, you'll have backward compatibility issues or technical debt. I'm not saying that's a compelling argument here, except that one of the main alleged problems is that users don't understand mutable defaults. So adding more and more layers of support for default arguments is making matters worse, I suspect. (Remember, they're going to be reading "arg=None" and "@arg=[]" for a long long time.) This one is Worth Doing Right the first time, I think. And IMO David Mertz is right: doing it right means a more general deferred-evaluation object (not to be confused with Deferreds that need to be queried about their value).

And maybe no one will care enough to ever implement the ability to modify the code of a late bound variables expression as a separate object later.

Hear! Hear! That's exactly how I feel about *this* proposal! With all due respect to Chris and Steve who have done great work advocating, implementing, and clarifying the proposal, IAGNU (I am gonna not use). Too much muscle memory, and more important, existing code whose style I want to be consistent and don't wanna mess with because it works, around "arg=None".

Chris Angelico writes:

On Sun, Dec 5, 2021 at 3:08 PM Stephen J. Turnbull <stephenjturnbull@gmail.com> wrote:

...

This one is Worth Doing Right the first time, I think. And IMO David Mertz is right: doing it right means a more general deferred-evaluation object (not to be confused with Deferreds that need to be queried about their value).

If you think that deferred evaluation objects are the right way to do it, then write up a proposal to compete with PEP 671.

That's not your call, I'm afraid. "Keep the status quo" is always a viable option, regardless of other options. And other things equal, it's the preferred option.

In my opinion, it is a completely independent idea,

You're welcome to your opinion, of course. But if you want to claim that's a reason for implementing your proposal, you need to support it. You also need to explain why the additional potential complexity of a third kind of default argument (evaluated at definition, evaluated during call, evaluated when referenced) isn't a big problem.

which is not a complete replacement for late-bound defaults;

Why not? If we have such objects, we could simply specify that in the case where such an object is specified as a function parameter default, it is evaluated in the same environment as your late-bound defaults. So we can have your schsemantics if that's what we want. On the other hand, it might turn out that 90% of the time, it doesn't matter if it's evaluated as part of the calling process, 9% of the time the natural place to evaluate it is at the point of first use, and 1% of the time it should be evaluated before the function body proper is entered. In that case the late-bound default would be of some utility, but is it worth it? David's "infinitesimal utility" argument seems likely to apply. What else would such a deferred-evaluation object be unable to do that your late-bound default can do?

we could continue to have neither.

That's where I am. More than any of these issues, the lack of a well- defined, properly introspectable object bothers me. In 2021, we should provide that.

On Mon, Dec 6, 2021 at 1:45 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Sat, Dec 04, 2021 at 06:11:08PM +0000, Barry Scott wrote:

...

There are many possible implementation of the late bound idea that could create an object/default expression. But is it reasonable to bother with that added complexity/maintenance burden for a first implementation.

Chris wants to throw the late-bound defaults into the body of the function because that's what we are currently forced to do to emulate late-bound defaults. But we are designing the feature from scratch, and we shouldn't be bound by the limitations of the existing idiom.

Not quite true. I want to have them syntactically as part of the body of the function, and semantically as part of the function call. As a function begins executing, a ton of stuff happens, including allocating arguments to parameters, and providing default values for optional parameters that weren't passed. All I want to change is the way that defaults can be provided. Tell me, what's the first bytecode instruction in the function g here? def f(x): print(x) return lambda: x In current CPython, it's not "LOAD_GLOBAL print". It's "MAKE_CELL x". Conceptually, that's part of the framework of setting up the function - setting up its local variables, providing a place for nonlocals. But it's a bytecode at the start of the function's code object. (I'm not sure when that became a bytecode instruction. It wasn't one in Python 2.7.) CPython bytecode is an implementation detail, and the fact that there's bytecode to do this or that is not part of the language definition.

Encapsulation is good. Putting late-bound expressions in their own function so that they can be tested is a good thing, not a problem to be avoided. Most of us have, from time to time, already moved the late bound default into its own function just to make it easy to test in isolation:

def func(arg=None): if arg is None: arg = _implementation() # and now handle arg

With my strategy, we get that isolation for almost for free. There is no extra effort needed on the programmer's side, no new name or top-level function needed.

And if you want an _implementation function for external testing, you're still welcome to do that. def func(arg=>_implementation()): ... No magic, just perfectly normal coding practices. If something's important enough to test separately, refactor it into a private function so you can call it. Why should this happen automatically for late-bound argument defaults? From what I can tell, the only syntactic constructs which form separate code objects are those which need them for namespacing purposes: class blocks, nested functions, genexps, comprehensions. Default argument values are not separate namespaces, so they shouldn't need dedicated stack frames, dedicated code objects, or anything like that.

Will people take advantage of it? Of course people won't bother if the default is `[]` but beyond a certain level of complexity, people will want to test the default expressions in isolation, to be sure that it does the right thing.

Beyond a certain level of complexity, the fact that argument defaults are in the signature will naturally pressure people to refactor them into their own functions. I don't think we'll see people putting multiline argument default expressions directly into the function header. ChrisA

On Mon, Dec 6, 2021 at 4:13 AM Steven D'Aprano <steve@pearwood.info> wrote:

On Mon, Dec 06, 2021 at 02:08:46AM +1100, Chris Angelico wrote:

...

I want to have them syntactically as part of the body of the function, and semantically as part of the function call.

Then you'll love the status quo, because that's exactly what we have now! *wink*

Aaaaaand that's what I get for typing emails at 2AM. LOL. Syntactically as part of the signature of the function, is what I meant to say. Thanks for the correction.

...

As a function begins executing, a ton of stuff happens, including allocating arguments to parameters, and providing default values for optional parameters that weren't passed.

And neither of those things are part of the function body. They are set up *before* the function's code object is called.

Correct. Including handling default values.

...

All I want to change is the way that defaults can be provided.

Right. And the community reaction on this mailing list is underwhelming. Apart from Abe, who is rather vigourly in favour of the PEP as it stands, I'm not really sure this PEP has much support as it stands now.

And one of the most frequent issues raised is that **all** you want to do is change the way that the late defaults can be provided.

You're the PEP author, but maybe you should consider listening to community feedback? Just sayin'.

Believe you me, I am listening. And quite frankly, the tone of this list is sounding like "shut up, go away, don't do anything, because there are other proposals that nobody can be bothered writing up, but if they existed, they'd be way better than what you're doing". Not exactly encouraging, since nobody is willing to do the work of writing up proposals, but is plenty willing to criticize.

...

Tell me, what's the first bytecode instruction in the function g here?

def f(x): print(x) return lambda: x

Why do you think that is an argument against my position? As you said:

...

CPython bytecode is an implementation detail, and the fact that there's bytecode to do this or that is not part of the language definition.

Bytecode can and does change.

I'm talking about the language definition: late bound defaults should be compiled into their own publicly accessible callable functions. I don't care what bytecode is emitted to do that. Whatever it is, it will change in the future, without changing the Python semantics one whit.

Yes. So why is it a problem for the default's evaluation to be part of the function's bytecode? I don't understand why this is such a big hassle.

...

def func(arg=>_implementation()): ...

No magic, just perfectly normal coding practices.

You've just made a good argument against your own PEP. We don't need new syntax for late-bound defaults, just use the perfectly normal coding practices that have worked fine for 30 years.

Hmm.... you mean that "arg=>_implementation()" works currently? You're complaining that complicated argument defaults can't be separately tested, and that we need magic to do the refactoring for us and make them testable. I say that, if it needs to be separately tested, refactor it, and if it's simple enough to not be worth refactoring, it doesn't get separately tested. That's an argument against your "break it into a separate magical subfunction" proposal, but not against the PEP itself.

Sure. And why should early-bound defaults be available in a publicly accessible dunder attribute, when they could be compiled directly into the function code object?

Because they're part of the FUNCTION, not the CODE. There's a big difference. Consider: for i in range(10): def func(i=i): ... There is only one code object. There are multiple function objects, each with their own early-bound defaults. By definition, since the defaults are evaluated as part of the function definition, the code to evaluate that MUST be part of the surrounding context, not the function being defined. And it's not introspectable at the moment, only the resulting value is. You are asking for far more than the language currently offers in any related context.

Hell, the entire code object, and most of the function object, could be compiled into one great big opaque ball of mud. Aside from standard attributes inherited from object, all the function really needs is a name and a binary blob of everything else.

But that's not the sort of language we have.

Indeed. We have all kinds of useful introspection tools. And late-bound defaults already have quite a few introspection features, most notably a source-code-reconstruction of the expression. That's more than you get for early-bound defaults. (In discussing this, I have noted a hole in the specification: it would be nice to have a way to see, on the code object, which args have late-bound defaults. Consider that a specification bug to be fixed.) You're setting the bar for PEP 671 a long way higher than literally everything else in the language... and yet you refuse to code your own reference implementation for the better version that you propose. Any plans to put some code behind your words? ChrisA

On Mon, Dec 6, 2021 at 5:38 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

On Sun, Dec 5, 2021, 12:33 PM Chris Angelico

...

And quite frankly, the tone of this list is sounding like "shut up, go away, don't do anything, because there are other proposals that nobody can be bothered writing up, but if they existed, they'd be way better than what you're doing". Not exactly encouraging, since nobody is willing to do the work of writing up proposals, but is plenty willing to criticize.

I'll write up my proposal:

"Keep the status quo"

All done.

PEP 671 is very much the same. It does something worthwhile. But it does vastly less than needed to warrant new syntax and semantics. I hope it takes less than 19 years, but a generalized deferred construct is worth waiting for.

You: "Keep the status quo, all done" Also you: "Let's wait for something better" That's what I take issue with. You are simultaneously telling me that this proposal is bad because there's another proposal that would be better, AND saying that you don't want to push for any other proposal. So you're welcome to keep waiting. Meanwhile, I'm going to try to actually accomplish something NOW, not wait for some hypothetical future. ChrisA

On Mon, Dec 6, 2021 at 5:51 AM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:

On Sun, Dec 5, 2021, 1:48 PM Chris Angelico

...

You: "Keep the status quo, all done" Also you: "Let's wait for something better"

Now is better than never. Although never is often better than *right* now.

Neither of which says "the distant and uncertain future of an unwritten proposal is better than either now or never". If you JUST said "keep the status quo, all done", that is a reasonable and consistent position (which I disagree with, but I fully respect). But you then taint your claims with this statement that the reason for keeping the status quo is not that the status quo is better, but that there is a hypothetical idea that might be even better. That's not "now is better than never". That's not "never is better than right now" either. I think it's time for me to drop this entire topic for a while. This list is getting more and more noisy and I'm going to stop contributing to that. ChrisA

Rob Cliffe via Python-ideas writes:

Nobody has attempted (or at least completed) a PEP, never mind an implementation, of a "generalized deferred object/type", in the last N years or decades.

Haskell anything. Ruby blocks. Closer to home, properties and closures. So I don't think the object part is that hard, it's the syntax and semantics that's devilish. The evaluation syntax doesn't seem hard: you just reference the identifier the object is bound to. The creation syntax, though, probably needs to be concise, maybe even familiar, as Chris's proposal is. For the semantics, just in the case of defining an alternative to PEP 671, consider: if you just stuff a generalized deferred expression object into a formal argument's default, you still have to solve the problems that Chris's proposal does: where do you get the appropriate namespace to extract variable values from? and when do you do it? I do admire the elegance of Chris's solution to those questions, but if I were to fit a generalized deferred into that role, I might not adopt the "evaluate deferreds just before entering the function body proper" strategy. And then there's the question of whether referencing the deferred "collapses the waveform" into an object and binds it to the original identifier (as in Chris's proposal, but for more general contexts), or whether the deferred gets reevaluated each time in the current context (as Ruby blocks and Python properties usually do). Steve

Rob Cliffe via Python-ideas writes:

I think you're making my point.

*shrug* You wrote "object", I took you at your word.

You're saying that the object part isn't that hard, but other parts of it are.

For values of "hard" == "non-trivial but mostly bikeshedding". I don't think it will be that much harder to get through than Chris's. And in theory it could be easier: it could be implemented with a new builtin such as "quote_expression" that takes a string, and thus needing no new syntax. I actually don't think that will pass, too clumsy to be of practical use. Although it might fly on the same logic as function annotations did: we add the instance flag that says that "this object is a thunk to be evaluated in the current context" and the code in the interpreter that's needed to use it, and postpone syntax until somebody comes up with a really good proposal. In fact, we could combine this strategy with Steven d'Aprano's proposal for a thunk object, in which case Chris doesn't need syntax himself. I'm sure he won't like that, but it's an idea.

And there seems to be no likelihood of anyone tackling it soon.

There's no way to judge that. David Mertz might post a PEP tomorrow for all you know.

I can't see the point of rejecting something that provides a tangible benefit, now, because some fictitious vapourware *might*, one day, provide another way of doing it.

That's a strawman. The argument is not "Your proposal is good, but not perfect, so we reject it." The basic argument is "Your proposal isn't good enough to deserve syntax for reasons given elsewhere, but if you do this other stuff we would likely support it." Sure, there's some component of "it might interfere with the general facility, or be a small cognitive burden, a warty technical debt, if the general facility is implemented", but that's only a tiny part of why people opposing the proposal oppose it.

On Tue, Dec 07, 2021 at 06:42:18PM +0000, Rob Cliffe via Python-ideas wrote: [The other Stephen]

...

That's a strawman. The argument is not "Your proposal is good, but not perfect, so we reject it."

That IMO is exactly the argument.  It's like saying "I won't buy a car today because in 10/20/50 years time I can probably buy a driverless one".

I think a better analogy for those who reject late-bound defaults but would accept a general deferred evaluation mechanism is: "I won't buy a driverless car today because I have a perfectly good car now, and **driving is not a sufficient burden** that I care for this feature alone. But fully autonomous robots that could drive my car, clean my house, do my chores, now *that's* something I would buy!" [the other Stephen]

...

if _*you*_ do this other stuff we would likely support it." [My emphasis - RC.]

You want *Chris* to implement deferred-evaluation objects?

Clearly not. It's obvious in context that Stephen is talking about *generic* "you". He's not addressing his comment to Chris. Anyone could read the comment and interpret that "you" as themselves, and respond "What a great idea! I'm going to implement deferred evaluation!". You surely don't imagine that Stephen thinks, or implies, that those who want a generic deferred evaluation feature would reject it if it wasn't done specifically by Chris himself. I know that the Python-Ideas community is changable like the wind and rather mercurial, but we've never yet demanded a feature we want be implemented by a *particular person* or else we will reject it. -- Steve