PEP 671 review of default arguments evaluation in other languages
A woefully incomplete review of default argument evaluation in other languages. Updates and corrections are welcome. Out of 22 languages apart from Python: - 3 use early binding (default is evaluated at compile or function definition time); - 12 use late binding (default is evaluated at call time); - 1 simulates late binding with a standard idiom; - and 6 do not support default arguments. Note that R's model for defaults in particularly interesting. Early binding ------------- PHP: function f($arg = const) {body} PHP default arguments must be constant expressions, not variables or function calls. I infer from this that they are evaluated at function definition time (compile time?). https://www.php.net/manual/en/functions.arguments.php#functions.arguments.de... Dart: f({arg=const}) {body} Dart default values appear to be restricted to constants, by which I infer that they are evaluated at compile-time. Visual Basic: Sub F(Optional arg As Type = constant) body End Sub VB default values are restricted to constants, by which I infer that they are evaluated at compile time. https://docs.microsoft.com/en-us/dotnet/visual-basic/language-reference/modi... Late binding ------------ Javascript: function f(arg=expression) { body } ECMAScript 2015 (ES6) introduced default arguments to Javascript. Javascript default arguments are evaluated when the function is called (late binding). https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/... https://dev.to/kenbellows/javascript-vs-python-default-function-parameter-va... CoffeeScript: f = (arg = expression) -> body which compiles to JavaScript: f = function(arg) { if (arg == null) { arg = expression; } body; }; CoffeeScript default arguments are evaluated when the function is called. https://stackoverflow.com/questions/23763825/coffeescript-default-arguments C++: void f(type arg = expression); {body} C++ default arguments are evaluated when the function is called (late binding). The rules for C++ default arguments are complicated, for example local variables *usually* cannot be used in the default expression. https://en.cppreference.com/w/cpp/language/default_arguments https://edux.pjwstk.edu.pl/mat/260/lec/PRG2CPP_files/node61.html Ruby: def f(arg = expression) body end Ruby default values are evaluated when the function is called. https://asquera.de/blog/2012-06-29/2-detect-default-argument-evaluation/ Kotlin: fun f(arg: Type = expression) {body} Kotlin default arguments are evaluated when the function is called. https://kotlinlang.org/spec/expressions.html#function-calls-and-property-acc... Elixir: def f(arg \\ expression) do body end Elixir default arguments are evaluated when the function is called. https://til.mirego.com/2021-07-14-elixir-and-default-argument-evaluation https://hexdocs.pm/elixir/Kernel.html#def/2-default-arguments Scala: def f(arg: Type = expression) : Type = {body} Scala default arguments are evaluated when the function is called. https://docs.scala-lang.org/sips/named-and-default-arguments.html#default-ar... D: void f(Type arg = value) {body} It is not entirely clear to me when D evaluates default arguments, but I think it is when the function is called. https://dlang.org/spec/function.html#function-default-args Julia: function f(arg::Type=expression) body end Julia default arguments are evaluated when the function is called. https://docs.julialang.org/en/v1/manual/functions/ Swift: func f(arg: Type = expression) {body} Swift default arguments are evaluated when the function is called. https://docs.swift.org/swift-book/ReferenceManual/Declarations.html#//apple_... https://stackoverflow.com/questions/38464715/when-are-swift-function-default... Raku (Perl 6): sub f($arg = expression) {body} It is unclear to me when Raku evaluates default arguments, but I think it is when the function is called. https://raku.guide/#_default_and_optional_parameters https://perl6advent.wordpress.com/2009/12/09/day-9-having-beautiful-argument... R: f <- function(arg=expression) {body} Default arguments in R are evaluated at need, at call time (lazy late binding). Because they are not evaluated until the argument is needed, they can refer to local variables defined in the body of the function: g <- function(arg=x) {x = 1; arg+1} Calling g with no arguments will return 2. http://r.babo.ist/#/en/lang_4_3_3_argument_evaluation.html https://www.johndcook.com/blog/2008/10/16/default-arguments-and-lazy-evaluat... Simulating late binding ----------------------- Lua: function f(arg) arg = arg or expression body end Lua does not have syntax for default values, but it has a standard idiom for setting a default at call-time. Calling f() with no arguments assigns the special value *nil* to arg, and the "or" operator evaluates to the given expression. https://www.lua.org/manual/5.1/manual.html No default arguments -------------------- Rust, Java, Go, Eiffel, Cobra, Perl: None of these languages support default arguments. Rust apparently has a RFC for function defaults, so it may support them in the future. In the case of Java, Go and Eiffel, it is apparently a deliberate design principle that they not support defaults. In the case of Perl, functions do not have declared arguments at all, the function manually unpacks whatever arguments it requires from the special global variable `@_`. -- Steve
Thank you for doing this research, Steven. The designers of 12 languages have chosen to provide late binding; those of 3 or 4 have provided early binding. I think this is at least tenuous evidence in favour of my belief that late binding is more useful than early binding. Best wishes Rob Cliffe On 03/12/2021 21:05, Steven D'Aprano wrote:
On Sun, Dec 5, 2021 at 2:14 PM Rob Cliffe via Python-ideas <python-ideas@python.org> wrote:
Perhaps, but more importantly, it provides strong evidence that late-binding of argument defaults is a real, viable concept and not a hack. (I also find it notable that quite a few of those blog posts, and even the JavaScript language reference on MDN, call out Python as having surprising behaviour. To people coming from those languages, Python's current behaviour is a gotcha, which would become a much smaller one if late-binding were a language-supported feature.) ChrisA
On Sat, Dec 4, 2021, 10:14 PM Rob Cliffe via Python-ideas
As the person probably most vociferous in opposing this PEP, I absolutely agree that late-binding is more useful. If I were creating a new programming language today, I would certainly make arguments be evaluated on call, not on definition. There are perfectly good ways to "fake" either one if you only have the other. Probably more work is needed to simulate early binding, but there are ways to achieve the same effect. However, that language would not be Python. That ship sailed in 1991. What's being discussed here isn't changing the behavior of binding in `def f(foo=bar)`. Instead, it's a discussion of adding ADDITIONAL syntax for late-binding behavior. I think the proposed syntax is the worst of all the options discussed. But the real issue is that the cases where it is relevant are vanishingly rate, and the extra cognitive, teaching, and maintenance burden is significant. In 90%+ of the functions I've written, default arguments are non-sentinel immutable values. If those were late bound, nothing whatsoever would change. Yes, maybe slightly different bytecodes would exist, but at the Python level, everything works work the same. So this issue only issue is only remotely relevant to <10% of functions with default arguments. However, of those <10%, 98% work perfectly fine with None as a sentinel. Probably fewer than half of functions I've written use named parameters at all. In other words, for somewhere fewer than one in a thousand functions, this new syntax might serve any purpose at all. That purpose is predominantly "avoid using a custom sentinel." A custom sentinel is a SMALL lift. I agree that a custom sentinel, while rare, is a slight wart in a program. I also believe that in this 1/1000 case, there could be a slightly prettier automatic docstrings. But not prettier than writing an explicit docstring in any case. The cost here is that EVERY SINGLE student learning Python needs to add this new construct to their mental load. EVERY book and tutorial needs to be updated. EVERY experienced developer has to spend extra effort understanding and writing code. The COST of implementing this PEP is *quite literally* tens of millions of person days. The benefit is a rare savings of two lines of function body code or a docstring line.
On Sun, Dec 5, 2021 at 3:17 PM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
I wouldn't. There are no default arguments, and nothing needs to be changed.
(which only pertains to named parameters, not positional)?
Actually PEP 671 applies identically to arguments passed by name or position, and identically to keyword-only, positional-or-keyword, and positional-only parameters.
You can put early-bound or late-bound defaults on all three types of parameter, and you can either provide or omit both kinds of argument; the entire matrix has always been possible [1], and the entire matrix will continue to be possible. ChrisA [1] For values of "always" that go back as far as PEP 570, at least, since that's when pos-only params came in. Before that, it was a 2x2 matrix.
On Sat, Dec 4, 2021 at 11:25 PM Chris Angelico <rosuav@gmail.com> wrote:
I do recognize that I *could* call that with named arguments. I also recognize that the long post I wrote in the bath from my tablet is rife with embarrassing typos :-). Technically, I'd need `def add(a, b, /)` to be positional-only. But in practice, almost everyone who writes or calls a function like that passes by position. I'm not sure that I've *ever* actually used the explicit positional-only `/` other than to try it out. If I have, it was rare enough that I had to look it up then, as I did just now. Actually PEP 671 applies identically to arguments passed by name or
Wow! That's an even bigger teaching nightmare than I envisioned in my prior post. Nine (3x3) different kinds of parameters is already too big of a cognitive burden. Doubling that to 18 kinds makes me shudder. I admit I sort of blocked out the positional-only defaults thing. I understand that it's needed to emulate some of the builtin or standard library functions, but I would avoid allowing that in code review... specifically because of the burden on future readers of the code. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, Dec 5, 2021 at 3:39 PM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
No problem. Doesn't really matter. In any case, argument defaults have always been orthogonal with parameter passing styles (named or positional), and that's not changing.
What you're missing here is that it's not 3x3 becoming 3x3x2. Everything is completely orthogonal. For instance, we don't consider string arguments to be a fundamentally different thing from integer arguments: def f(text, times): ... f("spam", 5) They're just... arguments! And it's not massively more cognitive load to be able to pass string arguments by name or position AND to be able to pass integer arguments by name or position. f(text="spam", times=5) f("spam", times=5) That's not a problem, because the matrix is absolutely complete: there is no way in which these interact whatsoever. It's the same with positional and named parameters: the way the language assigns argument values to parameters is independent of the types of those objects, whether they have early-bound defaults, whether they have late-bound defaults, whether they have annotations, and whether the function returns "Hello world". It's not quadratic cognitive load to comprehend this. It is linear. The only thing you need to understand is the one thing you're looking at right now. With function defaults, it is currently the case that any parameter can have a default, so long as there are no positional parameters to its right which lack defaults. That's the only restriction on default arguments. And that restriction is not changing at all by my proposal: it is neither weakened nor strengthened by the fact that the default might be an expression rather than a precomputed value. (By the way, if I ever get the words "argument" and "parameter" wrong, my apologies; but truth be told, everyone does that. The Python grammar specifies that a def statement has a block of params, which is of type arguments_ty. So that's a thing.) CPython currently states that a function's parameters consist of three groups (or five, kinda): def func(pos_only, /, pos_or_kwd, *, kwd_only): ... def func(pos_only, /, pos_or_kwd, *args, kwd_only, **kwargs): ... Inside each group (not counting the collectors *args and **kwargs if present), legality is defined by... well, I'll just quote the grammar file: # There are three styles: # - No default # - With default # - Maybe with default # # There are two alternative forms of each, to deal with type comments: # - Ends in a comma followed by an optional type comment # - No comma, optional type comment, must be followed by close paren # The latter form is for a final parameter without trailing comma. And if you've never thought about type comments, don't worry, neither had I till I was tinkering with the grammar, and we can for the most part ignore them. :) Either the parameter has a default, or it doesn't. A "maybe with default" either looks like a "with default" or a "no default" (and grammatically, it's there to handle that one restriction of "def f(a=1, b):" being invalid, but everything else is fine). I'm not changing any of that. All I'm changing is the default itself: default[default_ty]: | '=' a=expression { _PyPegen_arg_default(p, a, DfltValue) } | '=' '>' a=expression { _PyPegen_arg_default(p, a, DfltExpr) } When a parameter has a default, two options: either it's a default value, or it's a default expression. This change doesn't care what group of parameters it's in - and in fact, it cannot - because any parameter can have, or not have, a default. This is the correct way to add features. Cognitive load goes up linearly, but expressiveness goes up polynomially. Yes, it's possible to write all the different kinds of functions. That's great, that's power! But you don't need to think about every possibility all at once, in a massive N-dimensional matrix. ChrisA
On Sat, Dec 04, 2021 at 11:39:00PM -0500, David Mertz, Ph.D. wrote:
How do you get nine in the first place? Putting aside *args and **kwargs, the existing parameter matrix is either 3x2 or 3x2x∞ depending on whether you include types as part of the matrix. There are three calling conventions for parameters: - positional only - positional or keyword - keyword only and currently two states: - no default - default which makes 3x2 = 6, not 9. Adding a distinction between early and late defaults would make it 3x3=9, not 18. But surely you don't teach those six (or nine) permutations as *independent* features to be learned by rote as separate concepts? Of course when you enumerate through all the possibilities, you get six seperate choices (positional only with no default, etc), but I would hope you aren't teaching them as six independent concepts! The calling convention and the presence of a default are concepts which are orthogonal to each other, so we need only teach them as two choices: - choose a calling convention (and by all means leave out positional only in an introductory course for beginners); - choose whether or not to supply a default. If the PEP is accepted, it doesn't double the number of choices. It just adds one more: - choose between no default, early default or late default. -- Steve
On Sun, Dec 5, 2021 at 6:44 PM Steven D'Aprano <steve@pearwood.info> wrote:
I probably counted wrong. I wrote it late last night in my timezone. I think trying again, we actually have this currently: position-only, no default position-only, early default position-or-keyword, no default position-or-keyword, early default keyword-only, no default (weird one that I've definitely never used before just now) keyword-only, early default Under the proposal we'd add: position-only, late default position-or-keyword, late default keyword-only, late default It's not nearly as much as I was thinking. Of course, it *is* a 50% increase in the number of parameter types. Still, that's not the main reason I oppose the PEP. I don't really think any of these are fully orthogonal. But the number isn't absurdly large. Hmm... I suppose `*args` and `**kws` are also different types as well. Of course, defaults don't make sense for those. So maybe eleven rather than nine (and lower percentage increase). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 05/12/2021 04:01, David Mertz, Ph.D. wrote:
I think you exaggerate, e.g.: - Many students may never use late-binding, and be perfectly happy not knowing about it). - Not just saving two lines but IMO adding simplicity and clarity But no matter. If this were always a compelling argument, Python would *never* be changed. Best wishes Rob Cliffe
On 2021-12-04 20:01, David Mertz, Ph.D. wrote:
This is a key point that I mentioned in another message (although that message doesn't seem to have reaches the list for some reason). Steven's list is very useful but I don't see any mention there of languages that allow BOTH late-binding and early-binding, and distinguishes them with some kind of syntactic flag in the signature. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, Dec 5, 2021, 12:11 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Is its not being done before really a good argument against it? It may be that there simply hasn't been a need in the other languages. Also, on a kind of side note, what would be a situation where early binding is advantageous to late binding? I can't think of one off the top of my head. --
-- Finn (Mobile)
On Sun, Dec 05, 2021 at 05:31:58PM -0700, Finn Mason wrote:
If your language only has one, early binding is better. You can easily simulate late binding if your language only gives you early: def func(arg=None): # You know the drill... if arg is None: arg = expression And you only pay the cost of call-time evaluation of the expression when you actually need it to be evaluated at call-time. But to go the other way is inconvenient and annoying, requiring the use of a global variable (or some other storage) for every parameter: FUNC_DEFAULT_VALUE = expression def func(arg=>None): if arg is None: arg = FUNC_DEFAULT_VALUE And now, your early-bound default still pays the cost of evaluating the sentinel expression (in this case None), but you also pay the cost of a global lookup to get the cached value of the expression. You lose encapsulation (the cached value is no longer part of the function object) and efficiency. Compiled languages with good optimizing compilers may be able to optimize away some of that cost. If your language supports static storage for functions, you can use that. But in general, without come sort of language support, simulating early binding in a language which only provides late binding is not as easy, convenient or efficient as doing it the other way. -- Steve
On 06/12/2021 08:45, Steven D'Aprano wrote:
And if your language gives you late binding, it automatically gives you early binding with no effort at all. Simply make the default value something that you never change.
And you only pay the cost of call-time evaluation of the expression when you actually need it to be evaluated at call-time.
True, late-binding has the overhead of evaluating the default value on every call. I concede that disadvantage. I would think that in many cases it is unimportant. But if it matters - hey, use early binding instead. That's a reason to have both.
See above. No contortions necessary. You simply write: def func(arg=>expression) or if expression is expensive to evaluate: FUNC_DEFAULT_VALUE = expression def func(arg=>FUNC_DEFAULT_VALUE) And you've avoided the "hack" (in quotes because not everyone agrees that it it a hack) of needing a sentinel value. Best wishes Rob Cliffe
On Mon, Dec 06, 2021 at 10:17:06AM +0000, Rob Cliffe via Python-ideas wrote:
Of course it works. It does *exactly* what it is designed to do. There's no exception, no crash, the computer doesn't catch fire. You might as well claim that "import doesn't work" because people have trouble importing modules that aren't on the PYTHONPATH. What they should be asking is "Why is my expectation different from what Python does?"
It might never change, but it is still re-evaluated every time it is needed. That's what late binding means. As I said, an optimizing compiler might be smart enough to recognise that some expressions are a constant, like `1`, but you can't count on it. And if the constant expression is `[]` and you want to use the same list each time, instead of a new list, then you have to store that list away in a global variable that you have to manage.
Indeed. But aren't we discussing what to use if we can only have one? -- Steve
On 06/12/2021 23:13, Steven D'Aprano wrote:
No, why? We already have one. We are discussing (re PEP 671) adding the other. Nobody is proposing to remove early binding and replace it with late binding. Steven, sometimes I think you argue solely because you like arguing. Best wishes Rob Cliffe
Finn Mason writes:
Is its not being done before really a good argument against it?
No, but that's not the whole of the argument being made. The implied argument is that having two different ways to evaluate defaults is complexity that arguably isn't needed: either because late binding is a YAGNI in view of the perfectly usable if not usably perfect arg=sentinel idiom, or because it's expected to be subsumed by a more general deferred evaluation scheme. Then "not done before" means we have no experience to help judge whether the benefits of having *both* are worth the costs of having *both*.
It may be that there simply hasn't been a need in the other languages.
In Lisp, that's true: you just quote the expression in the usual way to turn a funcall into an object.[1] I don't know about the other languages.
Also, on a kind of side note, what would be a situation where early binding is advantageous to late binding?
Besides performance, as Steven d'A mentions, there is the case of a mutable default that should be static (the same object used in all calls to that function). Footnotes: [1] CLtL2 is explicit that a Sufficiently Smart Compiler is allowed to optimize the code of the function if it can determine that the default expression will resolve to a constant. It's not clear to me that that is ever possible in Lisp, for the same reasons it's quite difficult in Python.
On Mon, Dec 06, 2021 at 09:25:23PM +0000, Barry Scott wrote:
You can shoot your self in the foot with early-binding with ease. You have to try harder with late-binding :-)
def func(arg, sessionid=>secrets.token_hex()): ... "Why does the default session ID change every time I call the function, instead of using the same value each time?" -- Steve
Has anyone done a survey of, say, the cpython library codebase to see what proportion of arguments lists would significantly benefit from this PEP given as a proportion of the total codebase? In all the Python I've ever written or seen, I can only thing of one example of a custom sentinel ever being needed. I think people are significantly underestimating the cognitive load this feature would add. On Saturday, December 4, 2021 at 11:01:44 PM UTC-5 David Mertz, Ph.D. wrote:
I absolutely agree that late-binding is more useful. If I were creating a
On Monday, December 6, 2021 at 6:28:10 PM UTC-5 Steven D'Aprano wrote:
Yes, but that's not a big win in my opinion. The None sentinels are not going away. They're baked into the type annotations, and decades of idiomatic usage. If None sentinels are slightly annoying, please let's consider pushing the none-aware operators. Also, I like passing None sentinels because it means that I don't have to edit custom argument dicts, as I mentioned in my other comment. Besides default constructed containers and custom sentinels, I don't see this feature as a big win.
Thank you for doing this research, Steven. The designers of 12 languages have chosen to provide late binding; those of 3 or 4 have provided early binding. I think this is at least tenuous evidence in favour of my belief that late binding is more useful than early binding. Best wishes Rob Cliffe On 03/12/2021 21:05, Steven D'Aprano wrote:
On Sun, Dec 5, 2021 at 2:14 PM Rob Cliffe via Python-ideas <python-ideas@python.org> wrote:
Perhaps, but more importantly, it provides strong evidence that late-binding of argument defaults is a real, viable concept and not a hack. (I also find it notable that quite a few of those blog posts, and even the JavaScript language reference on MDN, call out Python as having surprising behaviour. To people coming from those languages, Python's current behaviour is a gotcha, which would become a much smaller one if late-binding were a language-supported feature.) ChrisA
On Sat, Dec 4, 2021, 10:14 PM Rob Cliffe via Python-ideas
As the person probably most vociferous in opposing this PEP, I absolutely agree that late-binding is more useful. If I were creating a new programming language today, I would certainly make arguments be evaluated on call, not on definition. There are perfectly good ways to "fake" either one if you only have the other. Probably more work is needed to simulate early binding, but there are ways to achieve the same effect. However, that language would not be Python. That ship sailed in 1991. What's being discussed here isn't changing the behavior of binding in `def f(foo=bar)`. Instead, it's a discussion of adding ADDITIONAL syntax for late-binding behavior. I think the proposed syntax is the worst of all the options discussed. But the real issue is that the cases where it is relevant are vanishingly rate, and the extra cognitive, teaching, and maintenance burden is significant. In 90%+ of the functions I've written, default arguments are non-sentinel immutable values. If those were late bound, nothing whatsoever would change. Yes, maybe slightly different bytecodes would exist, but at the Python level, everything works work the same. So this issue only issue is only remotely relevant to <10% of functions with default arguments. However, of those <10%, 98% work perfectly fine with None as a sentinel. Probably fewer than half of functions I've written use named parameters at all. In other words, for somewhere fewer than one in a thousand functions, this new syntax might serve any purpose at all. That purpose is predominantly "avoid using a custom sentinel." A custom sentinel is a SMALL lift. I agree that a custom sentinel, while rare, is a slight wart in a program. I also believe that in this 1/1000 case, there could be a slightly prettier automatic docstrings. But not prettier than writing an explicit docstring in any case. The cost here is that EVERY SINGLE student learning Python needs to add this new construct to their mental load. EVERY book and tutorial needs to be updated. EVERY experienced developer has to spend extra effort understanding and writing code. The COST of implementing this PEP is *quite literally* tens of millions of person days. The benefit is a rare savings of two lines of function body code or a docstring line.
On Sun, Dec 5, 2021 at 3:17 PM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
I wouldn't. There are no default arguments, and nothing needs to be changed.
(which only pertains to named parameters, not positional)?
Actually PEP 671 applies identically to arguments passed by name or position, and identically to keyword-only, positional-or-keyword, and positional-only parameters.
You can put early-bound or late-bound defaults on all three types of parameter, and you can either provide or omit both kinds of argument; the entire matrix has always been possible [1], and the entire matrix will continue to be possible. ChrisA [1] For values of "always" that go back as far as PEP 570, at least, since that's when pos-only params came in. Before that, it was a 2x2 matrix.
On Sat, Dec 4, 2021 at 11:25 PM Chris Angelico <rosuav@gmail.com> wrote:
I do recognize that I *could* call that with named arguments. I also recognize that the long post I wrote in the bath from my tablet is rife with embarrassing typos :-). Technically, I'd need `def add(a, b, /)` to be positional-only. But in practice, almost everyone who writes or calls a function like that passes by position. I'm not sure that I've *ever* actually used the explicit positional-only `/` other than to try it out. If I have, it was rare enough that I had to look it up then, as I did just now. Actually PEP 671 applies identically to arguments passed by name or
Wow! That's an even bigger teaching nightmare than I envisioned in my prior post. Nine (3x3) different kinds of parameters is already too big of a cognitive burden. Doubling that to 18 kinds makes me shudder. I admit I sort of blocked out the positional-only defaults thing. I understand that it's needed to emulate some of the builtin or standard library functions, but I would avoid allowing that in code review... specifically because of the burden on future readers of the code. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, Dec 5, 2021 at 3:39 PM David Mertz, Ph.D. <david.mertz@gmail.com> wrote:
No problem. Doesn't really matter. In any case, argument defaults have always been orthogonal with parameter passing styles (named or positional), and that's not changing.
What you're missing here is that it's not 3x3 becoming 3x3x2. Everything is completely orthogonal. For instance, we don't consider string arguments to be a fundamentally different thing from integer arguments: def f(text, times): ... f("spam", 5) They're just... arguments! And it's not massively more cognitive load to be able to pass string arguments by name or position AND to be able to pass integer arguments by name or position. f(text="spam", times=5) f("spam", times=5) That's not a problem, because the matrix is absolutely complete: there is no way in which these interact whatsoever. It's the same with positional and named parameters: the way the language assigns argument values to parameters is independent of the types of those objects, whether they have early-bound defaults, whether they have late-bound defaults, whether they have annotations, and whether the function returns "Hello world". It's not quadratic cognitive load to comprehend this. It is linear. The only thing you need to understand is the one thing you're looking at right now. With function defaults, it is currently the case that any parameter can have a default, so long as there are no positional parameters to its right which lack defaults. That's the only restriction on default arguments. And that restriction is not changing at all by my proposal: it is neither weakened nor strengthened by the fact that the default might be an expression rather than a precomputed value. (By the way, if I ever get the words "argument" and "parameter" wrong, my apologies; but truth be told, everyone does that. The Python grammar specifies that a def statement has a block of params, which is of type arguments_ty. So that's a thing.) CPython currently states that a function's parameters consist of three groups (or five, kinda): def func(pos_only, /, pos_or_kwd, *, kwd_only): ... def func(pos_only, /, pos_or_kwd, *args, kwd_only, **kwargs): ... Inside each group (not counting the collectors *args and **kwargs if present), legality is defined by... well, I'll just quote the grammar file: # There are three styles: # - No default # - With default # - Maybe with default # # There are two alternative forms of each, to deal with type comments: # - Ends in a comma followed by an optional type comment # - No comma, optional type comment, must be followed by close paren # The latter form is for a final parameter without trailing comma. And if you've never thought about type comments, don't worry, neither had I till I was tinkering with the grammar, and we can for the most part ignore them. :) Either the parameter has a default, or it doesn't. A "maybe with default" either looks like a "with default" or a "no default" (and grammatically, it's there to handle that one restriction of "def f(a=1, b):" being invalid, but everything else is fine). I'm not changing any of that. All I'm changing is the default itself: default[default_ty]: | '=' a=expression { _PyPegen_arg_default(p, a, DfltValue) } | '=' '>' a=expression { _PyPegen_arg_default(p, a, DfltExpr) } When a parameter has a default, two options: either it's a default value, or it's a default expression. This change doesn't care what group of parameters it's in - and in fact, it cannot - because any parameter can have, or not have, a default. This is the correct way to add features. Cognitive load goes up linearly, but expressiveness goes up polynomially. Yes, it's possible to write all the different kinds of functions. That's great, that's power! But you don't need to think about every possibility all at once, in a massive N-dimensional matrix. ChrisA
On Sat, Dec 04, 2021 at 11:39:00PM -0500, David Mertz, Ph.D. wrote:
How do you get nine in the first place? Putting aside *args and **kwargs, the existing parameter matrix is either 3x2 or 3x2x∞ depending on whether you include types as part of the matrix. There are three calling conventions for parameters: - positional only - positional or keyword - keyword only and currently two states: - no default - default which makes 3x2 = 6, not 9. Adding a distinction between early and late defaults would make it 3x3=9, not 18. But surely you don't teach those six (or nine) permutations as *independent* features to be learned by rote as separate concepts? Of course when you enumerate through all the possibilities, you get six seperate choices (positional only with no default, etc), but I would hope you aren't teaching them as six independent concepts! The calling convention and the presence of a default are concepts which are orthogonal to each other, so we need only teach them as two choices: - choose a calling convention (and by all means leave out positional only in an introductory course for beginners); - choose whether or not to supply a default. If the PEP is accepted, it doesn't double the number of choices. It just adds one more: - choose between no default, early default or late default. -- Steve
On Sun, Dec 5, 2021 at 6:44 PM Steven D'Aprano <steve@pearwood.info> wrote:
I probably counted wrong. I wrote it late last night in my timezone. I think trying again, we actually have this currently: position-only, no default position-only, early default position-or-keyword, no default position-or-keyword, early default keyword-only, no default (weird one that I've definitely never used before just now) keyword-only, early default Under the proposal we'd add: position-only, late default position-or-keyword, late default keyword-only, late default It's not nearly as much as I was thinking. Of course, it *is* a 50% increase in the number of parameter types. Still, that's not the main reason I oppose the PEP. I don't really think any of these are fully orthogonal. But the number isn't absurdly large. Hmm... I suppose `*args` and `**kws` are also different types as well. Of course, defaults don't make sense for those. So maybe eleven rather than nine (and lower percentage increase). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 05/12/2021 04:01, David Mertz, Ph.D. wrote:
I think you exaggerate, e.g.: - Many students may never use late-binding, and be perfectly happy not knowing about it). - Not just saving two lines but IMO adding simplicity and clarity But no matter. If this were always a compelling argument, Python would *never* be changed. Best wishes Rob Cliffe
On 2021-12-04 20:01, David Mertz, Ph.D. wrote:
This is a key point that I mentioned in another message (although that message doesn't seem to have reaches the list for some reason). Steven's list is very useful but I don't see any mention there of languages that allow BOTH late-binding and early-binding, and distinguishes them with some kind of syntactic flag in the signature. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, Dec 5, 2021, 12:11 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Is its not being done before really a good argument against it? It may be that there simply hasn't been a need in the other languages. Also, on a kind of side note, what would be a situation where early binding is advantageous to late binding? I can't think of one off the top of my head. --
-- Finn (Mobile)
On Sun, Dec 05, 2021 at 05:31:58PM -0700, Finn Mason wrote:
If your language only has one, early binding is better. You can easily simulate late binding if your language only gives you early: def func(arg=None): # You know the drill... if arg is None: arg = expression And you only pay the cost of call-time evaluation of the expression when you actually need it to be evaluated at call-time. But to go the other way is inconvenient and annoying, requiring the use of a global variable (or some other storage) for every parameter: FUNC_DEFAULT_VALUE = expression def func(arg=>None): if arg is None: arg = FUNC_DEFAULT_VALUE And now, your early-bound default still pays the cost of evaluating the sentinel expression (in this case None), but you also pay the cost of a global lookup to get the cached value of the expression. You lose encapsulation (the cached value is no longer part of the function object) and efficiency. Compiled languages with good optimizing compilers may be able to optimize away some of that cost. If your language supports static storage for functions, you can use that. But in general, without come sort of language support, simulating early binding in a language which only provides late binding is not as easy, convenient or efficient as doing it the other way. -- Steve
On 06/12/2021 08:45, Steven D'Aprano wrote:
And if your language gives you late binding, it automatically gives you early binding with no effort at all. Simply make the default value something that you never change.
And you only pay the cost of call-time evaluation of the expression when you actually need it to be evaluated at call-time.
True, late-binding has the overhead of evaluating the default value on every call. I concede that disadvantage. I would think that in many cases it is unimportant. But if it matters - hey, use early binding instead. That's a reason to have both.
See above. No contortions necessary. You simply write: def func(arg=>expression) or if expression is expensive to evaluate: FUNC_DEFAULT_VALUE = expression def func(arg=>FUNC_DEFAULT_VALUE) And you've avoided the "hack" (in quotes because not everyone agrees that it it a hack) of needing a sentinel value. Best wishes Rob Cliffe
On Mon, Dec 06, 2021 at 10:17:06AM +0000, Rob Cliffe via Python-ideas wrote:
Of course it works. It does *exactly* what it is designed to do. There's no exception, no crash, the computer doesn't catch fire. You might as well claim that "import doesn't work" because people have trouble importing modules that aren't on the PYTHONPATH. What they should be asking is "Why is my expectation different from what Python does?"
It might never change, but it is still re-evaluated every time it is needed. That's what late binding means. As I said, an optimizing compiler might be smart enough to recognise that some expressions are a constant, like `1`, but you can't count on it. And if the constant expression is `[]` and you want to use the same list each time, instead of a new list, then you have to store that list away in a global variable that you have to manage.
Indeed. But aren't we discussing what to use if we can only have one? -- Steve
On 06/12/2021 23:13, Steven D'Aprano wrote:
No, why? We already have one. We are discussing (re PEP 671) adding the other. Nobody is proposing to remove early binding and replace it with late binding. Steven, sometimes I think you argue solely because you like arguing. Best wishes Rob Cliffe
Finn Mason writes:
Is its not being done before really a good argument against it?
No, but that's not the whole of the argument being made. The implied argument is that having two different ways to evaluate defaults is complexity that arguably isn't needed: either because late binding is a YAGNI in view of the perfectly usable if not usably perfect arg=sentinel idiom, or because it's expected to be subsumed by a more general deferred evaluation scheme. Then "not done before" means we have no experience to help judge whether the benefits of having *both* are worth the costs of having *both*.
It may be that there simply hasn't been a need in the other languages.
In Lisp, that's true: you just quote the expression in the usual way to turn a funcall into an object.[1] I don't know about the other languages.
Also, on a kind of side note, what would be a situation where early binding is advantageous to late binding?
Besides performance, as Steven d'A mentions, there is the case of a mutable default that should be static (the same object used in all calls to that function). Footnotes: [1] CLtL2 is explicit that a Sufficiently Smart Compiler is allowed to optimize the code of the function if it can determine that the default expression will resolve to a constant. It's not clear to me that that is ever possible in Lisp, for the same reasons it's quite difficult in Python.
On Mon, Dec 06, 2021 at 09:25:23PM +0000, Barry Scott wrote:
You can shoot your self in the foot with early-binding with ease. You have to try harder with late-binding :-)
def func(arg, sessionid=>secrets.token_hex()): ... "Why does the default session ID change every time I call the function, instead of using the same value each time?" -- Steve
Has anyone done a survey of, say, the cpython library codebase to see what proportion of arguments lists would significantly benefit from this PEP given as a proportion of the total codebase? In all the Python I've ever written or seen, I can only thing of one example of a custom sentinel ever being needed. I think people are significantly underestimating the cognitive load this feature would add. On Saturday, December 4, 2021 at 11:01:44 PM UTC-5 David Mertz, Ph.D. wrote:
I absolutely agree that late-binding is more useful. If I were creating a
On Monday, December 6, 2021 at 6:28:10 PM UTC-5 Steven D'Aprano wrote:
Yes, but that's not a big win in my opinion. The None sentinels are not going away. They're baked into the type annotations, and decades of idiomatic usage. If None sentinels are slightly annoying, please let's consider pushing the none-aware operators. Also, I like passing None sentinels because it means that I don't have to edit custom argument dicts, as I mentioned in my other comment. Besides default constructed containers and custom sentinels, I don't see this feature as a big win.
participants (9)
-
Barry Scott -
Brendan Barnwell -
Chris Angelico -
David Mertz, Ph.D. -
Finn Mason -
Neil Girdhar -
Rob Cliffe -
Stephen J. Turnbull -
Steven D'Aprano