
On Sat, 11 Dec 2021 at 16:30, Christopher Barker <pythonchb@gmail.com> wrote:
Sorry, accidentally off-list.
I did exactly the same a few days ago. On Thu, 9 Dec 2021 at 07:49, Chris Angelico <rosuav@gmail.com> wrote:
BTW, did you intend for this to be entirely off-list?
Nope, and apologies to all, but at least it's given me the opportunity to correct a typo & do some slight reformatting. Here's it is: On Thu, 9 Dec 2021 at 07:25, Adam Johnson <mail.yogi841@gmail.com> wrote:
On Fri, 3 Dec 2021 at 22:38, Chris Angelico <rosuav@gmail.com> wrote:
On Sat, Dec 4, 2021 at 6:33 AM Adam Johnson <mail.yogi841@gmail.com> wrote:
The first unwelcome surprise was:
>>> def func(a=>[]): ... return a ...
>>> import inspect >>> inspect.signature(func).parameters['a'].default Ellipsis
Here the current behaviour of returning `Ellipsis` is very unfortunate, and I think could lead to a lot of head scratching — people wondering why they are getting ellipses in their code, seemingly from nowhere. Sure, it can be noted in the official documentation that `Ellipsis` is used as the indicator of late bound defaults, but third-party resources which aim to explain the uses of `Ellipsis` would (with their current content) leave someone clueless.
Yes. Unfortunately, since there is fundamentally no object that can be valid here, this kind of thing WILL happen. So when you see Ellipsis in a default, you have to do one more check to figure out whether it's a late-bound default, or an actual early-bound Ellipsis...
My discomfort is that any code that doesn't do that extra check will continue to function, but incorrectly operate under the assumption that `Ellipsis` was the actual intended value. I wouldn't go so far as to say this is outright backwards-incompatible, but perhaps 'backwards-misleading'.
When attempting to inspect a late-bound default I'd much rather an exception were raised than return value that, as far as any existing machinery is concerned, could be valid. (More on this thought later...)
Additionally I don't think it's too unreasonable an expectation that, for a function with no required parameters, either of the following (or something similar) should be equivalent to calling `func()`:
pos_only_args, kwds = [], {} for name, param in inspect.signature(func).parameters.items(): if param.default is param.empty: continue elif param.kind is param.POSITIONAL_ONLY: pos_only_args.append(param.default) else: kwds[name] = param.default
func(*pos_only_args, **kwds)
# or, by direct access to the dunders
func(*func.__defaults__, **func.__kwdefaults__)
The problem is that then, parameters with late-bound defaults would look like mandatory parameters. The solution is another check after seeing if the default is empty:
if param.default is ... and param.extra: continue
In some situations, though, late-bound defaults do essentially become mandatory. Picking an example you posted yourself (when demonstrating that not using the functions own context could be surprising):
def g(x=>(a:=1), y=>a): ...
In your implementation `a` is local to `g` and gets bound to `1` when no argument is supplied for `x` and the default is evaluated, however **supplying an argument for `x` leaves `a` unbound**. Therefore, unless `y` is also supplied, the function immediately throws an `UnboundLocalError` when attempting to get the default for `y`.
With the current implementation it is possible to avoid this issue, but it's fairly ugly — especially if calculating the value for `a` has side effects:
def g( x => (a:=next(it)), y => locals()['a'] if 'a' in locals() else next(it), ): ...
# or, if `a` is needed within the body of `g`
def g( x => (a:=next(it)), y => locals()['a'] if 'a' in locals() else (a:=next(it)), ): ...
The presence of the above if statement's first branch (which was technically unnecessary, since we established for the purpose of this example all arguments of `func` are optional / have non-empty defaults) hints that perhaps `inspect.Parameter` should grow another sentinel attribute similar to `Parameter.empty` — perhaps `Parameter.late_bound` — to be set as the `default` attribute of applicable `Parameter` instances (if not also to be used as the sentinel in `__defaults__` & `__kwdefaults__`, instead of `Ellipsis`).
Ah, I guess you didn't see .extra then. Currently the only possible meanings for extra are None and a string, and neither has meaning unless the default is Ellipsis; it's possible that, in the future, other alternate defaults will be implemented, which is why I didn't call it "late_bound". But it has the same functionality.
Correct, I did not initially see `.extra`.
Since the value of `.default` was potentially valid (not _obviously_ wrong, like `Parameter.empty`), there was nothing to prompt me to look elsewhere.
As above, even though **I** now know `.extra` exists, pre-PEP-671 code doesn't and will proceed to give misleading values until updated.
Even if the above were implemented, then only way to indicate that the late bound default should be used would still be by omission of that argument. Thus, if we combine a late bound default with positional-only arguments e.g.:
def func(a=>[], b=0, /): ...
It then becomes impossible to programmatically use the given late bound default for `a` whilst passing a value for `b`. Sure, in this simplistic case one can manually pass an empty list, but in general — for the same reasons that it could be "impossible" to evaluate a late bound default from another context — it would be impossible to manually compute a replacement value exactly equivalent to the default.
That's already the case. How would you call this function with a value for b and no value for a?
You're quite right, I couldn't call the function with **no** value for `a`, but (at present, with early-bound defaults) I can call the function with the exact object that's used as the default — by pulling it from `func.__defaults__` (likely directly, if I'm at the REPL — otherwise via `inspect.signature`).
---
Spending some time thinking about my issues with the current implementation and your exchanges with Steven D'Aprano regarding using semi-magical objects within `__defaults__` / `__kwdefaults__` to contain the code for calculating the defaults, I had an idea about a potential alternate approach.
As it stands, any object is valid within `__defaults__` / `__kwdefaults__` and none has intrinsic 'magical' meaning. Therefore, unless that were to change, there's no valid value you could use **within** them to indicate a late-bound default — that includes Steven's use of flagged code objects and your use of `Ellipsis` alike (again, pre-existing code doesn't know to look at `__defaults_extra__` / `__kwdefaults_extra__` / `inspect.Parameter.extra` to prove whether `Ellipsis`, or any other value, is present only as a placeholder).
However, to our advantage, current code also assumes that `__defaults__` and `__kwdefaults__` are, respectively, a tuple and a dict (or `None`) — what if that were no longer true in the case of functions with late-bound defaults?
Instead, one (or both, as appropriate) could be replaced by a callable with the same parameter list as the main function. Upon calling the main function, the `__defaults__` / `__kwdefaults__` would automatically be called (with the same arguments as the function) in order to supply the default values.
Consequently, existing code designed for handling the collection of default values as tuple/dict pair would raise an exception when attempting to iterate or subscript a callable value that was passed instead. Therefore preventing incorrect conclusions about the default values from being drawn.
Furthermore, this would make calculated default values become accessible via manually calling `__defaults__` / `__kwdefaults__`.
This is a (somewhat) basic example to hopefully demonstrate what I'm thinking:
>>> def func(a => [], b=0, /, *, c=1): ... ... >>> # callable since default of `a` is late-bound >>> defaults = func.__defaults__() >>> defaults ([], 0) >>> >>> # only set to a callable when necessary >>> func.__kwdefaults__ {'c': 1} >>> >>> # equivalent to passing only `b` >>> func(func.__defaults__()[0], b)
A slight wrinkle with this idea is that when late-binding is present defaults and keyword defaults may be defined interdependently, yet are normally are stored (and thus accessed) separately — therefore care must be taken. For example:
>>> import itertools >>> count = itertools.count(0) >>> def problematic(a => next(count), *, b => a): ... ... >>> # `a` supplied, default unevaluated >>> problematic.__defaults__(42) (42,) >>> # count remains at zero. >>> count count(0) >>> >>> >>> # `a` not given, thus... >>> problematic.__kwdefaults__() {'b': 0} >>> # ... count incremented (perhaps unintentionally) >>> count count(1) >>> >>> >>> # 'correct' approach >>> defaults = problematic.__defaults__(42) >>> problematic.__kwdefaults__(*defaults) {'b': 42} >>> # `a` supplied, default unevaluated, count remains at `1` >>> count count(1)
---
Finally (and hopefully not buried by the rest of this message), Chris, a heads-up that your reference implementation currently has `=>` as separate `=` & `>` tokens (thus whitespace is valid **between** them) — i.e. probably not what you intend.