Hi Steven, On Sun, Dec 12, 2021 at 4:52 PM Steven D'Aprano <steve@pearwood.info> wrote:
I think you have identified a real pain point, function wrapping, but it is a much bigger pain point that just function defaults, and it won't be made appreciably worse by late-bound defaults.
Unfortunately I don't think either of those things is true.
Over the years, I have written about this pain point a number of times. It is much more significant than just defaults: it is hits parameter naming, and order, and annotations.
The problem is that we have no good notation to say that one function inherits its signature (possibly with changes) from another function.
This is especially noticable during rapid experimental development, where the signature of functions might be changing rapidly, but it occurs in stable code too. Even if the signature is changing, the wrapper nevertheless has to be aware of the signature, and duplicate it.
The idea that this problem is unique, or especially acute, for function defaults is false. It is a problem across the board:
- if the *order of parameters* changes, the wrapping function must also change, unless it exclusively uses keyword arguments;
- if the *parameter names* change, the wrapping function likewise must also change, unless it exclusively uses positional arguments;
- if the *parameter type declarations* change, the wrapping function should also change, lest the reader get confused, and static or runtime type checkers flag the code as wrong.
Or to put it more succinctly:
The wrapper must be aware of the wrapped signature in order to duplicate it. Function signatures are code, and this is a violation of DRY.
I think in fact the default-arguments problem is significantly _more_ pervasive and significant than the other issues you've identified. One reason is the "spectrum of function wrapping" I mentioned earlier. Parameter order and often names are only relevant for wrappers that are attempting to exactly (or near-exactly) imitate the signature of the wrapped function, e.g. so they can be directly substituted for it. But there is a much broader set of functions (in the "middle" of the spectrum) that are not attempting to exactly mirror the signature of any called function, and yet must thread some particular defaulted argument through to it. Perhaps these should not be called "wrappers," and we should distinguish the broader "defaulted-argument-threading problem" from the narrower "function wrapping" problem. (I think also argument names are a much lesser problem since a) renaming things is a common operation in refactoring code, and many tools already have good support for it and b) failure to rename one argument when you rename the other leads to a discrepancy in signature, but not a bug in behavior. Forgetting to change one non-trivial default value when you change another one can easily lead to silent bugs. Similarly forgetting to change a matching type will easily be caught by a type checker, rather than silently leading to wrong behavior.)
The only solution to this right now is to use `*args, **kwargs`.
In fact, sticking to simple default argument values is an easy and very effective solution to the defaulted-argument-threading problem, and valuable in many cases where no other part of the function wrapping problem is relevant.
If that hurts introspection, then that demonstrates a weakness in our introspection tools that needs to be fixed.
No objection to hypothetical future improvements in introspection tools! But it is nevertheless the case that today a `*args, **kwargs` signature is much inferior for almost any use, in any tooling I've ever seen. Even in a hypothetical future with better tooling it will always remain less legible to a reader of the source code without tooling assistance, and there will always be plenty such readers of any code. So it remains contradictory for a PEP that claims to improve function signatures to recommend the use of the least legible form of signature available. [Skipping discussion of undefined. It's an interesting proposal with its own set of pros and cons, but I think it's too far off topic from PEP 671 and I don't want to digress.]
In any case, coming back to this PEP:
- late defaults do not make the wrapper problem appreciably worse;
I don't think this is true. Aside from the question of whether they encourage more use of complex defaults (numerous examples already given in favor of the PEP certainly suggest so!), late defaults make the problem qualitatively worse by introducing a new category of complex defaults that may be entirely impossible for a "wrapping" function to duplicate, even if they would prefer duplication over the other options. (The wrapping function could probably substitute a sentinel in these cases and then _also_ duplicate the logic of the complex default in its body, but it's not a great argument for the feature that its use will force related APIs back to sentinels, plus still needing duplicated logic, whereas if the function had just used a sentinel instead of the complex late-bound default there would be no duplication and no signature inconsistency.)
- arguably, the wrapper problem may reduce the scope for people to use late defaults, since they might prefer to use None or some other sentinel, but it doesn't eliminate it.
Of course people can still choose to use sentinels even if late defaults are an option. But that's not the relevant question. We add features and complexity to the language only if on the whole they will tend to improve code, not just because "well, this feature won't force people to make their code any worse." If sentinels are in general a better option than complex defaults, then complex defaults should not be used as an example of the benefits of the PEP, and we should focus on whether it actually solves the `[]` and `{}` problem, which (unlike complex defaults being calculated inside the function body) is an actual problem. Unfortunately I don't think the PEP solves that problem effectively either, because after the PEP `=[]` wil continue to be every bit as much the newbie foot-gun that silently does an unexpected thing as it is today. Carl