Objections to PEP 671 - Summary
There seems to be a problem understanding what the objections to PEP
671 are. Chris A wrote:
"Part of the problem is that it is really REALLY hard to figure out
what the actual objections are. I asked, and the one clear answer I
got was one subjective opinion that the cognitive load exceeded the
benefit. Great! That's one person's concern. I've responded to that
by clarifying parts of the cognitive load problem, and that's about
as far as that can go. But if there's nothing more specific than
that, what do you want me to respond to? How can I address the
objections if the objections are as vague as you're describing?"
Well, I have AFAIK re-read every post in the threads and am
attempting to summarise the objections to PEP 671 that I have found.
Disclaimer: I do NOT claim this is a completely objective survey. I
think by now everyone knows on which side of the fence I sit. In
some places I have added my own (biased) comments. Nonetheless I
have honestly tried to make a fair selection from all the relevant,
reasonably important posts. And if anyone thinks that their, or
anyone else's objection(s) have been
omitted/understated/misrepresented, they are welcome to edit this
post and put the result in a new one.
Disclaimer: I have not referred to Steven d'Aprano's posts, because
I lack the time (ability?) to understand them all. AFAICT he is in
favour of something like this PEP, but a bit different.
AFAIK these were all the objections that were raised. There may be
some overlap or some sub-cases but this is how I've classified them:
(A) Don't like the proposed `=>` syntax.
(B) Too few use cases to bother making a change to the
language. Status quo wins.
(C) Some form of "deferred evaluation object" would do
everything this PEP does and more, and might be incompatible with
it.
(D) The (late-binding) default value should be a first-class
object that can be accessed and manipulated.
(E) Calculation of a "default value" should be done in the
function body, not the function header.
(F) Concerns that functions using late-bound defaults were
harder to wrap.
(G) Backward compatibility.
(H) No other language supports both early- and late-binding.
Paul Moore and Brendan Barnwell were good enough to list their
objections when asked.
Paul Moore:
1. [OBJECTION B] "The problem that the PEP solves simply isn't
common enough, or difficult enough to work around, to justify new
syntax, plus a second way of defining default values."
2. [OBJECTION A] "There's no syntax that has gained consensus,
and the objections seem to indicate that there are some relatively
fundamental differences of opinion involved."
3. [OBJECTION H] "There's no precedent for languages having
*both* types of binding behaviour. Sure, late binding is more
common, but everyone seems to pick one form and stick with it."
4. [OBJECTION C] [paraphrased] deferred expressions are better
Brendan Barnwell:
1. [OBJECTION B] "The status quo is fine. Using None or another
sentinel and checking for it in the body has worked for many years
and is not that big a problem. In theory improvement is always
possible, but there is no urgency to change anything until we have a
proposal with fewer downsides. In addition, as discussed in some
posts on this list, not even all cases of None/sentinel defaults
will be obviated by this proposal."
2. [OBJECTION A] "Most of the proposed syntaxes make it
difficult to visually distinguish the late and early-bound defaults
(because they all look similar to a plain equals sign which will
still mean a regular early-bound default)."
3. [OBJECTION B AGAIN] "Regardless of the syntax, having the
potential for def-time and call-time behavior to be mixed and
interleaved in arbitrary ways within the same function signature is
confusing."
4. [OBJECTION D] 'Currently anything that is a function default
is some kind of Python object that can be inspected, interacted
with,
and used independently of the function/argument whose default
it is. This proposal breaks that assumption. In other words I
don't want anything that is "a default" but is not a "default
VALUE".'
5. [OBJECTION B AGAIN] "Miscellaneous wrinkles. By this I mean
the various sub-discussions about things like what order the late
and early defaults should be evaluated in. This is a sort of
second-order objection for me, because the objections I gave in my
previous message are enough for me to reject the proposal. But even
assuming I agreed with the broad outlines, these subsidiary concerns
leave enough room for confusion that I would not endorse the
proposal. In other words there are too many devils in the details
that I feel would lead to difficult-to-reason-about code and traps
for the unwary."
I now quote the other (most relevant) posts re each of the
objections:
(A) Don't like the proposed `=>` syntax.
Chris A asked if the cognitive burden of distinguishing between `=`
and '=>' was too great (leading with his chin, but hey).
Neil Giradhar: "Yes"
André Roberge: "Yes, prefer a keyword"
JL: "Yes. Something more distinctive feels better"
Barry Scott: [paraphrased] Wouldn't use with => syntax, but
would with @name syntax
David Mertz: "YES! A few weeks later than the prior long
discussion that I read in full, it took a triple take not to read it
as >= (which would mean something syntactical in many cases, just
not what is intended)."
David Mertz: prefers a keyword like `defer`
Ethan Furman: "Yes"
Barry Scott: "Yes"
Brendan Barnwell: "Yes, it is yet another reason not to do
this."
Nicholas Cole: "I would actively avoid using this feature and
discourage people from using it because: I think that this imposes
a significant cognitive burden, not for the simple cases, but when
combined with the more advanced function definition syntax. I think
this has the potential to make debugging large code-bases much
harder."
I quote from one of Chris A's replies re a `defer` keyword:
'The trouble is that this actually would be incompatible. If you can
defer an expensive calculation and have some "placeholder" value in
the variable 'result', then logically, you should be able to have
that placeholder as a function default argument, which would be an
early-bound default of the placeholder. That is quite different in
behaviour from a late-bound default, so if I were to use the word
"defer" for late-bound defaults, it would actually prevent the more
general proposal.'
(B) Too few use cases to bother making a change to the
language. Status quo wins.
potatochowder.com: "Don't find compelling."
David Mertz: "In particular, the cases where None will never, in
any conceivable circumstances, be a non-sentinel value are at least
98% of all the functions (that have named parameters) I've ever
written in Python."
Stephen J. Turnbull: "Want to stick with my None sentinel and
keep my code consistent."
Paul Moore: "I can't think of a realistic case where I'd want to
actually use the new feature."
(C) Some form of "deferred evaluation object" would do
everything this PEP does and more, and might be incompatible
with it.
I think that this has been thrashed out sufficiently that
anything I might say here could only stir up more pointless argument
(which in fact is still ongoing as I write).
(D) The (late-binding) default value should be a first-class
object that can be accessed and manipulated.
Eric V.Smith: [AFAIU, paraphrased] I want the default value to
be an object that I can inspect and change.
David Mertz: "For the reasons Eric Smith and others have pointed
out, I really WANT to keep inspectability of function signatures."
Stephen J. Turnbull: "More than any of these issues, the lack of
a well-defined, properly introspectable object bothers me."
Chris A gave a reply including 'If you consider the equivalent to be
a line of code in the function body, then the signature has become
MASSIVELY more useful. Instead of simply seeing "x=<object
object at 0x7fba1b318690>", you can see "x=>[]" and be able to
see what the value would be.'
(E) Calculation of a "default value" should be done in the
function body, not the function header.
potatochowder.com: [paraphrased] Computation of default belongs
in the function body.
Brendan Barnwell: "Just think that computation (as of defaults)
should be in the function body, not the signature."
I don't find this a compelling argument. Why (other than
"Because")? What works, works. Rob Cliffe
(F) Concerns that functions using late-bound defaults were
harder to wrap.
Paul Moore raised some concerns. AFAIU Chris A and Steven d'A
answered them. I'm not qualified to comment.
(G) Backward compatibility.
Inada Naoki: [paraphrased] Code that explicitly passes None or
other sentinel to a function means that we can't change the
signature of that function to use a late-bound default.
My take: We can use late-bound defaults when we write new functions,
but must be very careful changing the signature of existing
functions. Rob Cliffe
OTHER
On 03/12/2021 Eric V. Smith raised some objections which I
don't understand but quote here:
"Your version is less friendly to type checking. And it doesn't
work with positional-only arguments.
How is the sentinel value private information or an
implementation detail? It's part of the API. It should be clearly
documented. If nothing else, it's can be inspected and discovered."
"It is none the less true that default late-bound values cannot
be modified. Correct? Early-bound ones can."
"This is a tautology. You can't do these things if 671 is
accepted because they will defined as not doable by 671. That's a
strike against it.
My stance is that it should be possible, and a proposal that
makes them not possible with late-bound arguments is deficient."
"I think it's critical that Signature objects be usable with all
types of defaults. And having the default value available as a
string isn't very useful, except for displaying help text. It
wouldn't be possible to create a new Signature object from an
existing Signature object that contains a late-bound argument (say,
to create a new Signature with an additional argument). At least I
haven't seen how it would be possible, since the PEP makes no
mention of Signature objects. Which it definitely should, even if
only to say "late-bound arguments are not designed to work with
Signature objects".
Sentinels are not meaningless. I think you're alienating people
every time you suggest they are."
to which Chris A replied on the same day
"Positional-only args can be done with a list instead of a dict.
Not much harder, although less clear what's happening if you have
multiple."
"Depends on the sentinel. In this example, is the exact value of
the sentinel part of the API?
_SENTINEL = object()
def frobnicate(stuff, extra=_SENTINEL):
...
If the sentinel is None, then it may be part of the API. (Of course,
if it's a deliberately-chosen string or something, then that's
completely different, and then you're not really doing the "optional
argument" thing that I'm talking about here, and there's a very real
default value.) But when it's an arbitrary sentinel like this, I
argue that the precise value is NOT part of the function's API, only
that you can pass any object, or not pass an object at all."
'Yes, but I'm asking you to compare late-bound defaults with the
"sentinel and replace it in the function" idiom, which is a closer
parallel. Can you, externally to the function, change this to use a
new empty set instead of a new empty list? No.'
"My point is that you *already* cannot do them. My proposal doesn't
stop you from doing things you currently can do, it just makes it
easier to spell the parts you can do."
'Please revisit your concerns with regard to the "sentinel and
replace in the function" idiom, as I've stated in the past few
posts. The sentinel itself can be replaced, but that is useless if
the function's body is still looking for the old sentinel.'
'They most certainly ARE usable with all types of defaults, but
instead of a meaningless "=<object object at 0xasdfqwer>", you
get "=[]".'
'Some of them are. Some of them are not. When they are meaningful,
PEP 671 does not apply, because they're not the sort of sentinel I'm
talking about.
Unfortunately, the word "sentinel" means many different things.
There are many cases where a sentinel is a truly meaningful and
useful value, and in those cases, don't change anything. There are
other cases where the sentinel is a technical workaround for the
fact that Python currently cannot represent defaults that aren't
constant values, and those are meaningless sentinels that can be
removed.'
------------------------
Obviously this exchange is of limited use without context, please
see the 03/12/2021 actual posts.
General feedback
Neil Giradhar wrote "I think this question is quite the biased
sample on python-ideas. Please consider asking this to less
advanced python users"
Well, Neil, be careful what you wish for. There was a positive
response to this PEP from
Abdulla Al Kathiri, Jeremiah Vivian, Matt del Valle, Piotr Duda,
Abe Dillon, MarylandBall Productions, Andrew Jaffe, Adam Johnson and
myself,
9 respondents, as against the few who are arguing vociferously
against this PEP. We may or may not be "less advanced Python
users", but here we are.
And even some of those who were against the PEP didn't absolutely
rule out using late-bound defaults themselves (I class the first 2
as roughly neutral and the other 3 as anti):
Barry Scott: "Wouldn't use with => syntax, but would with
@name syntax"
"On a case-by-case basis I might still put
defaulting into the body of the function if that made the intent
clearer."
"I could see me using @file=sys.stdout."
Stephen J. Turnbull: "My issues with Chris's proposal are
described elsewhere, but I don't really see a problem in principle."
André Roberge: 'Currently, I'm thinking "absolutely not".
However, I thought the same about the walrus
operator and I now miss not being able to use it in a program that
includes support for Python 3.6 and where I have literally dozens of
places where I would use it if I could.'
Paul Moore: "Probably not"
"I think that the only thing I might use it for is
to make it easier to annotate defaults (as f(a: list[int] => [])
rather than as f(a: list[int] | None = None)."
"All I'm saying is that the only case when I can
imagine using this feature is for when I want a genuinely opaque way
of behaving differently if the caller omitted an argument (and using
None or a sentinel has been good enough all these years, so it's not
exactly a pressing need)."
Brendan Barnwell: "I hope not. Realistically I might wind up
using it at some point way down the line. I wouldn't start using it
immediately. I still almost never use the walrus and only
occasionally use f-strings."
Brendan, you're really missing out! f-strings are fantastic
(especially for debugging)!😁 Get in there! (I was skeptical too,
until I started using them.)
I hope this post will be useful.
Rob Cliffe