On Wed, Aug 07, 2019 at 07:47:45PM +1000, Chris Angelico wrote:
On Wed, Aug 7, 2019 at 7:33 PM Steven D'Aprano <steve@pearwood.info> wrote:
What's the rush? Let's be objective here: what benefit are we going to get from this change? Is there anyone hanging out desperately for "\d" and "\-" to become SyntaxErrors, so they can... do what?
So that problems can start to be detected. Time and again, Python users on Windows get EXTREMELY confused by the way their code worked perfectly with one path, then bizarrely fails with another. That is a very real problem, and the problem is that it appeared to work when actually it was wrong.
And this change won't fix that, because *good* paths that currently work today will fail in the future, but *bad* paths that silently do the wrong thing will continue to silently do the wrong thing. py> filename = "location\data" # will work correctly <stdin>:1: SyntaxWarning: invalid escape sequence \d py> filename = "location\temp" # doesn't work as expected, but no error py> Effectively, we are hoping that Windows users will infer from the failure of "\d" (say) that they shouldn't use "\t" even though it doesn't raise. Perhaps some of them will, but I maintain we're talking about a small, incremental improvement, not something that will once and for all fix the problem. I don't think this is a benefit for users of any operating system except Windows users. For Linux, Unix, Mac users, one could argue strongly that we're making the string escape experience a tiny bit *worse*, not better. Raymond's example of ASCII art for example. I think the subset of users that this will help is quite small: - users on Windows; - who haven't read or paid attention to the innumerable recommendations on the web and the documentation that they always use forwards slashes in paths; - who happen to use an escape like \d rather than \t; - and will read and understand the eventual SyntaxWarning/Error; - and infer from that error that they should change their path to use forward slashes instead of backslashes; - and all this happens *before* they get bitten by the \t problem and they learn the hard way not to use backslashes in paths. I'm not saying this isn't worth doing. I'm saying it's a small benefit that *right now* is a lot less than the cost to library authors and users.
Python has a history of fixing these problems. It used to be that b"\x61\x62\x63\x64" was equal to u"abcd", but now Python sees these as fundamentally different.
Yes, and we fixed that over a 10+ year period involving no fewer than three full releases in the Python 2.x series and eight full releases in the Python 3.x series, and the transition period is not over yet since 2.7 is not yet EOLed.
Data-dependent bugs caused by a syntactic oddity are a language flaw that needs to be fixed.
There is always a tradeoff between the severity of the flaw and how much pain we are willing to accept to fix it. I think Raymond has made a good case that in this instance, the pain of fixing it *now* is greater than the benefit. (I don't think he has made the case to reverse the depreciation altogether.) If the benefit versus pain never moves into the black, then we should keep the status quo indefinitely, like any other language wart or misfeature we're stuck with due to backwards compatibility. ("Although never is often better than *right* now.") But having said that, I'm confident that given an improved deprecation process that makes it easier for library authors to see the warning before end-users, we will be able to move forward in a release or two.
Because our processes don't work the way we assumed, it turns out that in practice we haven't given developers the deprecation period we thought we had. Read Nathaniel's post, if you haven't already done so:
https://mail.python.org/archives/list/python-dev@python.org/message/E7QCC74O...
He makes a compelling case that while we might have had the promised deprecation period by the letter of the law, in practice most developers will have never seen it, and we will be breaking the spirit of the promise if we continue with the unmodified plan.
Yes, that's a fair complaint. But merely pushing the deprecation back by a version is not solving it. There has to be SOMETHING done differently.
"We must do SOMETHING!!! This is something, therefore we must do it!!!" I agree that we ought to fix the problem with the deprecation warnings. What I don't agree with is the demand that unless I can give a fix for the deprecation warning issue *right now* we must stay the course no matter how annoying and painful it is for users and library authors.
And yet here we are rushing through a breaking change in an accelerated manner, for a change of marginal benefit.
It's not a marginal benefit. For people who try to teach Python on multiple operating systems, this is a very very real benefit.
You mean people like Raymond? *wink* As an educator, you can enable the warning for your students. You could even turn it into an error. You don't have to wait for the warning to be enabled by default.
Now is better than never. Although never is often better than *right* now.
Not sure how the Zen supports what you're saying there, since you're specifically saying "not never, not now, just later". But what do you actually mean by not rushing this into 3.8?
This warning was silent by default in 3.7. It should go back to being silent by default in 3.8, with the provisional aim of making it visible by default in 3.9 and an error in 4.0.
Right now, we're looking at a seriously compromised user-experience for 3.8. People are going to hate these warnings, many of them won't know what to do with them and will be sure that Python is buggy, and for very little benefit.
Then the problem is that people blame Python for these warnings. That is a problem to be solved; we need people to understand that a warning emitted by a library is a *library bug* not a language flaw.
Indeed, but let's be realistic here. How are you going to do that? We could take out an ad on the Superbowl, hire Instagram influencers, send people to re-education camps... *wink* Most experienced developers know that library bugs should be reported to the library, but there's always a few that missed the memo, or who think that b.p.o. is a tracker for all Python projects, or who simply aren't too clear on which libraries are in the std lib and which are not.
Library authors can start _right now_ fixing their code so it's more 3.8 compatible.
Provided that (1) they are aware that this is a problem that needs to be fixed, and (2) they have the round tuits to actually fix it by 3.8.0. Neither are guaranteed.
(1) Yes it is, see above; (2) fair point, but this is restricted to string literals and can be detected simply by compiling the code, so it's a reasonably findable problem.
I'm not saying this is a huge an onerous problem to fix for library authors. Most, I expect, will be able to fix it pretty quick. But there's lag between each step of this: - users notice the warning; - users get motivated to report it to the library; - the library gets patched; - users upgrade to the patched version. The first two steps are the reason for the annoyance factor and poor user-experience. That's why we prefer the library authors see the warnings first, before the end-users, and that's where the current process failed. [...]
And unless you have a plan to do something different in 3.8 that ensures that library devs see the warnings, there's no justification for the delay. All you'll do is defer the exact same problem by another eighteen months. If the warning remains silent in 3.8, how will library devs get any indication that they need to fix something?
We have good anecdotal evidence that most library authors test their libraries with warnings enabled. It is an accident of this specific warning that it didn't work for them. Libraries are, for the most part, trying to do the right thing. -- Steven