re.match(pattern, string, require=True)
Hey, I bet this has been discussed before but I couldn't find it. I'd appreciate it if anyone could point me to that thread. I'm sick of seeing "AttributeError: 'NoneType' object has no attribute 'foo'" whenever there's a `re.match` operation that fails while the code expects it to succeed. What do you think about a flag `require` such that `re.match(pattern, string, require=True)` would either return a match or raise an exception with an actually useful message? Thanks, Ram.
I don't see how it's more likely that people would remember to add a `require=True` flag than to add `if m: raise RuntimeError("No match")`. The problem here is people forgetting that a match can fail, not lack of a means to handle that problem. Paul On Sat, 21 Oct 2023 at 11:38, Ram Rachum <ram@rachum.com> wrote:
Hey,
I bet this has been discussed before but I couldn't find it. I'd appreciate it if anyone could point me to that thread.
I'm sick of seeing "AttributeError: 'NoneType' object has no attribute 'foo'" whenever there's a `re.match` operation that fails while the code expects it to succeed. What do you think about a flag `require` such that `re.match(pattern, string, require=True)` would either return a match or raise an exception with an actually useful message?
Thanks, Ram. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/PLF46R... Code of Conduct: http://python.org/psf/codeofconduct/
It's a little similar to the reasoning behind PEP 618 (adding the `strict` argument to `zip`). A keyword argument is easier to add, and makes the code less ugly, then an `if` clause. When I don't have that `if` clause you mentioned in my code, it's not because I forgot, it's because I don't want an extra clause for something I don't think is going to happen. Also, a keyword argument enables code linters to enforce a rule that the `require` argument must always be specified. (Example <https://docs.astral.sh/ruff/rules/zip-without-explicit-strict/>.) On Sat, Oct 21, 2023 at 1:45 PM Paul Moore <p.f.moore@gmail.com> wrote:
I don't see how it's more likely that people would remember to add a `require=True` flag than to add `if m: raise RuntimeError("No match")`. The problem here is people forgetting that a match can fail, not lack of a means to handle that problem.
Paul
On Sat, 21 Oct 2023 at 11:38, Ram Rachum <ram@rachum.com> wrote:
Hey,
I bet this has been discussed before but I couldn't find it. I'd appreciate it if anyone could point me to that thread.
I'm sick of seeing "AttributeError: 'NoneType' object has no attribute 'foo'" whenever there's a `re.match` operation that fails while the code expects it to succeed. What do you think about a flag `require` such that `re.match(pattern, string, require=True)` would either return a match or raise an exception with an actually useful message?
Thanks, Ram. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/PLF46R... Code of Conduct: http://python.org/psf/codeofconduct/
On Sat, 21 Oct 2023 at 21:57, Ram Rachum <ram@rachum.com> wrote:
It's a little similar to the reasoning behind PEP 618 (adding the `strict` argument to `zip`).
Not quite, since without strict, zip will truncate - it doesn't have a different return value.
A keyword argument is easier to add, and makes the code less ugly, then an `if` clause. When I don't have that `if` clause you mentioned in my code, it's not because I forgot, it's because I don't want an extra clause for something I don't think is going to happen. Also, a keyword argument enables code linters to enforce a rule that the `require` argument must always be specified. (Example.)
What about an if with the match inside it? if m := re.match(...): ... That's one of the motivating examples behind the walrus after all. ChrisA
On Sat, Oct 21, 2023 at 10:01 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sat, 21 Oct 2023 at 21:57, Ram Rachum <ram@rachum.com> wrote:
What about an if with the match inside it?
if m := re.match(...): ...
That's one of the motivating examples behind the walrus after all.
I love that, but it mostly makes sense for "if there's a match do this, otherwise do that" where most cases fall into "I'm absolutely sure there's a match here and here's what we should do with that match", and when that "absolutely sure" fails, the proper way to deal with that is by raising an exception.
On Sun, 22 Oct 2023 at 06:11, Ram Rachum <ram@rachum.com> wrote:
On Sat, Oct 21, 2023 at 10:01 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sat, 21 Oct 2023 at 21:57, Ram Rachum <ram@rachum.com> wrote:
What about an if with the match inside it?
if m := re.match(...): ...
That's one of the motivating examples behind the walrus after all.
I love that, but it mostly makes sense for "if there's a match do this, otherwise do that" where most cases fall into "I'm absolutely sure there's a match here and here's what we should do with that match", and when that "absolutely sure" fails, the proper way to deal with that is by raising an exception.
Oh, you mean like AttributeError? ChrisA
On Sat, Oct 21, 2023 at 10:30 PM Chris Angelico <rosuav@gmail.com> wrote:
I love that, but it mostly makes sense for "if there's a match do this, otherwise do that" where most cases fall into "I'm absolutely sure there's a match here and here's what we should do with that match", and when that "absolutely sure" fails, the proper way to deal with that is by raising an exception.
Oh, you mean like AttributeError?
What I propose is like AttributeError in that they are both exceptions, but unlike AttributeError in that it'll communicate the problem effectively in a way that's easy to understand, especially by people who aren't Python experts. When you and I see this: AttributeError: 'NoneType' object has no attribute 'strip' Our brain that has been subjected to over a decade of Python work automatically processes it, completely dismissing the 'strip' as a red herring and concluding that a few lines above that line there was a regex match that was expected to succeed but failed. Then we have to roll up our sleeves and rerun this code with some instrumentation to find out what the offending string was. I propose to skip that entire exercise and go straight to: re.NoMatchError: 'foobar' does not match pattern '^[0-9]+' That would be so much nicer both for beginners and experienced developers. What do you think about that? Thanks, Ram.
On Sun, 22 Oct 2023 at 06:37, Ram Rachum <ram@rachum.com> wrote:
On Sat, Oct 21, 2023 at 10:30 PM Chris Angelico <rosuav@gmail.com> wrote:
I love that, but it mostly makes sense for "if there's a match do this, otherwise do that" where most cases fall into "I'm absolutely sure there's a match here and here's what we should do with that match", and when that "absolutely sure" fails, the proper way to deal with that is by raising an exception.
Oh, you mean like AttributeError?
What I propose is like AttributeError in that they are both exceptions, but unlike AttributeError in that it'll communicate the problem effectively in a way that's easy to understand, especially by people who aren't Python experts.
When you and I see this:
AttributeError: 'NoneType' object has no attribute 'strip'
Which is strong evidence of a bug in your code. You're trying to tell me that you want a way to enforce that, if the regex doesn't match, it's a bug in your code. This seems to do that perfectly well. If it's NOT a bug when the regex doesn't match, you have the standard conditional form available. I'm not seeing a problem here. ChrisA
On 2023-10-21 21:15, Chris Angelico wrote:
On Sun, 22 Oct 2023 at 06:37, Ram Rachum <ram@rachum.com> wrote:
On Sat, Oct 21, 2023 at 10:30 PM Chris Angelico <rosuav@gmail.com> wrote:
I love that, but it mostly makes sense for "if there's a match do this, otherwise do that" where most cases fall into "I'm absolutely sure there's a match here and here's what we should do with that match", and when that "absolutely sure" fails, the proper way to deal with that is by raising an exception.
Oh, you mean like AttributeError?
What I propose is like AttributeError in that they are both exceptions, but unlike AttributeError in that it'll communicate the problem effectively in a way that's easy to understand, especially by people who aren't Python experts.
When you and I see this:
AttributeError: 'NoneType' object has no attribute 'strip'
Which is strong evidence of a bug in your code. You're trying to tell me that you want a way to enforce that, if the regex doesn't match, it's a bug in your code. This seems to do that perfectly well.
If it's NOT a bug when the regex doesn't match, you have the standard conditional form available. I'm not seeing a problem here.
I think what the OP wants is to have re.match either return a match or raise an exception.
On Sun, 22 Oct 2023 at 11:29, MRAB <python@mrabarnett.plus.com> wrote:
I think what the OP wants is to have re.match either return a match or raise an exception.
Yes, and my point is that simply attempting to access an attribute will do exactly that. It's not a silent failure. Why create a new argument, then mandate that you use it everywhere, just to achieve what's already happening? ChrisA
On 10/21/2023 8:31 PM, Chris Angelico wrote:
On Sun, 22 Oct 2023 at 11:29, MRAB<python@mrabarnett.plus.com> wrote:
I think what the OP wants is to have re.match either return a match or raise an exception. Yes, and my point is that simply attempting to access an attribute will do exactly that. It's not a silent failure.
Why create a new argument, then mandate that you use it everywhere, just to achieve what's already happening?
Because the end user message would be much better, and the exception would point to the exact line where the match didn't occur, instead of some subsequent line. Eric
Chris Angelico writes:
Why create a new argument, then mandate that you use it everywhere, just to achieve what's already happening?
"Newbies don't read code backwards very well" seems to be the point. While I'm not of the school that "I learned this painfully, so newbies should learn this painfully", I do think that novice Python programmers should learn that 1. "None has no .xxx attribute" means that some previous code (often but not always a regex match) was unable to perform some task and returned None to indicate failure. 2. If the failure was expectable, your code is buggy because it didn't test for None, and if it was unexpected, some code somewhere is buggy because it allowed an invariant to fail. On the cost side, there are so many cases where a more finely divided Exception hierarchy would help novices quite a bit but experts very little that this case (easy to learn) would open the floodgates. I believe Guido has specifically advised against such a hierarchy. I'm against this change. Steve
Just as a further note, it's perfectly possible to write a helper: def ensure_match(pattern, string): m = re.match(pattern, string) if m is None: raise ValueError(f"Provided string did not match {pattern}") return m If the project is concerned about failures to check the return value of matches, then using a helper like this seems like a reasonable way of addressing this (far more effective than living with the problem until a flag gets added to the stdlib and the project can drop support for older Python versions...) If the intention here is simply to "make it easier for people to remember" in the future, without being tied to any actual real world use case, then I don't see how adding a (just as easily forgettable) boolean flag is any significant improvement. Paul On Sun, 22 Oct 2023 at 10:19, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Chris Angelico writes:
Why create a new argument, then mandate that you use it everywhere, just to achieve what's already happening?
"Newbies don't read code backwards very well" seems to be the point.
While I'm not of the school that "I learned this painfully, so newbies should learn this painfully", I do think that novice Python programmers should learn that
1. "None has no .xxx attribute" means that some previous code (often but not always a regex match) was unable to perform some task and returned None to indicate failure. 2. If the failure was expectable, your code is buggy because it didn't test for None, and if it was unexpected, some code somewhere is buggy because it allowed an invariant to fail.
On the cost side, there are so many cases where a more finely divided Exception hierarchy would help novices quite a bit but experts very little that this case (easy to learn) would open the floodgates. I believe Guido has specifically advised against such a hierarchy. I'm against this change.
Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MIE4OF... Code of Conduct: http://python.org/psf/codeofconduct/
The *re* module is a black swan, because most of stdlib raises exceptions on invalid arguments or not being able to deliver. It's impossible to change *re* now, so wrapping the calls should be the right solution. -- Juancarlo Añez mailto:apalala@gmail.com On Sun, Oct 22, 2023 at 5:19 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Chris Angelico writes:
Why create a new argument, then mandate that you use it everywhere, just to achieve what's already happening?
"Newbies don't read code backwards very well" seems to be the point.
While I'm not of the school that "I learned this painfully, so newbies should learn this painfully", I do think that novice Python programmers should learn that
1. "None has no .xxx attribute" means that some previous code (often but not always a regex match) was unable to perform some task and returned None to indicate failure. 2. If the failure was expectable, your code is buggy because it didn't test for None, and if it was unexpected, some code somewhere is buggy because it allowed an invariant to fail.
On the cost side, there are so many cases where a more finely divided Exception hierarchy would help novices quite a bit but experts very little that this case (easy to learn) would open the floodgates. I believe Guido has specifically advised against such a hierarchy. I'm against this change.
Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MIE4OF... Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, 23 Oct 2023 at 01:14, Juancarlo Añez <apalala@gmail.com> wrote:
The re module is a black swan, because most of stdlib raises exceptions on invalid arguments or not being able to deliver.
This is a comparison function, and like every other comparison function, it signals its results with a return value. It returns a truthy value if the regular expression matches, and a falsy value if it does not. As such, it is perfectly in line with fnmatch.fnmatch(), all of the str.is*() methods, math.isclose(), and pretty much everything else. So I guess by your logic, Python's standard library must have come from Perth, as it is nothing but black swans. ChrisA
On 23/10/23 1:36 am, Juancarlo Añez wrote:
The *re* module is a black swan, because most of stdlib raises exceptions on invalid arguments or not being able to deliver.
Most of the time, failure to match an re is not a programming error. Often it's perfectly normal. Sometimes it's the result of invalid user input, but that's the fault of the user, not the programmer. -- Greg
It's hard to overstate how "normal" a non-match is. A typical program might examine thousands of strings to identify the ten that match a pattern. Exceptions shouldn't be used for cases that are in no way exceptional. On Sun, Oct 22, 2023, 7:27 PM Greg Ewing <gcewing@snap.net.nz> wrote:
On 23/10/23 1:36 am, Juancarlo Añez wrote:
The *re* module is a black swan, because most of stdlib raises exceptions on invalid arguments or not being able to deliver.
Most of the time, failure to match an re is not a programming error. Often it's perfectly normal. Sometimes it's the result of invalid user input, but that's the fault of the user, not the programmer.
-- Greg _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/4JNKSU... Code of Conduct: http://python.org/psf/codeofconduct/
I feel like this is all example of "not every one line function needs to be in the standard library." You can easily write your own 'match_or_raise()'... I guess it would take two lines, actually. On Sat, Oct 21, 2023, 2:42 PM Ram Rachum <ram@rachum.com> wrote:
Hey,
I bet this has been discussed before but I couldn't find it. I'd appreciate it if anyone could point me to that thread.
I'm sick of seeing "AttributeError: 'NoneType' object has no attribute 'foo'" whenever there's a `re.match` operation that fails while the code expects it to succeed. What do you think about a flag `require` such that `re.match(pattern, string, require=True)` would either return a match or raise an exception with an actually useful message?
Thanks, Ram. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/PLF46R... Code of Conduct: http://python.org/psf/codeofconduct/
participants (9)
-
Chris Angelico -
David Mertz, Ph.D. -
Eric V. Smith -
Greg Ewing -
Juancarlo Añez -
MRAB -
Paul Moore -
Ram Rachum -
Stephen J. Turnbull