Mailman 3 Target-and-expression lists and if-try - Python-ideas

Target-and-expression lists and if-try

Andrew Barnert

31 Dec 2019 31 Dec '19

4:28 p.m.

Every so often, someone suggests that Python should have a pattern matching case statement like Scala, C#, Swift, Haskell, etc. Or they just suggest that “Python needs algebraic data types” but end up concluding that the biggest part Python is missing is not the types themselves, but a practical way to do something useful with them, which basically means pattern matching. Anyway, with the addition of the walrus operator and @dataclass, Python is actually a lot closer than it used to be. I believe with just two smallish syntax features, it would be possible to do almost anything you’d ever want in a library. The first is to extend unpacking assignment to target-or-expression lists. Like this: x, 0, z = vec Just like `x, y, z = vec`, this raises a TypeError if vec isn’t Iterable, and a ValueError if it has more or fewer than 3 elements, and otherwise, it binds x and z to the first and third elements. But it doesn’t bind anything to the second element; instead, it checks if that element is 0, and, if not, raises a ValueError. The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. The actual value of the expression is discarded, but the walrus operator takes care of that: if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!') Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif. Anyway, neither of these features seems very useful on its own. (It should be obvious how to rewrite that example as something just as readable that just unpacks and then checks the values with normal if.) But I think the two of them together will allow a pure-library implementation of pattern matching syntax that reads nicely and can do everything most other languages do—in particular, concisely and readably pattern match and deconstruct dataclasses (and other types with an opt-in protocol or registry, but dataclasses would work out of the box the way Scala case classes, Swift structs, etc. do). If people think either of these features is too horrible to contemplate no matter what the potential benefits, I won’t bother working through the details. But if people do want to see the details (including some good motivating examples—I’ll go through tutorials for those other languages to find some that don’t involve recursing into cons lists and other stuff you wouldn’t want to do in Python…), I’ll be happy to do so, and, if it works out, turn this into a more detailed proposal. And obviously, if people want to bikeshed the spelling before even seeing what it’s good for, well, this is Python-ideas. :)

Show replies by date

Andrew Barnert

31 Dec 31 Dec

4:46 p.m.

To make things a little more concrete (still without actually defining the library or searching for better motivating examples), here’s a slightly rewritten example from the Scala tutorial: def showNotification(notification: Notification) = { notification match { case Email(sender, title, _) => notify(s"You got an email from $sender with title: $title") case SMS(6060842, message) => alert(s"The number's been reconnected! You got an SMS from $number! Message: $message") case SMS(number, message) => notify(s"You got an SMS from $number! Message: $message") case VoiceRecording(name, link) => notify(s"You received a Voice Recording from $name! Click the link to hear it: $link") } } Here it is in more Python-like syntax, with a match/case statement: def show_notification(notification: Notification): match notification: case Email(sender, title, _): notify(f"You got an email from {sender} with title: {title}") case SMS(6060842, message): alert(f"The number's been reconnected! You got an SMS from {number}! Message: {message}") case SMS(number, message): notify(f"You got an SMS from {number}! Message: {message}") case VoiceRecording(name, link): notify(f"You received a Voice Recording from {name}! Click the link to hear it: {link}") And here it is without needing two new keywords, just the if try and target-or-expression unpacking syntax and a (non-magical) library: def showNotification(notification: Notification): with matching(notification) as m: if try Email, sender, title, _ ::= m: notify(f"You got an email from {sender} with title: {title}") elif try SMS, 6060842, message := m alert(f"The number's been reconnected! You got an SMS from {number}! Message: {message}") elif try SMS, number, message := m: notify(f"You got an SMS from {number}! Message: {message}") elif try VoiceRecording, name, link := m: notify(f"You received a Voice Recording from {name}! Click the link to hear it: {link}")

...

On Dec 31, 2019, at 14:31, Andrew Barnert via Python-ideas wrote:

Every so often, someone suggests that Python should have a pattern matching case statement like Scala, C#, Swift, Haskell, etc. Or they just suggest that “Python needs algebraic data types” but end up concluding that the biggest part Python is missing is not the types themselves, but a practical way to do something useful with them, which basically means pattern matching.

Anyway, with the addition of the walrus operator and @dataclass, Python is actually a lot closer than it used to be. I believe with just two smallish syntax features, it would be possible to do almost anything you’d ever want in a library.

The first is to extend unpacking assignment to target-or-expression lists. Like this:

x, 0, z = vec

Just like `x, y, z = vec`, this raises a TypeError if vec isn’t Iterable, and a ValueError if it has more or fewer than 3 elements, and otherwise, it binds x and z to the first and third elements. But it doesn’t bind anything to the second element; instead, it checks if that element is 0, and, if not, raises a ValueError.

The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. The actual value of the expression is discarded, but the walrus operator takes care of that:

if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!')

Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif.

Anyway, neither of these features seems very useful on its own. (It should be obvious how to rewrite that example as something just as readable that just unpacks and then checks the values with normal if.)

But I think the two of them together will allow a pure-library implementation of pattern matching syntax that reads nicely and can do everything most other languages do—in particular, concisely and readably pattern match and deconstruct dataclasses (and other types with an opt-in protocol or registry, but dataclasses would work out of the box the way Scala case classes, Swift structs, etc. do).

If people think either of these features is too horrible to contemplate no matter what the potential benefits, I won’t bother working through the details.

But if people do want to see the details (including some good motivating examples—I’ll go through tutorials for those other languages to find some that don’t involve recursing into cons lists and other stuff you wouldn’t want to do in Python…), I’ll be happy to do so, and, if it works out, turn this into a more detailed proposal.

And obviously, if people want to bikeshed the spelling before even seeing what it’s good for, well, this is Python-ideas. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W64QFN... Code of Conduct: http://python.org/psf/codeofconduct/

Soni L.

4:50 p.m.

On 2019-12-31 7:28 p.m., Andrew Barnert via Python-ideas wrote:

...

Every so often, someone suggests that Python should have a pattern matching case statement like Scala, C#, Swift, Haskell, etc. Or they just suggest that “Python needs algebraic data types” but end up concluding that the biggest part Python is missing is not the types themselves, but a practical way to do something useful with them, which basically means pattern matching.

Anyway, with the addition of the walrus operator and @dataclass, Python is actually a lot closer than it used to be. I believe with just two smallish syntax features, it would be possible to do almost anything you’d ever want in a library.

The first is to extend unpacking assignment to target-or-expression lists. Like this:

x, 0, z = vec

Just like `x, y, z = vec`, this raises a TypeError if vec isn’t Iterable, and a ValueError if it has more or fewer than 3 elements, and otherwise, it binds x and z to the first and third elements. But it doesn’t bind anything to the second element; instead, it checks if that element is 0, and, if not, raises a ValueError.

The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. The actual value of the expression is discarded, but the walrus operator takes care of that:

if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!')

Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif.

Anyway, neither of these features seems very useful on its own. (It should be obvious how to rewrite that example as something just as readable that just unpacks and then checks the values with normal if.)

But I think the two of them together will allow a pure-library implementation of pattern matching syntax that reads nicely and can do everything most other languages do—in particular, concisely and readably pattern match and deconstruct dataclasses (and other types with an opt-in protocol or registry, but dataclasses would work out of the box the way Scala case classes, Swift structs, etc. do).

If people think either of these features is too horrible to contemplate no matter what the potential benefits, I won’t bother working through the details.

But if people do want to see the details (including some good motivating examples—I’ll go through tutorials for those other languages to find some that don’t involve recursing into cons lists and other stuff you wouldn’t want to do in Python…), I’ll be happy to do so, and, if it works out, turn this into a more detailed proposal.

And obviously, if people want to bikeshed the spelling before even seeing what it’s good for, well, this is Python-ideas. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W64QFN... Code of Conduct: http://python.org/psf/codeofconduct/ while try?

Andrew Barnert

5:19 p.m.

On Dec 31, 2019, at 14:58, Soni L. wrote:

...

...
On 2019-12-31 7:28 p.m., Andrew Barnert via Python-ideas wrote:

The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. The actual value of the expression is discarded, but the walrus operator takes care of that:

if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!')

Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif.

...

while try?

Yeah, maybe. One of the inspirations here was Swift’s if let, and Swift does have a corresponding while let—which isn’t useful nearly as often, but is definitely useful sometimes. The first example I found online is this: var view: UIView? = self while let superview = view.superview { count += 1 view = superview } … which seems like it would make sense in Python: view = self while try view := view.superview: count += 1 … even if it actually works for a different reason (we’re not ending when we fail to bind the value out of an Optional that’s nil, but when we try to access an attribute on a value that’s None). I suppose if it is worth doing, we do need to actually think about just if try, or if try and while try, or a general purpose try expression, rather than just crossing our fingers and saying YAGNI.

Greg Ewing

5:35 p.m.

On 1/01/20 11:28 am, Andrew Barnert via Python-ideas wrote:

...

The first is to extend unpacking assignment to target-or-expression lists. Like this:

x, 0, z = vec

But it doesn’t bind anything to the second element; instead, it checks if that element is 0, and, if not, raises a ValueError.

What if you want to use an expression to represent a value to be matched, rather than a literal? E.g. K = 42 x, K, z = vec The intention here that K is treated as a constant, but with your semantics K would be bound to element 1 of vec.

...

The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. If it truly swallows *any* exception, that's an extremely bad idea, for all the same reasons that using a bare "except" clause is bad.

It might be acceptable if it only swallowed a special exception such as PatternMatchError. But then would be fairly specialised towards pattern matching, so using the general word "try" doesn't seem appropriate. At that point you might be better off with a dedicated "switch" or "case" construct. -- Greg

Andrew Barnert

7:18 p.m.

On Dec 31, 2019, at 15:52, Greg Ewing wrote:

...

On 1/01/20 11:28 am, Andrew Barnert via Python-ideas wrote:

...
The first is to extend unpacking assignment to target-or-expression lists. Like this: x, 0, z = vec But it doesn’t bind anything to the second element; instead, it checks if that element is 0, and, if not, raises a ValueError.

What if you want to use an expression to represent a value to be matched, rather than a literal? E.g.

K = 42

x, K, z = vec

The intention here that K is treated as a constant, but with your semantics K would be bound to element 1 of vec.

Yes. I’m surveying the way other languages deal with this to try to figure out what might fit best for Python. Some languages use special syntax to mark either values or targets: let x, K, let z = vec x, @K, z = vec But the simplest solution is to nothing: you have to stick it in an expression that isn’t a valid target, or It’s a target. And I think that might actually work. If the pattern matching library includes this (or you write it yourself): def val(x): return x … then you just write this: x, val(K), z = vec Which doesn’t seem too bad. And notice that any object with a custom __eq__ will be matched by calling that, so your library can have a decorator that turns an arbitrary function into a matcher, or it can include regex matching like Perl or subclass matching (I’m not sure when you want to mix case class switching and inheritance, but Scala goes out of its way to make that work, so presumably there is a use for it…), and so on. I’m not sure this is the best answer, but it seems at least plausible.

...

...
The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. If it truly swallows *any* exception, that's an extremely bad idea, for all the same reasons that using a bare "except" clause is bad.

I haven’t worked through the details on what to swallow yet, because I need to build up a few more detailed examples first. But it’s actually not nearly as bad as a bare except:. First, it’s only guarding a single expression rather than an arbitrary suite. Second, it’s used in a very different context—you’re not catching errors and ignoring them, you’re failing a case and falling over to the next case. (I mean sure, you could misuse it to, say, put a whole try body inside a function and then `if try func(): pass` just to get around a linter warning about bare except, but I’m not too worried about that.) That still may be too bad, and in fact it doesn’t seem likely to be the best option a priori. But I don’t think we can make that call without seeing the worked our examples that I haven’t written yet.

...

It might be acceptable if it only swallowed a special exception such as PatternMatchError.

But then would be fairly specialised towards pattern matching, so using the general word "try" doesn't seem appropriate.

The goal is to use (an extension to) existing unpacking syntax inside if try as the primitive match, and unpacking raises ValueError today, so I think it has to be that or it’s not useful. (Unless we want to change all unpacking errors to a new subclass of ValueError?) And ValueError actually seems right for what “if try” means: we’ve either got an expression with the right value(s). or we’ve got an error from than that tells us there is no such right value(s). Anything else doesn’t seem to make sense in an “if try”. (Although that could easily be a failure of imagination on my part; maybe there are wider uses for the syntax that have nothing to do with pattern matching and don’t even use the walrus operator and I’m just not seeing them because I’m too narrowly focused.) I was worried about TypeError, because that’s what happens when you unpack something that isn’t Iterable. But on further thought, it seems like that’s always going to be a programming error (there’s something wrong with the library, or you just forgot to use the library and tried to match and deconstruct a Notification dataclass instance itself instead of calling matching on it and handling the result), so there’s no problem there. In the other direction, what if the expression you’re trying to match raises a ValueError itself? Or one of the match values raises one from ==? Well, you’ve already got that problem today. Is this code a bug magnet? try: name, domain = email.split('@', 1) except ValueError: raise EmailError(f'invalid address: {email}') We’re not distinguishing between the ValueError from unpacking and any ValueError that may have come from that split method. Plus, in the rare cases where that matters, you should move the split outside the try and stash it in a temporary. The usual way to use if try will already encourage that. Something like: with matching(email.split('@', 1) as m: if try name, val(DOMAIN) :=‘m: local(name) else: remote(domain, email)

...

At that point you might be better off with a dedicated "switch" or "case" construct.

Sure, if we’re willing to use up one or two new keywords and design a whole new syntax for them we can obviously make it do whatever we want. But if it’s possible to get what we want more flexibly, with fewer/smaller changes to the language, without being too ugly or hacky, that seems worth pursuing.

David Mertz

7:54 p.m.

On Tue, Dec 31, 2019 at 8:23 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...

...
K = 42 x, K, z = vec Yes. I’m surveying the way other languages deal with this to try to figure out what might fit best for Python. Some languages use special syntax to mark either values or targets:

let x, K, let z = vec x, @K, z = vec

But the simplest solution is to nothing: you have to stick it in an expression that isn’t a valid target, or It’s a target. And I think that might actually work. If the pattern matching library includes this (or you write it yourself):

What is and isn't a valid target. For example: 'map[key]', 'seq[pos]', 'obj.attr'. Generally a dictionary with key should be a target. Or likewise a sequence with position. But they might be invalid too at runtime. So maybe it's fine for parser to think they are a target, and just blow up with IndexError or the like, as now. But what about that attribute? Maybe it's an assignment target. Or maybe it's a read-only property that contains a desirable value to match. Or maybe it's not read only, but a plain attribute that already exists with a desirable value to match. The bigger problem in my mind is that what I'd like to match isn't generally a specific value. It's more like: type[int], type[int], type[float] = vec Or: type[int], y, z = vec ... I don't like my "proposed" syntax, but it kinda-sort vaguely resembles stuff in PEP 484 and typing module, so using that as placeholder. Or I even want to match: type[int], y < 42, z = vec So your syntax gets me maybe 20% of what I'd want to do if we had pattern matching. And it has edge cases that of course I could learn, but would not be intuitive.

Andrew Barnert

8:54 p.m.

On Dec 31, 2019, at 17:54, David Mertz wrote:

...

...
On Tue, Dec 31, 2019 at 8:23 PM Andrew Barnert via Python-ideas wrote:

...
...
K = 42 x, K, z = vec Yes. I’m surveying the way other languages deal with this to try to figure out what might fit best for Python. Some languages use special syntax to mark either values or targets:

let x, K, let z = vec x, @K, z = vec

But the simplest solution is to nothing: you have to stick it in an expression that isn’t a valid target, or It’s a target. And I think that might actually work. If the pattern matching library includes this (or you write it yourself):

What is and isn't a valid target. For example: 'map[key]', 'seq[pos]', 'obj.attr'.

Sorry, I thought that was obvious, but it isn’t. Anything that’s _syntactically_ a valid target is a target. It doesn’t matter if assigning to it would raise at runtime. Which is pretty much what you ended up working out anyway, but I should have actually said it.

...

The bigger problem in my mind is that what I'd like to match isn't generally a specific value. It's more like:

type[int], type[int], type[float] = vec

That can be done a library, so it doesn’t need any magic syntax. Remember, anything that’s an expression, we’re going to compare with ==. So (ignoring the error handling, documentation, etc. you’d want in a real library): class MatchType(type): def __eq__(self, other): return self == type(other) Now, this:

...

type[int], y, z = vec

is spelled (with that library): MatchType(int), y, z = vec (And if there’s a better way to spell it, or better behavior—maybe using issubclass, since, as I said elsewhere, Scala does go out of its way to let you integrate case pattern matching and inheritance, so maybe there’s a good reason for that—a competing library will do it better.)

...

Or I even want to match:

type[int], y < 42, z = vec

Well, I can’t think of a language where you can actually match that way. That doesn’t necessarily mean we should disallow it, but I can’t imagine how the compiler and interpreter are supposed to figure this out. So, what do you do in other languages? Normally you handle that with a guard clause, which is usually spelled with a postfix “where” or “if” clause that looks exactly like the if clause in a comprehension: MatchType(int), y, z = vec if y < 42 And we could add that. Although I think it belongs on the if try statement, not on assignment statements and expressions. (If you’re just doing a bare assignment, just do the if separately.) But with an as clause, you could actually do it pretty easily with a custom matcher: class MatchFunc: def __init__(self, func): self.func = func def __eq__(self, other): return self.func(other) And now: MatchType(int), MatchFunc(lambda y: y<42) as y, z = vec … which I don’t think you could do this way in any language. But I don’t think you actually want to do it this way. One last possibility is to put the smarts in the de-constructor function you use to get the right side: MatchType(int), 42, y, z = vec.deconstruct_with_ymax() … which you can do (with different syntax—the “unapplier” goes around the left side rather than the right) in at least Scala, but I don’t think it’s very readable here.

...

So your syntax gets me maybe 20% of what I'd want to do if we had pattern matching.

Sure, expanded unpacking alone doesn’t get you pattern matching at all. It’s just one of the primitives needed to built pattern matching. Swift or Haskell’s unpacking alone (without function definitions, case statements/expressions, if let, etc.) also only get you 20% of what you want, in exactly the same way. Most of the time, to do anything useful, you need to use it inside one of those other features, and the same would be true in Python. If people want to see a fully worked out proposal for pattern matching built on these primitives—which I think will get you 80% of what you want, and even more with a better library—I can work it out. If not, at least look at the off-the-cuff example I posted in the second message.

...

And it has edge cases that of course I could learn, but would not be intuitive.

What edge cases?

David Mertz

1 Jan 1 Jan

12:02 a.m.

...

class MatchType(type): def __eq__(self, other): return self == type(other)

Or I even want to match:

type[int], y < 42, z = vec

Well, I can’t think of a language where you can actually match that way. That doesn’t necessarily mean we should disallow it, but I can’t imagine how the compiler and interpreter are supposed to figure this out.

Actually, you've given the solution already: class MatchLT: def __eq__(self, other): return self < other MatchType(int), MatchLT(42), z = vec

...

MatchType(int), MatchFunc(lambda y: y<42) as y, z = vec

Or yeah... to be generic about predicate, that's it. The very ancient library PEAK in Python did multiple dispatch on a predicative basis, which amounts to pattern matching. For example, from a very old article I wrote on this: import dispatch @dispatch.generic() def doIt(foo, other): "Base generic function of 'doIt()'" @doIt.when("isinstance(foo,int) and isinstance(other,str)") def doIt(foo, other): print "foo is an unrestricted int |", foo, other @doIt.when("isinstance(foo,str) and isinstance(other,int)") def doIt(foo, other): print "foo is str, other an int |", foo, other @doIt.when("isinstance(foo,int) and 3<=foo<=17 and isinstance(other,str)") def doIt(foo, other): print "foo is between 3 and 17 |", foo, other @doIt.when("isinstance(foo,int) and 0<=foo<=1000 and isinstance(other,str)") def doIt(foo, other): print "foo is between 0 and 1000 |", foo, other doIt( 1, 'this') # -> foo is between 0 and 1000 | 1 this doIt('x', 1234) # -> foo is str, other an int | x 1234 doIt(10, 'this') # -> foo is between 3 and 17 | 10 this doIt(20, 'this') # -> foo is between 0 and 1000 | 20 this doIt(-7, 'this') # -> foo is an unrestricted int | -7 this try: doIt(2222, 66) except dispatch.interfaces.NoApplicableMethods: print "No Applicable Methods" # -> No Applicable Methods (http://gnosis.cx/publish/programming/charming_python_b22.html) -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

Andrew Barnert

31 Dec 31 Dec

9:18 p.m.

Let me make this bit a little clearer:

...

On Dec 31, 2019, at 17:54, David Mertz wrote:

So your syntax gets me maybe 20% of what I'd want to do if we had pattern matching.

I’m not proposing either of these changes in its own, because I don’t have any use for either one on its own. (Or for unpacking in walrus assignment, which I forgot to mention in the initial email, but Alexander reminded me of it.) I’m just feeling out whether either one is so horrible that any larger proposal (which I don’t have in anywhere near ready-to-share form yet) would have to be rejected on that basis. (Which I think is a serious possibility. Something about if-try still smells weird to me in a way I haven’t figured out yet.)

Guido van Rossum

11:35 p.m.

There were match proposals in the past — have you looked those up? Maybe they solve your problem without that syntax — or maybe they would benefit from it. On Tue, Dec 31, 2019 at 19:20 Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...

Let me make this bit a little clearer:

...
On Dec 31, 2019, at 17:54, David Mertz wrote:

So your syntax gets me maybe 20% of what I'd want to do if we had pattern matching.

I’m not proposing either of these changes in its own, because I don’t have any use for either one on its own. (Or for unpacking in walrus assignment, which I forgot to mention in the initial email, but Alexander reminded me of it.)

I’m just feeling out whether either one is so horrible that any larger proposal (which I don’t have in anywhere near ready-to-share form yet) would have to be rejected on that basis. (Which I think is a serious possibility. Something about if-try still smells weird to me in a way I haven’t figured out yet.)

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JHR55H... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido (mobile)

Guido van Rossum

1 Jan 1 Jan

10:31 a.m.

Here's a proposal for JavaScript that seems to be going through standardization: https://github.com/tc39/proposal-pattern-matching Here's the end of an older python-ideas thread: https://mail.python.org/archives/search?mlist=python-ideas%40python.org&q=pattern+matching+reprise On Tue, Dec 31, 2019 at 9:35 PM Guido van Rossum wrote:

...

There were match proposals in the past — have you looked those up? Maybe they solve your problem without that syntax — or maybe they would benefit from it.

On Tue, Dec 31, 2019 at 19:20 Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...
Let me make this bit a little clearer:

...
On Dec 31, 2019, at 17:54, David Mertz wrote:

So your syntax gets me maybe 20% of what I'd want to do if we had pattern matching.

I’m not proposing either of these changes in its own, because I don’t have any use for either one on its own. (Or for unpacking in walrus assignment, which I forgot to mention in the initial email, but Alexander reminded me of it.)

I’m just feeling out whether either one is so horrible that any larger proposal (which I don’t have in anywhere near ready-to-share form yet) would have to be rejected on that basis. (Which I think is a serious possibility. Something about if-try still smells weird to me in a way I haven’t figured out yet.)

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JHR55H... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido (mobile)

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Andrew Barnert

8:37 p.m.

...

On Jan 1, 2020, at 08:32, Guido van Rossum wrote: Here's a proposal for JavaScript that seems to be going through standardization: https://github.com/tc39/proposal-pattern-matching

The JS solution is interesting, but I’m not sure how well it works for a different language. Python is closer to JS than, say, Scala or Rust are, but it’s still different. The innovation that possibly applies to Python is that you don’t actually need destructuring in a dynamic language. When you discover that a Notification is an Email, you can just access the email attributes (and even methods) in the Notification, so there’s no need to pull them out into separate variables or to “downcast” the Notification to an Email with an as clause. The one that probably doesn’t is that if everything is an object and objects are just dicts of their attributes, you can spell pattern matching as a dict comparison, effectively `all(pattern.get(key, undefined) == match.get(key, undefined) for key in pattern)`. While you can simulate that in Python (with getattr, etc.) I don’t think it’s as natural. If someone else wants to work through that in more detail and try to come up with a JS-inspired proposal for Python, it might turn out very cool. But I think it would be pretty different from the tack I’m taking.

...

Here's the end of an older python-ideas thread: https://mail.python.org/archives/search?mlist=python-ideas%40python.org&q=pattern+matching+reprise

His section on Orthogonality is key here:

...

I feel that this dilemma is one of the core issues why the syntax of switch statements, or pattern matching seems so exceptionally hard.

In the end, it might therefore, indeed, make more sense to find a structure that is more in line with if/elif/else-chains. This would lead to a form of pattern matching with little support for switch statement, though.

The dilemma he’s referring to is that a switch/case (or case/of or match/when etc.) doesn’t fit into the Python statement model properly. So, what would we lose by having “little support for switch statement”? I think usually nothing. The reason it’s necessary in other languages is to preprocess and scope the match, which isn’t relevant to Python. If you need to preprocess a value in some way before passing it to all of the patterns, you can just do that in a separate statement above them; it’s still in the right scope. If you want block structure anyway, you can do that with a with statement (and that also gives you a place to hook exit for features like ensuring that you didn’t fall off the end, if you want that), but that can be optional. So, can we come up with a structure that’s more in line with if/elif? Unlike C switch statements, pattern matching does actually need something more than Python if/elif already provides, but I think extending if/elif rather than coming up with a whole new parallel structure might suffice. So I’m trying to find the simplest extension to if/elif that could work. The key inspiration comes from Scala and other languages building pattern matching structures out of a “match primitive” instead of the traditional solution (taken to its extreme in Haskell) of building the most complex and flexible structure and then designing everything else in terms of how it syntactically translates to that structure. Iterable unpacking is already part of the way to that primitive, but it can’t test values while destructuring (which has already been solved in many languages), and because it raises on failure it can’t be used in an if/elif-like structure (which has already been sort of solved in Swift and Rust). I’m not sure it can be made to work with only a few smallish changes, but I’m hoping it can.

Andrew Barnert

5:31 p.m.

On Dec 31, 2019, at 21:35, Guido van Rossum wrote:

...

There were match proposals in the past — have you looked those up? Maybe they solve your problem without that syntax — or maybe they would benefit from it.

Most of the existing proposals require a lot more syntax—a new form using two or three new keywords or symbols, and often a new dunder protocol as well. What I was trying to do this weekend was come up with something that reads as similarly as possible, with smaller changes to the language. The key is that before the walrus operator, it was hard to imagine how you could leverage if/elif chains (essentially the same way Python covers 90% of what C switch can do without needing a new structure) and tuple unpacking. Now it seems much more feasible. (Also, dataclass is close enough to a “case class”—and provides a model for building them manually—that the proposal no longer requires a new opt-in mechanism for simulating ADTs before it can even get off the ground.)

Random832

3:21 p.m.

On Tue, Dec 31, 2019, at 22:18, Andrew Barnert via Python-ideas wrote:

...

I’m just feeling out whether either one is so horrible that any larger proposal (which I don’t have in anywhere near ready-to-share form yet) would have to be rejected on that basis. (Which I think is a serious possibility. Something about if-try still smells weird to me in a way I haven’t figured out yet.)

1) I think it's not particularly to be useful for other kinds of exception-raising conditions 2) neither destructuring lists nor subscripts/attributes are valid targets for the := operator now. Neither of these constructs seems to be particularly useful outside the other one, which suggests to me they should not be orthogonal in the way you have proposed. Is there a good reason for wanting them to be, or do you just see that as the best way to get a nice looking syntax? I bring up (2) mainly because you mentioned "any syntactically valid target" with the suggestion of using subscripts and attributes in another reply, without any apparent regard for either the fact that these were excluded from PEP 572 or the reasons for doing so.

Andrew Barnert

8:49 p.m.

On Jan 1, 2020, at 13:23, Random832 wrote:

...

Neither of these constructs seems to be particularly useful outside the other one,

Yes, and you already quoted me as saying that.

...

which suggests to me they should not be orthogonal in the way you have proposed. Is there a good reason for wanting them to be, or do you just see that as the best way to get a nice looking syntax?

What I’m trying to do is get a nice looking syntax *with the smallest possible set of changes*. And I think these are candidates for that set. I’m not actually proposing either or both at this point. Because the nice looking syntax may turn out not to work, or the syntax may not be nice enough, or whatever, but I’m only going to discover that by trying and then sharing the result. But the other worry is that, no matter how nice it works out, if the small changes are themselves unacceptable it won’t matter. And that’s what I think other people can point out in advance. If the only problem with “if try” is that it has no compelling use on its own, that’s fine for now. But if, say, it’s syntactically ambiguous, or reads confusing to humans in even some reasonable-seeming case, or the try-expression PEP is up for reconsideration and incompatible with it, etc., that’s *not* fine; anything I build that requires “if try” will not be acceptable, so it’s not worth exploring further.

Random832

2 Jan 2 Jan

8:31 a.m.

On Wed, Jan 1, 2020, at 21:49, Andrew Barnert wrote:

...

On Jan 1, 2020, at 13:23, Random832 wrote:

...
Neither of these constructs seems to be particularly useful outside the other one,

Yes, and you already quoted me as saying that.

I think I read "not proposing them separately" as meaning you weren't submitting two independent proposals, not that they couldn't be used separately if (both) accepted, and it'd be weird for them not to if they're syntactically independent like this.

...

...
which suggests to me they should not be orthogonal in the way you have proposed. Is there a good reason for wanting them to be, or do you just see that as the best way to get a nice looking syntax?

What I’m trying to do is get a nice looking syntax *with the smallest possible set of changes*. And I think these are candidates for that set.

I’m not actually proposing either or both at this point.

Yeah, sorry, I used the word "proposing" because I couldn't think of a good near-synonym that meant something more informal. Mainly, you had mentioned you felt like "if try" had a 'smell' to it but couldn't figure out what it is, and I was trying help identify that, and... well, after thinking about it more, I realized - "if try" may not have many uses, but "if not try" has a million: any code of the form try: x except: y where x is an expression could be written as if not try x: y And therein lies the problem - it's a construct for catching exceptions with no provision for declaring what kind of exception is caught. You may have only envisioned it for ValueError, but it'd be weird for something that *looks* so general to be limited in that way. Incidentally, it turns out that unpacking with the := operator is syntactically problematic-- (a, b := 1, 2) is currently equivalent to (a, (b := 1), 2). Without the outer parentheses it's a syntax error both in an if statement and otherwise, but I'd find it extremely surprising for adding parentheses to radically change the meaning in this way if both forms are valid. I don't want to hijack your thread, but I've thought of something that may be an alternative. An "as" expression that returns a boolean [and so could be used as-is with if/while] could be useful for this and for the cases other people mentioned (ranges, type checking without necessarily requiring an awkward "type[...]" construct, etc) Consider something like if (x, y, z) := vec as ?, 0, ?: # if y == 0 or simply if tup as ?, <0 # if len(tup) == 2 and tup[1] < 0 If an element in the comparison list is ?, it would not be compared at all. If it is a type [perhaps any object with an __instancecheck__, to allow ABCs and perhaps some future effort at algebraic types], it is checked with isinstance. If it is None, it is checked with `is`. If it is any other object [maybe want a way to do truthy/falsy checks?] compare with ==. Or an explicit comparison operator can be used, or two along with a ?-placeholder (a <= ? < b). What do you think?

Andrew Barnert

11:55 a.m.

On Jan 2, 2020, at 06:32, Random832 wrote:

...

Mainly, you had mentioned you felt like "if try" had a 'smell' to it but couldn't figure out what it is, and I was trying help identify that, and... well, after thinking about it more, I realized - "if try" may not have many uses, but "if not try" has a million: any code of the form

try: x except: y

where x is an expression could be written as

if not try x: y

And therein lies the problem - it's a construct for catching exceptions with no provision for declaring what kind of exception is caught. You may have only envisioned it for ValueError, but it'd be weird for something that *looks* so general to be limited in that way.

I think you’re right. What I’m after is something like Swift’s “if let”, but with Python’s assignment semantics (which don’t deal with Optional unpacking but with Iterable unpacking, and which raise a ValueError on failure. Spelling it “if try” seemed good enough, and it works without adding a new keyword or giving it a completely new syntactic form that isn’t allowed elsewhere. But it looks like it isn’t good enough, and I don’t think there’s any other combination of existing keywords that could spell “if this works”. I’ll give it a little more thought, but it’s probably not going to pan out. Oh well.

...

Incidentally, it turns out that unpacking with the := operator is syntactically problematic-- (a, b := 1, 2) is currently equivalent to (a, (b := 1), 2). Without the outer parentheses it's a syntax error both in an if statement and otherwise, but I'd find it extremely surprising for adding parentheses to radically change the meaning in this way if both forms are valid.

Yeah, you’d need two sets of parens for this case, which is pretty ugly, and possibly a bug magnet.

...

I don't want to hijack your thread, but I've thought of something that may be an alternative. An "as" expression that returns a boolean [and so could be used as-is with if/while] could be useful for this and for the cases other people mentioned (ranges, type checking without necessarily requiring an awkward "type[...]" construct, etc)

This is kind of backward from the way most other languages do things, but it might work. After all, an as clause is already effectively a backward assignment, and it reads nicely.

...

Consider something like if (x, y, z) := vec as ?, 0, ?: # if y == 0

Even though the ? makes sense there, it doesn’t really feel like it fits in Python syntax somehow. And I think the bar for using up one of our two remaining ASCII symbol characters is probably even higher than adding a new keyword. Also, while it’s obvious how this could be extended with a guard clause later, could it be extended with an as clause (to bind a subpattern while decomposing and further matching it at the same time)? Like “match vec as (x, y, z) and z as (zreal, zcomplex) and y and zcomplex both must be 0”, which in a more typical syntax you could write as something like `case x, 0, (zreal, 0) as z`.

...

or simply if tup as ?, <0 # if len(tup) == 2 and tup[1] < 0

If an element in the comparison list is ?, it would not be compared at all. If it is a type [perhaps any object with an __instancecheck__, to allow ABCs and perhaps some future effort at algebraic types], it is checked with isinstance. If it is None, it is checked with `is`. If it is any other object [maybe want a way to do truthy/falsy checks?] compare with ==. Or an explicit comparison operator can be used, or two along with a ?-placeholder (a <= ? < b).

I think allowing <0 as an item is too magical. In a language where operator sectioning was already a valid syntactic construct (e.g., in Haskell it defines the functions lambda x: x<0, although you generally have to use it in parens because of their rules about converting between operators and functions) that might be different, in Python it looks like a typo. Or like a comparison between the tuple (x,) and 0. (I’m not sure whether that’s ambiguous to the parser or not, but to a human, once I realized you didn’t mean ?<0 or y<0 or something else that might make sense even if it isn’t legal or useful there, single-value tuple would be my best thought.) More generally, I think pattern matching proposals that try to throw in everything under the sun as syntax end up with a lot of weird syntax and still only covering half of what you need. For example, C# had to decide whether they wanted “is X” to be an exact type check or a subclass one, and then when they decided they needed the other one too they had to go back and add new syntax. If you use something extensible, like call syntax, that isn’t a problem—you have SubTypeMatch(T), and later you can add ExactTypeMatch(T). And, if the comparison is actually done with operator==, this doesn’t even require anything new; the SubTypeMatch and ExactTypeMatch and RegexMatch and ComparisonMatch and FunctionMatch and so on are all just in a library of classes that provide custom __eq__ methods. (And different libraries could live on PyPI to evolve and compete on faster than 18-month scales, instead of just having the feature set effectively frozen forever.) But I think that fits into your syntax as well as it does into mine: if tup as ?, Negative(): if tup as ?, MatchFunc(lambda y: y<0): if tup as ?, (_1 < 0): And then, given that your as syntax isn’t doing any binding, it’s almost just == between two tuples. Of course then you couldn’t use ? for don’t care values, but that could be in the library too, as AnyMatch(): if tup == Any(), Negative(): … which works in Python today. Except, of course, if tup has more or fewer than 2 elements, this isn’t a fail but a raise ValueError. Which takes us right back to where I started. But maybe that’s all your syntax has to do: `a as b` is truthy if a==b, falsey otherwise *including on ValueError*, and… that’s the whole thing. (And maybe it compares any two iterables instead of requiring the left one to be a tuple? But to destructure anything beyond namedtuple instance for binding, you’re going to need some library, and requiring that the library always destructure into a tuple might be fine.) This still doesn’t get you destructuring _binding_ without also extending the walrus operator, but maybe Python (like JavaScript) really doesn’t need that part. In almost all cases where this makes sense: if (x, _, z) := vec as ?, 0, ?: … you could just use vec.x and vec.z. You can’t do that in Swift, Scala, etc.: if your Vec3 value is an Optional[Vec3] or a Vec2|Vec3 or whatever variable, those types don’t have a .x attribute, so you need to “cast” the value into a Vec3 variable. In Python, your Vec3 value is a Vec3 value and therefore it has a .x attribute. One last thing here. The paradigm vector-matching case from other languages is actually more like this: case Vec3(x, 0, z): … which checks that your match value is a Vec3, and can be deconstructed as a Vec3 into x, 0, z. I was planning on breaking that out with a library function as (type(x),) + astuple(x) so you’d match the result like this: m = Matching(vec) if try (Vec3, x, 0, z) := m: I’m not sure if that still fits with your design, but I think it could.

...

What do you think?

I think it could be worth following up on. If you plan to do so, I could dump my incomplete notes for the other syntax on you so you can see which bits are worth stealing, if you want.

Dan Sommers

12:22 p.m.

On 1/2/20 12:55 PM, Andrew Barnert via Python-ideas wrote:

...

On Jan 2, 2020, at 06:32, Random832 wrote:

...

...
Mainly, you had mentioned you felt like "if try" had a 'smell' to it but couldn't figure out what it is, and I was trying help identify that, and... well, after thinking about it more, I realized - "if try" may not have many uses, but "if not try" has a million: any code of the form

try: x except: y

where x is an expression could be written as

if not try x: y

And therein lies the problem - it's a construct for catching exceptions with no provision for declaring what kind of exception is caught. You may have only envisioned it for ValueError, but it'd be weird for something that *looks* so general to be limited in that way.

[...]

...

But it looks like it isn’t good enough, and I don’t think there’s any other combination of existing keywords that could spell “if this works”. I’ll give it a little more thought, but it’s probably not going to pan out. Oh well.

What about "if except" (any time I can eliminate a "not," that's a good thing): if except x: y which could even leave room to specify which exception(s) is/are caught: if except ValueError: x: y except that the two colons on one line look a little odd in Python.

Random832

2:07 p.m.

On Thu, Jan 2, 2020, at 13:22, Dan Sommers wrote:

...

What about "if except" (any time I can eliminate a "not," that's a good thing):

wait, is this meant to be for the doesn't-throw case or does-throw? how are we eliminating the not?

...

if except x: y

which could even leave room to specify which exception(s) is/are caught:

if except ValueError: x: y

except that the two colons on one line look a little odd in Python.

if x except ValueError?

Dan Sommers

8:32 p.m.

On 1/2/20 3:07 PM, Random832 wrote:

...

On Thu, Jan 2, 2020, at 13:22, Dan Sommers wrote:

...

...
What about "if except" (any time I can eliminate a "not," that's a good thing):

... is this meant to be for the doesn't-throw case or does-throw? ...

...
if except x: y

That would be the does-throw case. Read it as "if there's an exception evaluating x, then do the following."

...

... how are we eliminating the not?

I changed "not try" to "except," thus eliminating the "not."

...

...
which could even leave room to specify which exception(s) is/are caught:

if except ValueError: x: y

except that the two colons on one line look a little odd in Python.

if x except ValueError?

Andrew Barnert

3 Jan 3 Jan

11:12 a.m.

...

On Jan 2, 2020, at 18:34, Dan Sommers <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:

On 1/2/20 3:07 PM, Random832 wrote:

...
On Thu, Jan 2, 2020, at 13:22, Dan Sommers wrote:

...
...
What about "if except" (any time I can eliminate a "not," that's a good thing): ... is this meant to be for the doesn't-throw case or does-throw? ... if except x: y

That would be the does-throw case. Read it as "if there's an exception evaluating x, then do the following."

But then you have the opposite of what you need. You want “if it matches this, do that with these bindings”, not “if it doesn’t match this, do that (with these failed bindings)”. Also, you can’t chain these up and keep trying patterns until one succeeds, or default if all of them fail; instead you can only chain them up and keep trying patterns until one fails, or default if they all succeed. So, this syntax might be useful for other things, but not for pattern matching.

...

...
... how are we eliminating the not?

I changed "not try" to "except," thus eliminating the "not."

But there’s no “not try”, just a “try”. We want to run code if the try condition succeeds, which is positive, not negative. Or, if you prefer, it’s a double negative, and you’ve removed one of the negatives: if try (x, 0, z) := vec: xz_stuff(x, z) … would be roughly equivalent to: try: x, 0, z = vec except ValueError. pass else: xz_stuff(x, z) What you’ve given is a way to put code in the one place where we didn’t want any (but had to put a “pass” just for syntactic reasons) without putting code in the place we did.

Neil Girdhar

4:56 p.m.

This is very cool, well thought-out, and solves an important problem. I would have definitely benefited from Python having better pattern matching. As an idealist though, if you're going to add pattern matching, why not just do it completely? I agree that your proposal would be useful in some cases, but then what happens when someone wants to go just a step farther? For example, def parse(text): if try "(", *a, ")" = text[i, i+n]: # how do you set n? do you try them all? parse(a) # etc. and even more complicated: def parse(text): if try "\begin{", *a, "}", *b, "\end{", *a, "}" = ... # how do you ensure that the two a's match? how do you ensure that a can only consist of letter characters? I feel like it would be more idealistic to find a powerful, expressive way to specify parsers. (As far as I know, no Python parser library can specify expression yet because they don't execute code on their "rules"). If we had such a parsing library or extension, then we could compare whether it's overkill or reasonable to use that library in situations that you're proposing to address with "if try". Best, Neil On Friday, January 3, 2020 at 12:15:42 PM UTC-5, Andrew Barnert via Python-ideas wrote:

...

...
On Jan 2, 2020, at 18:34, Dan Sommers <2QdxY4Rz...@potatochowder.com javascript:> wrote:

On 1/2/20 3:07 PM, Random832 wrote:

...
On Thu, Jan 2, 2020, at 13:22, Dan Sommers wrote:

...
...
What about "if except" (any time I can eliminate a "not," that's a good thing): ... is this meant to be for the doesn't-throw case or does-throw? ... if except x: y

That would be the does-throw case. Read it as "if there's an exception evaluating x, then do the following."

But then you have the opposite of what you need. You want “if it matches this, do that with these bindings”, not “if it doesn’t match this, do that (with these failed bindings)”.

Also, you can’t chain these up and keep trying patterns until one succeeds, or default if all of them fail; instead you can only chain them up and keep trying patterns until one fails, or default if they all succeed.

So, this syntax might be useful for other things, but not for pattern matching.

...
...
... how are we eliminating the not?

I changed "not try" to "except," thus eliminating the "not."

But there’s no “not try”, just a “try”. We want to run code if the try condition succeeds, which is positive, not negative.

Or, if you prefer, it’s a double negative, and you’ve removed one of the negatives:

if try (x, 0, z) := vec: xz_stuff(x, z)

… would be roughly equivalent to:

try: x, 0, z = vec except ValueError. pass else: xz_stuff(x, z)

What you’ve given is a way to put code in the one place where we didn’t want any (but had to put a “pass” just for syntactic reasons) without putting code in the place we did.

_______________________________________________ Python-ideas mailing list -- python...@python.org javascript: To unsubscribe send an email to python-id...@python.org javascript: https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5HOJJF... Code of Conduct: http://python.org/psf/codeofconduct/

Steven D'Aprano

1 Jan 1 Jan

6:20 a.m.

On Tue, Dec 31, 2019 at 05:18:59PM -0800, Andrew Barnert via Python-ideas wrote:

...

Some languages use special syntax to mark either values or targets:

let x, K, let z = vec x, @K, z = vec

But the simplest solution is to nothing: you have to stick it in an expression that isn’t a valid target, or It’s a target. And I think that might actually work. If the pattern matching library includes this (or you write it yourself):

def val(x): return x

… then you just write this:

x, val(K), z = vec

Which doesn’t seem too bad.

That is a bug-magnet. It's such a powerful bug magnet, that the mere suggestion caused me to commit three bugs in a script I was writing *yesterday*, that's how powerful it is. But seriously, and all hyperbole aside, that does seem too bad to me. It's a syntactic construct which will sucker the user (i.e. me) into a sense of false security, then bite them when they least expect it. Match constructs seem to work: # constants are fine spam, 42, eggs = obj # function calls and expressions are fine spam, foo(arg), eggs = obj spam, myconstant or fallback, eggs = obj and then you write this: spam, db['key'], eggs = obj and the feces hits the rotary air circulating machine. I'd almost rather have a rule that says you can only use literals for the pattern matching than syntax which is so prone to laying landmines.

...

And notice that any object with a custom __eq__ will be matched by calling that, so your library can have a decorator that turns an arbitrary function into a matcher

I'm sorry, I don't understand what that means. Can you give an example? [...]

...

...
At that point you might be better off with a dedicated "switch" or "case" construct.

Sure, if we’re willing to use up one or two new keywords and design a whole new syntax for them we can obviously make it do whatever we want.

If something is worth doing, it's worth doing right. -- Steven

Andrew Barnert

6:06 p.m.

...

...
On Jan 1, 2020, at 04:21, Steven D'Aprano wrote:

On Tue, Dec 31, 2019 at 05:18:59PM -0800, Andrew Barnert via Python-ideas wrote:

Some languages use special syntax to mark either values or targets: let x, K, let z = vec x, @K, z = vec But the simplest solution is to nothing: you have to stick it in an expression that isn’t a valid target, or It’s a target. And I think that might actually work. If the pattern matching library includes this (or you write it yourself): def val(x): return x … then you just write this: x, val(K), z = vec Which doesn’t seem too bad.

That is a bug-magnet. It's such a powerful bug magnet, that the mere suggestion caused me to commit three bugs in a script I was writing *yesterday*, that's how powerful it is.

But seriously, and all hyperbole aside, that does seem too bad to me. It's a syntactic construct which will sucker the user (i.e. me) into a sense of false security, then bite them when they least expect it. Match constructs seem to work:

# constants are fine spam, 42, eggs = obj

# function calls and expressions are fine spam, foo(arg), eggs = obj spam, myconstant or fallback, eggs = obj

and then you write this:

spam, db['key'], eggs = obj

and the feces hits the rotary air circulating machine.

I'd almost rather have a rule that says you can only use literals for the pattern matching than syntax which is so prone to laying landmines.

The first thing people ask for is “How do I match against a value in a local variable?” (As Greg Ewing and someone else did on this thread.) It actually isn’t needed as often as people who don’t do a lot of pattern matching suspect, but it is certainly needed sometimes, so not having any way to allow it would be a serious flaw. However, there are multiple ways to handle this. Adding special syntax that’s always required for marking values works. There’s some precedent for prefix @ for values, and someone already suggested it on this thread: `x, @K, v = …` matches the value of K, and there’s no possibility of ambiguity. Anything without an @ is a target (and if it’s not a valid target, it’s a syntax error). Requiring marking the other way works too, but I don’t think it fits into Python as well. (Typically you use something based on the declaration/binding syntax, but that’s implicit in Python…) Loosening this up to allow literals to be values unmarked, but not other values, works. (Although then you need to work out which definition of “literal” you want. Does -3 have to get marked because it’s actually an expression? What about a list display? And so on. All of these things can be answered, and bikeshedded.) Loosening it to allow calls to be values without being marked already means you can get rid of the marking syntax entirely, by just having a wrapper function to serve as the marker. Loosening it all the way to anything that isn’t syntactically a target seemed like a good idea to me, mainly because it’s the simplest possible rule to keep in your head once you’ve learned it. But it isn’t a necessary part of any proposal. Something that requires more val() calls or even @ marks will still look fundamentally similar.

...

...
And notice that any object with a custom __eq__ will be matched by calling that, so your library can have a decorator that turns an arbitrary function into a matcher

I'm sorry, I don't understand what that means. Can you give an example?

I already gave multiple examples of custom marchers. I’ll repeat the shortest one here; if you want other examples (including a wrapper that turns any function into a matcher), read the thread. David Mertz asked for a way to match a vector only if the first value is an int: # In the library class MatchSubclass(type): def __eq__(self, other): return isinstance(other, self) # In user code MatchSubclass(int), y, z = vec

...

[...]

...
...
At that point you might be better off with a dedicated "switch" or "case" construct. Sure, if we’re willing to use up one or two new keywords and design a whole new syntax for them we can obviously make it do whatever we want.

If something is worth doing, it's worth doing right.

Sure, but who says switch and case is “right”, much less weird doubly-indented things that are not statements despite ending in colons and that (unlike except, else, etc.) don’t line up with the start of the statement? That was the best anyone had come up with in the 70s, but plenty of newer languages (Swift, Scala, C#, etc.) build their pattern matching out of a match primitive that’s closer to Python’s Iterable unpacking than it is to SML or C, and I think they’re better languages for it. So if Python could do the same thing, why shouldn’t we explore it? It may turn out not to work, but that doesn’t imply that ignoring the last 4 decades of language design because you already know it’s doable the 70s way is a good answer. Meanwhile, that wouldn’t actually solve the problem you’re focusing on. A case statement still needs a way to both match values and destructure to bindings. If that way isn’t at all related to unpacking, that means two similar syntaxes for similar purposes that you need to learn. If it is, then you need to resolve the ambiguity, and you’ve got the same problem with the same possible solutions that we’re already dealing with.

Steven D'Aprano

9:02 a.m.

On Tue, Dec 31, 2019 at 02:28:26PM -0800, Andrew Barnert via Python-ideas wrote:

...

if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!')

Comments below.

...

Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif.

Have you read the exception catching PEP? https://www.python.org/dev/peps/pep-0463/

...

Anyway, neither of these features seems very useful on its own. (It should be obvious how to rewrite that example as something just as readable that just unpacks and then checks the values with normal if.)

Indeed. The ordinary "if" version is two lines shorter, although you lose the chance to provide a custom exception message.

...

But I think the two of them together will allow a pure-library implementation of pattern matching syntax that reads nicely and can do everything most other languages do—in particular, concisely and readably pattern match and deconstruct dataclasses (and other types with an opt-in protocol or registry, but dataclasses would work out of the box the way Scala case classes, Swift structs, etc. do).

I don't know about "reads nicely". I really like the F# pattern matching syntax, even if it would require two new keywords (match, when) and an arrow symbol (F# uses -> but Python could use a colon). That reads nicely to me (and also supports guard clauses, although that's not needed in your example): # F# syntax match vec with | 0, y, z -> yz_expression | x, 0, z -> xz_expression | x, y, 0 -> xy_expression | x, y, z -> xyz_expression | _ -> fail In F# matching is an expression. I don't know if that's an advantage or disadvantage. I find the F# explanation for what pattern matching is all about *much* more understandable than the Haskell version -- even the "Gentle Introduction". https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/pattern-ma... https://www.haskell.org/tutorial/patterns.html Also relevant: Coconut has pattern matching: https://marnee.silvrback.com/a-bit-of-railway-oriented-programming-with-coco... https://coconut.readthedocs.io/en/master/DOCS.html#match We don't need anything new to get prototype pattern matching. Patterns are first class values in SNOBOL (and maybe in Icon?). Using that as inspiration, we could create Pattern objects to do the work, and a match function that we call like this: # Fail() is a function that raises an exception; # __ is a special predefined Pattern that always matches. result = match(vec, # object to match Pattern(0, 'y', 'z'), handle_yz, Pattern('x', 0, 'z'), handle_xz, Pattern('x', 'y', 0), handle_xy, Pattern('x', 'y', 'z'), handle_xyz, __, Fail('not a vector') ) The match() function would simply pass `vec` to each Pattern object in turn, until one matches, then calls the handle_* callback function with the variables stored in the pattern object after a successful match. def match(value, *args): if len(args)%2 != 0: raise TypeError for pattern, callback in pairs(args): if pattern.match(value): callback(**pattern.vars()) The implementation of match() is trivial: all the pattern matching goodies are in the Pattern object: - if the match succeeds, the Pattern object stores the matched variables ('x', 'y', 'z' in this case) and returns itself; - if the match fails, it returns None. - pattern.vars() returns a dict with the matched variables. The pros: no new syntax required! Not even the walrus operator. This could, in principle, work all the way back to Python 1.5 if you were so inclined. But the cons... * it's tediously verbose * it's a nuisance having to pre-define the functions handle_* ahead of time (in Ruby, you would just pass a block) * but then the same applies for the old "use a dict dispatch table in as a switch/case" idiom * having to quote the names in the Pattern constructor is a pain. But as a prototype, I think I've seen worse ideas. -- Steven

Dan Sommers

9:17 a.m.

On 1/1/20 10:02 AM, Steven D'Aprano wrote:

...

# Fail() is a function that raises an exception; # __ is a special predefined Pattern that always matches.

result = match(vec, # object to match Pattern(0, 'y', 'z'), handle_yz, Pattern('x', 0, 'z'), handle_xz, Pattern('x', 'y', 0), handle_xy, Pattern('x', 'y', 'z'), handle_xyz, __, Fail('not a vector') )

...

But the cons...

...

* having to quote the names in the Pattern constructor is a pain.

And it means that you can't match a string directly?

Andrew Barnert

7 p.m.

On Jan 1, 2020, at 07:03, Steven D'Aprano wrote:

...

On Tue, Dec 31, 2019 at 02:28:26PM -0800, Andrew Barnert via Python-ideas wrote:

...
if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!')

Comments below.

...
Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif.

Have you read the exception catching PEP?

https://www.python.org/dev/peps/pep-0463/

Yes. It’s a red herring here—except for the fact they’re incompatible proposals (I believe an if-try statement and an if statement with a try-except condition would be impossible to disambiguate without look ahead), so this proposal would mean that rejected PEP couldn’t be revived. (Which I think is a bit of a shame, because I liked PEP 463, but it was rejected, and nobody’s come up with any good counters to the arguments against it in the succeeding half decade, so…)

...

I really like the F# pattern matching syntax, even if it would require two new keywords (match, when) and an arrow symbol (F# uses -> but Python could use a colon). That reads nicely to me (and also supports guard clauses, although that's not needed in your example):

# F# syntax match vec with | 0, y, z -> yz_expression | x, 0, z -> xz_expression | x, y, 0 -> xy_expression | x, y, z -> xyz_expression | _ -> fail

F# is similar to most other ML-family languages here. I’ve looked at them, but also at Scala, Swift, Haskell, C#, and other languages. But I don’t think we want to just copy any one of them. While I like the bar syntax used by a lot of ML derivatives, I don’t think it fits nicely with Python. You have a compound statement that doesn’t end with a colon or have an indebted suite, a symbol that acts much like an INDENT token but not quite and that can only be disambiguated contextually, …

...

In F# matching is an expression. I don't know if that's an advantage or disadvantage.

Well, it’s an advantage in the same way that making _all_ statements into expressions is an advantage. Python benefits from making if, for, def, class, etc. into statements and having a rigid statement syntax (which uses indentation and other things to aid readability). It does give up a bit of expressiveness for doing so, but it’s generally worth it. And then, for a small subset of them, it adds a handful of explicitly restricted forms (ternary expression, comprehension, lambda) that are useful for special cases where you very commonly have something short and simple that is worth doing in the middle of an expression. But when you don’t have something short and simple that needs to be in the middle of an expression (or you do, but it isn’t close enough to one of the three super-common cases to have a concise form), you use the more powerful statement form. I’m pretty sure the same would be true for pattern matching. (Also note that F# matches are let expressions in disguise. You can introduce a new binding (or rebind an existing one), except by evaluating an expression inside a let that chains in front of the current environment. And that restricts the way you can design pattern matching in ML-family languages.

...

I find the F# explanation for what pattern matching is all about *much* more understandable than the Haskell version -- even the "Gentle Introduction".

Yeah, Haskell’s gentle introduction isn’t all that gentle about almost anything. If you want to learn the core features of the whole language (and are interested in how to use what features and why, rather than how to mathematically model each feature), Learn You a Haskell for Great Good is a lot better. But it’s not very good for learning a single feature in isolation. I don’t actually know of any Haskell resource that’s good for that. If you want to compare some other languages, maybe look at Scala and Swift before Haskell.

...

We don't need anything new to get prototype pattern matching.

Patterns are first class values in SNOBOL (and maybe in Icon?).

String patterns aren’t really the same thing as deconstructing patterns. String patterns are first class values in Python too with the re module or a third-party module of your choice. They even have dedicated syntax in Ruby and Perl.

...

Using that as inspiration, we could create Pattern objects to do the work, and a match function that we call like this:

# Fail() is a function that raises an exception; # __ is a special predefined Pattern that always matches.

result = match(vec, # object to match Pattern(0, 'y', 'z'), handle_yz, Pattern('x', 0, 'z'), handle_xz, Pattern('x', 'y', 0), handle_xy, Pattern('x', 'y', 'z'), handle_xyz, __, Fail('not a vector') )

Yes, and there are already multiple libraries on PyPI that do effectively this. And by the same token, we don’t need if statements or if exoressions, because you can do that with a function: def if_(cond, truefunc, falsefunc): return [falsefunc, truefunc][bool(cond)]() Now, instead of this: if spam: print('spam!') else: print(f'spam is falsey: {spam}') … you can write this: if_(spam, lambda: print('spam'), lambda: print(f'spam is falsey: {spam}')) And, because expressions don’t care about whitespace, you can indent that however you like. And you can do the same for for statements—but you don’t have to, because that’s already a builtin, map. Syntax for these things is just sugar, but it’s useful sugar. Being able to refer to variables as variables rather than passing around their names as strings, and to refer to other targets as well, and to be able to write bodies as statement suites inline instead of having to wrap them up in functions, and to access and even rebind locals, and so on—these are all useful for if, and for for, and for pattern matching, for the exact same reasons. And I believe all the cons of doing things the Lisp way for pattern matching are exactly way nobody uses those existing libraries.

David Mertz

3 Jan 3 Jan

5:14 p.m.

There have been a number of comments in this thread, and some slight variations in the idea. I'm afraid that it all looks awkward to me, so far. I understand trying to work out the smallest change that would allow the style, but it just feels "bolted on" in a bad way. I believe that if Python gets pattern matching, it should either be full F#-ish syntax *OR* be done as a library with a mini-language (which is not actual Python syntax) Would it be so bad to put something like the F# syntax in tripple quotes, or perhaps in some nested data structure, then process it with a function call? At least as something that could live on PyPI that could try to gain users and see how much reaction the style gets. On Tue, Dec 31, 2019, 5:30 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:

...

Every so often, someone suggests that Python should have a pattern matching case statement like Scala, C#, Swift, Haskell, etc. Or they just suggest that “Python needs algebraic data types” but end up concluding that the biggest part Python is missing is not the types themselves, but a practical way to do something useful with them, which basically means pattern matching.

Anyway, with the addition of the walrus operator and @dataclass, Python is actually a lot closer than it used to be. I believe with just two smallish syntax features, it would be possible to do almost anything you’d ever want in a library.

The first is to extend unpacking assignment to target-or-expression lists. Like this:

x, 0, z = vec

Just like `x, y, z = vec`, this raises a TypeError if vec isn’t Iterable, and a ValueError if it has more or fewer than 3 elements, and otherwise, it binds x and z to the first and third elements. But it doesn’t bind anything to the second element; instead, it checks if that element is 0, and, if not, raises a ValueError.

The second is an “if try” statement, which tries an expression and runs the body if that doesn’t raise (instead of if it’s truthy), but jumps to the next elif/else/statement (swallowing the exception) if it does. The actual value of the expression is discarded, but the walrus operator takes care of that:

if try 0, y, z := vec: # do yz plane stuff elif try x, 0, z := vec: # do xz plane stuff elif try x, y, 0 := vec: # do xy plane stuff elif x, y, z := vec: # do slow/imprecise/whatever 3D stuff else: raise TypeError(f'{vec} is not a 3-vector!')

Alternatively, this could just be a try expression that can be used anywhere: it’s truthy if evaluating doesn’t raise, falsey if it does. But I don’t think it’s needed anywhere but if/elif.

Anyway, neither of these features seems very useful on its own. (It should be obvious how to rewrite that example as something just as readable that just unpacks and then checks the values with normal if.)

But I think the two of them together will allow a pure-library implementation of pattern matching syntax that reads nicely and can do everything most other languages do—in particular, concisely and readably pattern match and deconstruct dataclasses (and other types with an opt-in protocol or registry, but dataclasses would work out of the box the way Scala case classes, Swift structs, etc. do).

If people think either of these features is too horrible to contemplate no matter what the potential benefits, I won’t bother working through the details.

But if people do want to see the details (including some good motivating examples—I’ll go through tutorials for those other languages to find some that don’t involve recursing into cons lists and other stuff you wouldn’t want to do in Python…), I’ll be happy to do so, and, if it works out, turn this into a more detailed proposal.

And obviously, if people want to bikeshed the spelling before even seeing what it’s good for, well, this is Python-ideas. :) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W64QFN... Code of Conduct: http://python.org/psf/codeofconduct/

1572

Age (days ago)

1575

Last active (days ago)

List overview

Download

28 comments

9 participants

participants (9)

Andrew Barnert
Dan Sommers
David Mertz
Greg Ewing
Guido van Rossum
Neil Girdhar
Random832
Soni L.
Steven D'Aprano

Target-and-expression lists and if-try

tags

participants (9)