Thoughts on PEP 634 (Structural Pattern Matching)
Hi everyone,
PEP 634/5/6 presents a possible implementation of pattern matching for Python.
Much of the discussion around PEP 634, and PEP 622 before it, seems to imply that PEP 634 is synonymous with pattern matching; that if you reject PEP 634 then you are rejecting pattern matching.
That simply isn't true.
Can we discuss whether we want pattern matching in Python and the broader semantics first, before dealing with low level details?
Do we want pattern matching in Python at all? ---------------------------------------------
Pattern matching works really well in statically typed, functional languages.
The lack of mutability, constrained scope and the ability of the compiler to distinguish let variables from constants means that pattern matching code has fewer errors, and can be compiled efficiently.
Pattern matching works less well in dynamically-typed, functional languages and statically-typed, procedural languages. Nevertheless, it works well enough for it to be a popular feature in both erlang and rust.
In dynamically-typed, procedural languages, however, it is not clear (at least not to me) that it works well enough to be worthwhile.
That is not say that pattern matching could never be of value in Python, but PEP 635 fails to demonstrate that it can (although it does a better job than PEP 622).
Should match be an expression, or a statement? ----------------------------------------------
Do we want a fancy switch statement, or a powerful expression? Expressions have the advantage of not leaking (like comprehensions in Python 3), but statements are easier to work with.
Can pattern matching make it clear what is assigned? ----------------------------------------------------
Embedding the variables to be assigned into a pattern, makes the pattern concise, but requires discarding normal Python syntax and inventing a new sub-language. Could we make patterns fit Python better?
Is it possible to make assignment to variables clear, and unambiguous, and allow the use of symbolic constants at the same time? I think it is, but PEP 634 fails to do this.
How should pattern matching be integrated with the object model? ----------------------------------------------------------------
What special method(s) should be added? How and when should they be called? PEP 634 largely disregards the object model, meaning it has many special cases, and is inefficient.
The semantics must be well defined. -----------------------------------
Language extensions PEPs should define the semantics of those extensions. For example, PEP 343 and PEP 380 both did. https://www.python.org/dev/peps/pep-0343/#specification-the-with-statement https://www.python.org/dev/peps/pep-0380/#formal-semantics
PEP 634 just waves its hands and talks about undefined behavior, which horrifies me.
In summary, I would ask anyone who wants pattern matching adding to Python, to not support PEP 634. PEP 634 just isn't a good fit for Python, and we deserve something better.
Cheers, Mark.
Can we discuss whether we want pattern matching in Python and the broader semantics first, before dealing with low level details?
This is a huge step backward. These discussions have already taken place, over the last 10 years.
Here's just a sampling:
- https://mail.python.org/archives/list/python-ideas@python.org/thread/IEJFUSF... - https://mail.python.org/archives/list/python-ideas@python.org/thread/GTRRJHU... - https://mail.python.org/archives/list/python-ideas@python.org/thread/EURSG3M... - https://mail.python.org/archives/list/python-ideas@python.org/thread/NTQEL3H... - https://mail.python.org/archives/list/python-ideas@python.org/thread/NEC54II... - https://mail.python.org/archives/list/python-ideas@python.org/thread/T3VBUFE...
We read all of these and more back way in March, before we even started brainstorming syntax and semantics.
Do we want a fancy switch statement, or a powerful expression?
It's right here that you lose me. Anyone who reduces pattern matching to "a fancy switch statement" probably isn't the right person to be discussing its semantics and usefulness with. It seems that some people just can't separate the two ideas in their mind. It's like calling a class a "fancy module".
It's okay that you feel that way, but hopefully you'll understand if people start to tune out messages that contain these sorts of comments.
What special method(s) should be added?
None. PEP 622 originally added one, but even that is more than we need right now. Some people may need to register their mappings or sequences as Mappings or Sequences, but otherwise that's it.
I would ask anyone who wants pattern matching adding to Python, to not support PEP 634.
Seriously?
I would ask anyone who wants pattern matching added to Python to carefully consider the PEPs for themselves (particularly PEP 636, which is much less dry and contains more examples and commentary). We've written four of the largest, most detailed PEPs of any new feature I've seen, complete with a working implementation that we've made available from any browser. Of course it's not the *only* way of getting pattern matching... but if you want it, this is probably your *best* shot at getting it.
Brandt
Hi Brandt,
On 30/10/2020 4:09 pm, Brandt Bucher wrote:
Can we discuss whether we want pattern matching in Python and the broader semantics first, before dealing with low level details?
This is a huge step backward. These discussions have already taken place, over the last 10 years.
So what you are saying is that I'm not allowed to voice my opinions, because it is outside a time frame of your choosing?
Here's just a sampling:
- https://mail.python.org/archives/list/python-ideas@python.org/thread/IEJFUSF...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/GTRRJHU...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/EURSG3M...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/NTQEL3H...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/NEC54II...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/T3VBUFE...
We read all of these and more back way in March, before we even started brainstorming syntax and semantics.
Do we want a fancy switch statement, or a powerful expression?
It's right here that you lose me. Anyone who reduces pattern matching to "a fancy switch statement" probably isn't the right person to be discussing its semantics and usefulness with. It seems that some people just can't separate the two ideas in their mind. It's like calling a class a "fancy module".
Pattern matching is a fancy switch statement, if you define "fancy" appropriately ;)
Reducing pattern matching to some sort of switch statement is exactly what a good implementation should do. It's what compilers are for. The comparison seems entirely reasonable to me.
OOI, what is the reasoning for choosing a statement, not an expression?
It's okay that you feel that way, but hopefully you'll understand if people start to tune out messages that contain these sorts of comments.
What does "these sorts of comments" mean? Ones that you disagree with?
If I am wrong, please explain why in an as objective a fashion as possible.
What special method(s) should be added?
None. PEP 622 originally added one, but even that is more than we need right now. Some people may need to register their mappings or sequences as Mappings or Sequences, but otherwise that's it.
Much of the language uses special methods. Why should pattern matching be so different?
Why make this opt-out, rather than opt-in, given the potential for unwanted side effects?
I would ask anyone who wants pattern matching adding to Python, to not support PEP 634.
Seriously?
Yes. Absolutely. PEP 634 is seriously flawed.
I would ask anyone who wants pattern matching added to Python to carefully consider the PEPs for themselves (particularly PEP 636, which is much less dry and contains more examples and commentary). We've written four of the largest, most detailed PEPs of any new feature I've seen, complete with a working implementation that we've made available from any browser. Of course it's not the *only* way of getting pattern matching... but if you want it, this is probably your *best* shot at getting it.
Given the size of the proposed change to the language, it really isn't that detailed.
The browser based implementation is nice, though :)
Cheers, Mark.
Brandt _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DEUZMFMT... Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, 30 Oct 2020 at 18:30, Mark Shannon mark@hotpy.org wrote:
Hi Brandt,
On 30/10/2020 4:09 pm, Brandt Bucher wrote:
Can we discuss whether we want pattern matching in Python and the broader semantics first, before dealing with low level details?
This is a huge step backward. These discussions have already taken place, over the last 10 years.
So what you are saying is that I'm not allowed to voice my opinions, because it is outside a time frame of your choosing?
I certainly wouldn't say that, but I do think that if you're not taking into account the extended history of ideas and of people expressing a desire for some kind of pattern matching in Python, then you shouldn't be surprised if people consider your objections insufficiently well-informed.
Here's just a sampling:
- https://mail.python.org/archives/list/python-ideas@python.org/thread/IEJFUSF...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/GTRRJHU...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/EURSG3M...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/NTQEL3H...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/NEC54II...
- https://mail.python.org/archives/list/python-ideas@python.org/thread/T3VBUFE...
We read all of these and more back way in March, before we even started brainstorming syntax and semantics.
Do we want a fancy switch statement, or a powerful expression?
It's right here that you lose me. Anyone who reduces pattern matching to "a fancy switch statement" probably isn't the right person to be discussing its semantics and usefulness with. It seems that some people just can't separate the two ideas in their mind. It's like calling a class a "fancy module".
Pattern matching is a fancy switch statement, if you define "fancy" appropriately ;)
Everything is a goto if you want to play reductionist games. In practice, though, the proposed match statement feels more like the constructs in recent languages like Rust than like the C switch statement.
Reducing pattern matching to some sort of switch statement is exactly what a good implementation should do. It's what compilers are for. The comparison seems entirely reasonable to me.
Yes, the implementation might do this. But the *semantics* are what matter here, and they are firmly in the area of "how developers should think about the construct". Anyone thinking about pattern matching with the C switch statement in mind is going to miss a huge amount of the power and expressiveness of the construct.
OOI, what is the reasoning for choosing a statement, not an expression?
It's okay that you feel that way, but hopefully you'll understand if people start to tune out messages that contain these sorts of comments.
What does "these sorts of comments" mean? Ones that you disagree with?
Personally, I'm likely to tune out messages that seem to be ignoring or dismissing big chunks of the discussion. Not "because I disagree", but because there's no option for discussion or compromise if you don't address the arguments already made in favour of pattern matching.
To give one example, you say "Can we discuss whether we want pattern matching in Python and the broader semantics first". I'd say yes we do. There are many examples in the discussions so far on the PEPs (and plenty more in the 10 years of history that Brandt mentioned) where people have said they like how a given piece of code looks as a match statement. You may not agree, but language features don't have to be something that *everyone* likes, as long as there's sufficient value to a sufficiently large user base to offset the costs of the feature.
Can you provide a counter-argument, or are you just asking that we go round that discussion again, with no new information to add?
If I am wrong, please explain why in an as objective a fashion as possible.
You are wrong that there is no support for pattern matching. See the number of posts in support. You are wrong that pattern matching is "just a fancy switch" - see examples in the PEP of many things a C switch can't do (destructuring, for a start). If you want to claim that's just "fancy", then I'd ask for a clarification of how far you consider "fancy" to go, and why you consider a sufficiently fancy switch to be useless based on an assumption that a non-fancy switch would be. Hopefully that's a start towards objectively explaining why I consider that you are wrong.
What special method(s) should be added?
None. PEP 622 originally added one, but even that is more than we need right now. Some people may need to register their mappings or sequences as Mappings or Sequences, but otherwise that's it.
Much of the language uses special methods. Why should pattern matching be so different?
Why make this opt-out, rather than opt-in, given the potential for unwanted side effects?
I would ask anyone who wants pattern matching adding to Python, to not support PEP 634.
Seriously?
Yes. Absolutely. PEP 634 is seriously flawed.
I'm not going to make a decision like that just because you say so. I have considered the PEP and am basically in favour of it. If you're telling me I've made an incorrect judgement, then surely the onus is on you to offer me additional data to convince me to change my mind, not simply to state "PEP 634 is seriously flawed" without backing that up. (Or at least, without backing it up any more than you have done in previous posts on the matter, which I have read and so you can assume didn't convince me).
I guess that's my explanation for why I'm likely to tune out your posts on this. I'm not finding anything new in them, I'm afraid.
Paul
On Sat, Oct 31, 2020 at 5:31 AM Mark Shannon mark@hotpy.org wrote:
It's right here that you lose me. Anyone who reduces pattern matching to "a fancy switch statement" probably isn't the right person to be discussing its semantics and usefulness with. It seems that some people just can't separate the two ideas in their mind. It's like calling a class a "fancy module".
Pattern matching is a fancy switch statement, if you define "fancy" appropriately ;)
Reducing pattern matching to some sort of switch statement is exactly what a good implementation should do. It's what compilers are for. The comparison seems entirely reasonable to me.
"What compilers are for" is turning useful high level constructs into low level executable forms. I don't think anyone would say that a for loop is just a fancy GOTO, even though it gets compiled into one.
I don't care how the compiler represents something; I care how I'll be using it. A regular expression isn't, to me, a state machine - it's a way to describe a text pattern.
It's okay that you feel that way, but hopefully you'll understand if people start to tune out messages that contain these sorts of comments.
What does "these sorts of comments" mean? Ones that you disagree with?
If I am wrong, please explain why in an as objective a fashion as possible.
Considering how subjective your posts in this thread are, there's some irony in begging for the complaint to be objective.
I would ask anyone who wants pattern matching adding to Python, to not support PEP 634.
Seriously?
Yes. Absolutely. PEP 634 is seriously flawed.
FUD. You haven't given a single reason for it to be considered flawed, you've just said that it 'is' flawed, as though we should already know what's wrong with it.
I don't draw this parallel lightly, but Mark, you're actually sounding scarily similar to jmf and his anti-PEP393 posts. Extremely light on actual arguments, and entirely based on emotive complaints and an assumption that your readers already know why this is somehow a bad thing.
If you want PEP 634 to be rejected, explain why. Don't try to get us to go back over the past thousands of posts (never mind the ones stretching back ten years even).
ChrisA
On 31/10/20 7:22 am, Mark Shannon wrote:
On 30/10/2020 4:09 pm, Brandt Bucher wrote:
Anyone who reduces pattern matching to "a fancy switch statement" probably isn't the right person to be discussing its semantics and usefulness with.
Pattern matching is a fancy switch statement, if you define "fancy" appropriately ;)
I think Brandt's point is that it sounded as though you were making a derogatory remark about the statement option, and that people are likely to stop listening to someone who does that kind of thing.
OOI, what is the reasoning for choosing a statement, not an expression?
Which is a *much* better way to say what I think you were trying to say.
On Sat, 31 Oct 2020 at 00:44, Mark Shannon mark@hotpy.org wrote:
Should match be an expression, or a statement?
Do we want a fancy switch statement, or a powerful expression? Expressions have the advantage of not leaking (like comprehensions in Python 3), but statements are easier to work with.
PEP 635 already includes a good rationale for using a statement in https://www.python.org/dev/peps/pep-0635/#the-match-statement
Can pattern matching make it clear what is assigned?
Embedding the variables to be assigned into a pattern, makes the pattern concise, but requires discarding normal Python syntax and inventing a new sub-language. Could we make patterns fit Python better?
Is it possible to make assignment to variables clear, and unambiguous, and allow the use of symbolic constants at the same time? I think it is, but PEP 634 fails to do this.
This concern I agree with, and just posted PEP 642 as a proposal to address it by requiring an explicit syntactic prefix on any value lookups that occur in a match pattern.
How should pattern matching be integrated with the object model?
What special method(s) should be added? How and when should they be called? PEP 634 largely disregards the object model, meaning it has many special cases, and is inefficient.
PEP 642 eliminates at least some of the special cases. For the special cases that remain, I think we'll be OK to tackle generalisation as a "later" concern (similar to the way we lived without a Path-like protocol for a long time to allow objects other than strings in filesystem APIs).
(As Brandt noted, the early iterations of PEP 622 did include a match customisation protocol, and the authors were persuaded to drop it and adopt a "Let's wait and see how far we get with just `__match_args__`, `__eq__`, and `ABC.register()` before adding anything else" approach instead)
The semantics must be well defined.
Language extensions PEPs should define the semantics of those extensions. For example, PEP 343 and PEP 380 both did. https://www.python.org/dev/peps/pep-0343/#specification-the-with-statement https://www.python.org/dev/peps/pep-0380/#formal-semantics
PEP 634 just waves its hands and talks about undefined behavior, which horrifies me.
Is this in relation to the evaluation-order caveat, and the impact of using non-pure operations inside patterns?
Even for with statements, we've tinkered with the exact timing of the __enter__ and __exit__ method lookups over the years in order to tweak exactly which exception gets thrown when both are missing (the original implementation followed the PEP in looking up "__exit__" first, as it used multiple CPython bytecodes, and wanted to be able to use "__enter__" immediately after loading it on to the stack. The current implementation instead looks up "__enter__" first, since that's the error users expect to see when both are missing, and we now have a dedicated SETUP_WITH opcode that made the change simple to implement).
The problem with providing that level of specificity in PEP 634 is that fully mandating a specific evaluation order for all subpatterns and guard subexpressions means that potentially interesting optimisation opportunities for match statement evaluation could be ruled out.
Even if the initial pattern matching implementation uses a straightforward top to bottom, left to right approach where each pattern is checked for a structural match, re-evaluating repeated subexpressions each time they appear, then its associated guard expression evaluated, it makes sense to allow for more clever implementations that aggressively cache repeated subexpressions in the structural matching patterns (which are supposed to be side-effect free, even though we have no way to enforce that restriction), and inspect the guard expressions for subclauses that can be evaluated early and potentially skip the full structural match if the guard clause becomes necessarily false (while abiding by the requirement that the guard expressions still follow normal left-to-right evaluation rules - the subexpression evaluations are just allowed to be interleaved with the evaluation of subpatterns in the structural matching part by a sufficiently capable compiler).
Cheers, Nick.
I'm late, but I still wanted to add that I share some of the criticism that Mark has brought up.
I'm in love with Rust's pattern matching, so I feel like I'm not against pattern matching generally. However I feel like while the PEP is well written, there are some things that it does not tackle:
- Sum Types aka Tagged Unions are what makes pattern matching necessary. I think we should rather have a discussion about inclusion of Sum Types. - Match in Python would not be an expression, which would be the most useful application in Python IMO. - It's quite a lot of additional syntax and complexity for a feature that does not enable something fundamental.
I feel like in my 10 years of Python I have rarely wished to have pattern matching. I would have wished to have Sum Types, because that would have enabled me to structure my code in a more readable way. Python enums are unfortunately not as useful and dataclasses/namedtuples don't feel right either.
I also think that using isinstance is usually a sign of code smell and I'm trying to avoid it whenever I can (except for `assert isinstance`). Many of the examples in PEP 635 (Motivation & Rationale) I would consider to be a bit smelly. The examples about tree traversal are useful, but it's exactly where I wished I had Sum Types to make my code more readable. Pattern matching wouldn't even be necessary in these cases, because something like Rust's `if let ...` would be good enough.
Since I'm obviously quite late for this discussion, I can understand if this comment is ignored. I also understand that not everyone can be happy with the final decision about pattern matching. I just think that this is not the best addition. Thanks anyway to the people working on this and continue to have a civil discussion.
Happy New Year & Stay Safe! ~ Dave
Am Fr., 30. Okt. 2020 um 15:43 Uhr schrieb Mark Shannon mark@hotpy.org:
Hi everyone,
PEP 634/5/6 presents a possible implementation of pattern matching for Python.
Much of the discussion around PEP 634, and PEP 622 before it, seems to imply that PEP 634 is synonymous with pattern matching; that if you reject PEP 634 then you are rejecting pattern matching.
That simply isn't true.
Can we discuss whether we want pattern matching in Python and the broader semantics first, before dealing with low level details?
Do we want pattern matching in Python at all?
Pattern matching works really well in statically typed, functional languages.
The lack of mutability, constrained scope and the ability of the compiler to distinguish let variables from constants means that pattern matching code has fewer errors, and can be compiled efficiently.
Pattern matching works less well in dynamically-typed, functional languages and statically-typed, procedural languages. Nevertheless, it works well enough for it to be a popular feature in both erlang and rust.
In dynamically-typed, procedural languages, however, it is not clear (at least not to me) that it works well enough to be worthwhile.
That is not say that pattern matching could never be of value in Python, but PEP 635 fails to demonstrate that it can (although it does a better job than PEP 622).
Should match be an expression, or a statement?
Do we want a fancy switch statement, or a powerful expression? Expressions have the advantage of not leaking (like comprehensions in Python 3), but statements are easier to work with.
Can pattern matching make it clear what is assigned?
Embedding the variables to be assigned into a pattern, makes the pattern concise, but requires discarding normal Python syntax and inventing a new sub-language. Could we make patterns fit Python better?
Is it possible to make assignment to variables clear, and unambiguous, and allow the use of symbolic constants at the same time? I think it is, but PEP 634 fails to do this.
How should pattern matching be integrated with the object model?
What special method(s) should be added? How and when should they be called? PEP 634 largely disregards the object model, meaning it has many special cases, and is inefficient.
The semantics must be well defined.
Language extensions PEPs should define the semantics of those extensions. For example, PEP 343 and PEP 380 both did. https://www.python.org/dev/peps/pep-0343/#specification-the-with-statement https://www.python.org/dev/peps/pep-0380/#formal-semantics
PEP 634 just waves its hands and talks about undefined behavior, which horrifies me.
In summary, I would ask anyone who wants pattern matching adding to Python, to not support PEP 634. PEP 634 just isn't a good fit for Python, and we deserve something better.
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MJ4RQ3R5... Code of Conduct: http://python.org/psf/codeofconduct/
Hello,
On Sun, 27 Dec 2020 14:10:59 +0100 Dave Halter davidhalter88@gmail.com wrote:
I'm late, but I still wanted to add that I share some of the criticism that Mark has brought up.
I'm in love with Rust's pattern matching, so I feel like I'm not against pattern matching generally. However I feel like while the PEP is well written, there are some things that it does not tackle:
- Sum Types aka Tagged Unions are what makes pattern matching
necessary. I think we should rather have a discussion about inclusion of Sum Types.
Python had sum types (well, superset of them) since ~forever (just like any other object-oriented language). Your typical Haskell sum datatype:
data Tree a = Leaf a | Branch (Tree a) (Tree a)
directly translates to:
class Tree: pass class Leaf(Tree): def __init__(self, val): ... class Branch(Tree): def __init__(self, l, r): ...
"Leaf" and "Branch" are the tags you're looking for.
Recent dataclasses cut on the amount of boilerplate required.
участники (8)
-
Brandt Bucher -
Chris Angelico -
Dave Halter -
Greg Ewing -
Mark Shannon -
Nick Coghlan -
Paul Moore -
Paul Sokolovsky