Hi Guido, On 16/11/2020 4:41 pm, Guido van Rossum wrote:

Thanks Mark, this is a helpful and valuable contribution.

I will try to understand and review it in the coming weeks (there is no hurry since the decision is up to the next SC) but I encourage you to just put it in PEP form and check it into the PEP repo.

Because I only skimmed very briefly, I don't have an answer to one question: does your PEP also define a precise mapping from the PEP 634 syntax to your "desugared" syntax? I think that ought to be part of your PEP.

No it doesn't define a precise mapping, and I don't think it will. I'm not familiar enough with ever corner of PEP 634 to do that. I could add a general "how to" guide though. It fairly straightforward conceptually but, as you know, the devil is in the details. Cheers, Mark.

--Guido

On Mon, Nov 16, 2020 at 6:41 AM Mark Shannon mailto:mark@hotpy.org> wrote:

Hi everyone,

There has been much discussion on the syntax of pattern matching for Python (in case you hadn't noticed ;)

Unfortunately the semantics seem to have been somewhat overlooked. What pattern matching actually does seems at least as important as the syntax.

I believe that a pattern matching implementation must have the following properties:

* The semantics must be precisely defined. * It must be implemented efficiently. * Failed matches must not pollute the enclosing namespace. * Objects should be able determine which patterns they match. * It should be able to handle erroneous patterns, beyond just syntax errors.

PEP 634 and PEP 642 don't have *any* of these properties.

I've written up a document to specify a possible semantics of pattern matching for Python that has the above properties, and includes reasons why they are necessary.

https://github.com/markshannon/pattern-matching/blob/master/precise_semantic...

It's in the format of a PEP, but it isn't a complete PEP as it lacks surface syntax.

Please, let me know what you think.

Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org mailto:python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org mailto:python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BTPODVYL... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido http://python.org/~guido) /Pronouns: he/him //(why is my pronoun here?)/ http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On Wed, Nov 18, 2020 at 7:04 PM Daniel Moisset wrote:

[sorry for the duplicate, meant to reply-all]

Thank you for this approach, I find it really helpful to put the conversation in these terms (semantics and guiding principles).

It does help to rationalise discussion ;-)

[...] 4. "Objects should be able determine which patterns they match." This is something that you and I, and most of the authors of 622 agree on. What we found out when discussing this is that we didn't have clear what and how to open that customization. Some customization options added a lot of complexity at the cost of performance, some others were very simple but it wasn't clear that they would be actually useful, or extensible in the future. This has a lot to do with this being a somewhat new paradigm in Python, and our lack of knowledge on what the user community may do with it beyond what we imagined. So the decision was "pattern matching as it is presented without extensibility is useful, let's get this in, and once we see how it is used in the wild we'll understand better what kind of extensibility is valuable". For a crude analogy, imagine trying to get the descriptor protocol right when the basic python object model was being implemented. These things happened as different times as the use of the language evolved, and my opinion is that customization of the match protocol must follow a similar path.

I haven't so far understood whether the change proposed is intended to start out in __future__. Given the preliminary nature of this development, it would seem wise to retain the option to withdraw it or modify it radically before production code depends on it.

On Thu, Nov 19, 2020 at 8:08 PM Brett Cannon wrote:

On Thu, Nov 19, 2020 at 5:16 AM Steve Holden wrote:

...

On Wed, Nov 18, 2020 at 7:04 PM Daniel Moisset wrote:

...

[sorry for the duplicate, meant to reply-all]

Thank you for this approach, I find it really helpful to put the conversation in these terms (semantics and guiding principles).

It does help to rationalise discussion ;-)

...

[...] 4. "Objects should be able determine which patterns they match." This is something that you and I, and most of the authors of 622 agree on. What we found out when discussing this is that we didn't have clear what and how to open that customization. Some customization options added a lot of complexity at the cost of performance, some others were very simple but it wasn't clear that they would be actually useful, or extensible in the future. This has a lot to do with this being a somewhat new paradigm in Python, and our lack of knowledge on what the user community may do with it beyond what we imagined. So the decision was "pattern matching as it is presented without extensibility is useful, let's get this in, and once we see how it is used in the wild we'll understand better what kind of extensibility is valuable". For a crude analogy, imagine trying to get the descriptor protocol right when the basic python object model was being implemented. These things happened as different times as the use of the language evolved, and my opinion is that customization of the match protocol must follow a similar path.

I haven't so far understood whether the change proposed is intended to start out in __future__. Given the preliminary nature of this development, it would seem wise to retain the option to withdraw it or modify it radically before production code depends on it.

The current proposal is not for it to be put behind a __future__ statement as it doesn't conflict with or break any pre-existing source code.

Ok, thanks for the clarification. All I will say is just because we aren't *required* to implement it in __future__ that doesn't mean we can't or shouldn't. Everything should be done to underline the tentative nature of these developments, or we risk the potential of functionality frozen because "we're already using it in production." The discussion to date suggests that such a breathing space would be useful.

Fair enough. On Fri, Nov 20, 2020 at 11:45 AM Thomas Wouters wrote:

On Fri, Nov 20, 2020 at 8:38 AM Steve Holden wrote:

...

On Thu, Nov 19, 2020 at 8:08 PM Brett Cannon wrote:

All I will say is just because we aren't *required* to implement it in __future__ that doesn't mean we can't or shouldn't. Everything should be done to underline the tentative nature of these developments, or we risk the potential of functionality frozen because "we're already using it in production."

On the other hand, __future__ imports have never been provisional, and it was never the *intent *of __future__ imports to be provisional. PEP 236, which introduced __future__ imports after a vigorous debate on backward compatibility, explicitly states "fully backward- compatible additions can-- and should --be introduced without a corresponding future_statement". I think it would be very surprising if a feature introduced under a future import turned out to go away rather than become the default.

-- Thomas Wouters

Hi! I'm an email virus! Think twice before sending your email to help me spread!

On 20/11/2020 6:57 pm, Tobias Kohn wrote:

Hi Daniel and Mark,

Sorry for being slightly late to the party, but please let me add a few remarks of my own to the discussion here.

*1. Must have precisely defined semantics*

Yes, there are some aspects that we left open intentionally.  Most

Why? Daniel has written up a formal semantics, why not use that or something like it?

prominently the question of how often the pattern matching engine will check whether the subject is an instance of a particular class.  Take the following trivial example::

  match some_data:       case Pair(12, 34):           ...       case Triple(12, 34, z):           ...       case Pair(12, y):           ...       case Pair(x, y):           ...

In a perfect world, the compiler discovers that it must check whether ``some_data`` is an instance of ``Pair`` exactly once and not three times.  This, of course, plays right into Mark's second point on efficiency and seems obvious enough.  Yet, as soon as we are considering nested patterns, it turns much less obvious whether the compiler is supposed to cache repeated isinstance-checks.  Can we really expect that the compiler must discover that in both case clauses the first element is checked against the same class?  Or would it make more sense to simply expect the compiler to potentially perform this ``Num`` instance check twice::

  match some_data:       case [ Num(), 12 ]:           ...       case [ Num(), y, *z ]:           ...

It is easy to think of cases where we accidentally end up calling an isinstance check more than once because the compiler could not prove that they are equal.  Still, whenever possible we want to give the compiler the freedom to optimise the pattern matching statement by caching.

An optimization (in the CS sense, not the math sense) is a performance-improving, semantic-preserving transformation. How can I preserve semantics if you won't tell me what the semantics are?

In a static language, all of this would not be an issue at all, of course.  In Python, however, we end up being caught between its dynamic features and the desire to make pattern matching reasonably efficient. So, we ended up leaving the question open as how often the pattern matching engine is allowed or supposed to check instances.  Naturally, if you go and write some isinstance-check on a class with side-effects, you can break it.

*2. Users should not have to pay an unnecessary performance penalty to use pattern matching*

To quote Mark [1] here:

/> Users should not have to pay an unnecessary performance penalty to use pattern matching./

Alright, what does this even mean?  What is an unnecessary performance penalty?  How should that be measured or compared?

If we shipped Python compiled with -O1, instead of -O3. It would be slower and unnecessarily so. That would be an unnecessary performance penalty. In this context, I mean that there is an obvious performance improvement available and it hasn't been implemented. Or that the specification prevents an optimization without any compensating benefit. Without a well defined semantics, I can't optimize at all. That is an unnecessary performance penalty.

Pattern matching is not just fancy syntax for an if-elif-statement, but a new way of writing and expressing structure.  There is currently nothing in Python that fully compares to pattern matching (which is obviously why we propose to add in the first place).  So, do you want to compare a pattern matching structure to an if-elif-chain or rather an implementation using reflection and/or the visitor pattern?  When implementing pattern matching, would we be allowed to trade off a little speed handling the first pattern for moving faster to patterns further down?

Do our PEPs really read to you like we went out of our ways to make it slow or inefficient?  Sure, we said let's start with an implementation that is correct and worry about optimising it later.  But I thought this is 101 of software engineering, anyway, and am thus rather surprised to find this item on the list.

*3. Failed matches should not pollute the enclosing namespace*

This is a slightly wider issue that has obviously sparked an entire discussion on this mailing list on scopes.

If there is a good solution that only assigns variables once the entire pattern matched, I would be very happy with that.  However, I think that variables should be assigned in full before evaluating any guards---even at the risk of the guard failing and variables being assigned that are not used later on.  Anything else would obviously introduce a mini-scope and lead to shadowing, which hardly improves anything with respect to legibility.

How would it introduce shadowing? Since `var` is a capture pattern, it can't read and thus can't be shadowed.

*4. Objects should be able determine which patterns they match*

Short version: no!

Yes! ;)

Class patterns are an extension of instance checks.  Leaving out the meta-classes at this point, it is basically the class that is responsible for determining if an object is an instance of it.  Pattern matching follows the same logic, whereas Mark suggests to put that upside-down.  Since you certainly do not want to define the machinery in each instance, you end up delegating the entire thing to the class, anyway.

"Mark suggests to put that upside-down" I have no idea what you mean by that, or the the rest of this paragraph.

I find this suggestion also somewhat strange in light of the history of our PEPs.  We started with a more complex protocol that would allow for customised patterns, which was then ditched because it was felt as being too complicated.  There is still a possibility to add it later on, of course.  But here we are with Mark proposing to introduce a complex protocol again.  It would obviously also mean that we could not rely as much on Python's existing infrastructure, which makes efficient pattern matching harder, again.  I completely fail to see what should be gained by this.

"But here we are with Mark proposing to introduce a complex protocol again". Again? Please list the actual faults you see with whatever it is you are seeing faults with. I really don't know what you are getting at here. If you are criticizing my proposal in https://github.com/markshannon/pattern-matching/blob/master/precise_semantic... then address that, please.

*5. It should distinguish between a failed match and an erroneous pattern*

This seems like a reasonable idea.  However, I do not think it is compatible to Python's existing culture.  Let's pick up Mark's example of an object ``RemoteCount`` with two attributes ``success`` and ``total``.  You can then execute the following line in Python without getting any issues from the interpreter::

Python has loads of runtime type-checking. If something is clearly an error, then why not report it?

...

...

"" + 0 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can only concatenate str (not "int") to str

  my_remote_count.count = 3

Python does not discover that the attribute should have been ``total`` rather than ``count`` here.  From a software engineering perspective, this is unfortunate and I would be surprised if there was anyone on this list who was never bitten by this.  But this is one of the prices we have to pay for Python's elegance in other aspects.  Requiring that pattern matching suddenly solves this is just not realistic.

*6. Syntax and Semantics*

There are some rather strange elements in this, such as the idea that the OR-pattern should be avoided.  In the matching process, you are also talking about matching an expression (under point 2), for instance; you might not really be aware of the issues of allowing expressions in patterns in the first place.

Please don't assume that people who disagree with you don't understand something or are ignorant.

---

It is most certainly a good idea to start with guiding principles to then design and build a new feature like pattern matching. Incidentally, this is what we actually did before going into the details of our proposal.  As evidenced by extensive (!) documentation on our part, there is also a vision behind our proposal for pattern matching.

In my view, Mark's proposal completely fails to provide a vision or any rationale for the guiding principles, other than reference to some mysterious "user" who "should" or "should not" do certain things. Furthermore, there are various obvious holes and imprecisions that would have to be addressed.

You've just addressed my guiding principles, but here they again: * The semantics of pattern matching must be precisely defined. * It must be implemented efficiently. That is, it should perform at least as well as an equivalent sequence of if, elif statements. * Failed matches should not pollute the enclosing namespace. * Objects should be able determine which patterns they match. * It should distinguish, at much as possible, between a failed match and an erroneous pattern. Where the guiding principles for PEP 634? Cheers, Mark.

Kind regards, Tobias

[1] https://github.com/markshannon/pattern-matching/blob/master/precise_semantic...

Quoting Daniel Moisset mailto:dfmoisset@gmail.com>:

...

[sorry for the duplicate, meant to reply-all] Thank you for this approach, I find it really helpful to put the conversation in these terms (semantics and guiding principles). This is not an answer to the proposal (which I've read and helps me contextualize) but to your points below and how they apply to PEP-634. I'm also answering personally, with a reasonable guess about what the other authors of 634-636 would agree, but they may correct me if I'm wrong.

On Mon, 16 Nov 2020 at 14:44, Mark Shannon mailto:mark@hotpy.org> wrote:

(...) I believe that a pattern matching implementation must have the following properties:

* The semantics must be precisely defined. * It must be implemented efficiently. * Failed matches must not pollute the enclosing namespace. * Objects should be able determine which patterns they match. * It should be able to handle erroneous patterns, beyond just syntax errors.

PEP 634 and PEP 642 don't have *any* of these properties.

Let me answer this one by one: 1. "The semantics must be precisely defined": If this happens in PEP634 I don't think it was intentional, and I'm pretty sure the authors would be happy to complete any incompleteness that it has. I would happily have a more accurate description (I drafted a non-official one for a much earlier version of PEP-622, https://github.com/dmoisset/notebook/blob/master/python/pep622/semantic-spec... ). Can you clarify where you see these imprecisions? 2. "It must be implemented efficiently": I don't think "efficient implementation" was a priority in PEP634, although I saw your proposal defines this as "same performance as the equivalent if statement", and I'm quite sure that level of performance can be achieved (if it isn't already by Brandt's implementation). Finding the best way to optimise wasn't a priority, but I think if there was anything in our implementation that would make optimisations harder we would consider them as a change. Do you think anything like that has been presented? 3. "Failed matches must not pollute the enclosing namespace": This for me is one of the less-desirable parts of the proposal, and was agreed more as a matter of practicality and an engineering tradeoff. If you have a reasonable way of solving this (like putting matched variables in the stack and popping it later) reasonably I'd be much happier putting that in. 4. "Objects should be able determine which patterns they match." This is something that you and I, and most of the authors of 622 agree on. What we found out when discussing this is that we didn't have clear what and how to open that customization. Some customization options added a lot of complexity at the cost of performance, some others were very simple but it wasn't clear that they would be actually useful, or extensible in the future. This has a lot to do with this being a somewhat new paradigm in Python, and our lack of knowledge on what the user community may do with it beyond what we imagined. So the decision was "pattern matching as it is presented without extensibility is useful, let's get this in, and once we see how it is used in the wild we'll understand better what kind of extensibility is valuable". For a crude analogy, imagine trying to get the descriptor protocol right when the basic python object model was being implemented. These things happened as different times as the use of the language evolved, and my opinion is that customization of the match protocol must follow a similar path. 5. "It should be able to handle erroneous patterns, beyond just syntax errors." I'll be answering this based on the example in your document, matching RemoteCount(success=True, count=count) where RemoteCount is a namedtuple. The code is likely an error, and I'm in general all for reporting errors early, but the kind of error detection proposed here is the kind of errors that python normally ignore. I find that example really similar to the kind of error you could make writing "if remcount.count == 3: ..." or going beyond this example "maxelems = configfile.read(); if len(elems) == maxelems: ...". There are many type errors that python ignores (especially related to equality), and Python has already made the decision of allowing mixed type equality comparisons everywhere, so why do you think pattern matching should be different with respect to this? In my opinion trying to "fix" this (or even agreeing if this is a bug or not) is a much more general issue unrelated to pattern matching. Given the current status-quo I normally trust python type checkers to help me with these errors, and I'd expect them to do the same with the "erroneous" match statement. If there are other examples you had in mind when you wrote this I'd also be happy to discuss those. I'll try to get some time to review your specific counterproposal later, thanks for it. Best,     Daniel

_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SIK62L2A... Code of Conduct: http://python.org/psf/codeofconduct/

Hi Daniel, On 20/11/2020 10:50 am, Daniel Moisset wrote:

Hello again Mark, I took some time looking in more detail at your proposal, and these are my thoughts. I'm writing with the assumption that this proposal describes some "internal" representation of match statements which is never exposed to the users (so I'd mostly steer away from lexical/syntactic preferences).

My first general thought is that this is a useful way to describe and discuss implementation, although I wouldn't wait on refinement of this to choose whether to accept/reject PEP 634 (or 642, or your favourite alternative), work can be done on parallel. It may be a good idea to wait for your proposal to be refined before landing a specific implementation into CPython (including how the chosen implementation, assuming it's accepted, desugars into your semantics).

Going into more details, here's a list of thoughts on more specific points:

1. You mention a goal about "erroneous patterns" (which I'm still not sure I agree on), and your proposal addresses that by forcing classes to be explicit (via __atributes__ and __deconstruct__) about what attributes are accepted as matches. This is against one design principle that's not in your list but it was (at least implicitly) in PEP622 and 634: "pattern matching must allow matching objects on types not specifically designed for it"; this will allow to apply this feature to classes that you can not modify (like instances created by a 3rd party library ). That's why PEP634 relies on getattr() ; that could be extended in the feature (providing some override using attributes like yours) but that wouldn't be required by default

Why force pattern matching onto library code that was not designed for pattern matching? It seems risky. Fishing arbitrary attributes out of an object and assuming that the values returned by attribute lookup are equivalent to the internal structure breaks abstraction and data-hiding. An object's API may consist of methods only. Pulling arbitrary attributes out of that object may have all sorts of unintended side-effects. PEP 634 and the DLS paper assert that deconstruction, by accessing attributes of an object, is the opposite of construction. This assertion seems false in OOP. You might consider OOP as a hangover of the 1990s and that FP is the way forward, and you may well be right, but the vast majority of Python libraries have a distinctly OO flavor, and we need to respect that. When we added the "with" statement, there was no attempt to force existing code to support it. We made the standard library support it, and let the community add support as and when it suited them. We should do the same with pattern matching.

2. We considered and discussed something similar to the __deconstruct__ approach (as an override for getattr()), and also noted is potentially expensive, requiring to allocate an object and evaluate *all* attributes, even if only one is required. That is against the principle of " it should perform at least as well as an equivalent sequence of if, elif statements."

The majority of cases will want more than one attribute, and evaluating all attributes at once may well be more efficient than doing so one at a time, especially for builtin classes. I assure you that this is not going to be a performance bottleneck until the implementation is a lot slicker than the PEP 634 "reference implementation".

3. You removed or patterns, and added multiple "case P" for each case body. This easily covers cases where the multiple options are at top level in the pattern, but if the or-pattern is in a subpattern you have to duplicate much of the outer context. And this "duplication" is actually exponential on the number of or patterns, so for matching the pattern "[0|1, 0|1, 0|1, 'foo'|'bar'|'baz']" you need to desugar this into 24 case clauses.

Why is this a problem? The compiler can handle millions of cases without it being a problem. The number of tests to distinguish N cases is log(N), so the number of tests remains the same.

4. Something else about decomposing patterns into multiple case clauses is that it makes it harder to express the constraint that all alternatives must bind the same variable.

Why? It is a mechanical transformation.

5. I found a bit confusing to read the multiple cases on top. It looks like C switch statements, which fall-through by default, but in a context where some case clauses do fall-through (if empty) and others aren't (if they have a body, even if it's "pass"). I know this is not user exposed syntax so it's not that important, but this made the description harder to read for me.

You'll just have to let go of your C-based preconceptions ;)

6. Given the previous points, added to not seeing the gains of not putting the or patterns into the desugared version, I'd prefer it to be included in the desugaring. A key point of the syntax is that if something looks like a Python expression, then it is a Python expression. That is hard to do with a "pattern or" operator embedded in the patterns. That `0|1` means something completely different to `0+1` in PEP 634 seems like an unnecessary trap.

7. I think there's some surprising behaviour in the assignments being done after a successful match but before the guard is evaluated. In your proposal the guard has no access to the variables, so it has to be compiled differently (using $0, $1, ... rather than the actual names that appear in the expression). And if this guard calls a function which exposes those variables in any way (for example if the variable is in a closure) I think the behaviour may be unexpected /surprising; same if I stop a debugger inside that function and try to inspect the frame where the matching statement is.

All the "$0, $1" variables have a very limited scope. They're not visible outside of that scope, so they won't be visible to the debugger at all. Modifying variables during matching, as you describe, is a serious flaw in PEP 634/642. You don't need a debugger for them to have surprising behavior, failed matches can change global state in an unspecified way. Worse than that, PEP 634 comes close to blaming the user for any unwanted side-effects. https://www.python.org/dev/peps/pep-0634/#side-effects-and-undefined-behavio...

8. I like your implementation approach to capture on the stack and then assign. I was curious if you considered, rather than using a variable number of stack cells using a single object/dict to store those values. The compiler and the generated bytecode could end up being simpler, and you need less stack juggling and possibly no PEEK operation. a small list/array would suffice, but a dict may provide further debugging opportunities (and it's likely that a split table dict could make the representation quite compact). I know this is less performant but I'm also thinking of simplicity.

Implement it how you please, as long as it's correct, maintainable, and not too slow :)

9. I think your optimisation approaches are great, the spec was made lax expecting for people like you to come up with a proposal of this kind :) I don't think the first implementation of this should be required to optimize/implement things in a certain way, but if the spec is turned into implementation dependent and then fixed, it shouldn't break anything (it's like the change in dictionary order moving to "undefined/arbitrary" to "preserving insertion order") and can be done later one

I think it is important that *all* implementations, including the first, respect the exact semantics as defined. The first implementation should be reasonably efficient, but doesn't have to be super quick. Cheers, Mark.

Thanks again,

Daniel

On Mon, 16 Nov 2020 at 14:44, Mark Shannon mailto:mark@hotpy.org> wrote:

Hi everyone,

There has been much discussion on the syntax of pattern matching for Python (in case you hadn't noticed ;)

Unfortunately the semantics seem to have been somewhat overlooked. What pattern matching actually does seems at least as important as the syntax.

I believe that a pattern matching implementation must have the following properties:

* The semantics must be precisely defined. * It must be implemented efficiently. * Failed matches must not pollute the enclosing namespace. * Objects should be able determine which patterns they match. * It should be able to handle erroneous patterns, beyond just syntax errors.

PEP 634 and PEP 642 don't have *any* of these properties.

I've written up a document to specify a possible semantics of pattern matching for Python that has the above properties, and includes reasons why they are necessary.

https://github.com/markshannon/pattern-matching/blob/master/precise_semantic...

It's in the format of a PEP, but it isn't a complete PEP as it lacks surface syntax.

Please, let me know what you think.

Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org mailto:python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org mailto:python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BTPODVYL... Code of Conduct: http://python.org/psf/codeofconduct/

On Fri, Nov 20, 2020 at 02:23:45PM +0000, Mark Shannon wrote:

Why force pattern matching onto library code that was not designed for pattern matching? It seems risky.

Can you give a concrete example of how this will "force" pattern matching onto library code? I don't think that anyone has suggested that we go around to third-party libraries and insert pattern matching in them, so I'm having trouble understanding your fear here.

Fishing arbitrary attributes out of an object and assuming that the values returned by attribute lookup are equivalent to the internal structure breaks abstraction and data-hiding.

Again, can we have a concrete example of what you fear? Python is not really big on data-hiding. It's pretty close to impossible to hide data in anything written in pure Python.

An object's API may consist of methods only. Pulling arbitrary attributes out of that object may have all sorts of unintended side-effects.

Well, sure, but merely calling print() on an object might have all sorts of unintended side-effects. I think that almost the only operation guaranteed to be provably side-effect free in Python is the `is` operator. So I'm not sure what you fear here? If I have a case like: match obj: case Spam(eggs=x): I presume any sensible implementation is going to short-cut the attempted pattern match for x if obj is *not* an instance of Spam. So it's not going to be attempting to pull out arbitrary attributes of arbitrary objects, but only specific attributes of Spam objects. To the degree that your objection here has any validity at all, surely it has been true since Python 1.5 or older that we can pull arbitrary attributes out of unknown objects? That's what duck-typing does, whether you guard it with a LBYL call to hasattr or an EAFP try...except block. if hasattr(obj, 'eggs'): result = obj.eggs + 1 Not only could obj.eggs have side-effects, but so could the call to hasattr. Can you explain how pattern matching is worse than what we already do?

PEP 634 and the DLS paper assert that deconstruction, by accessing attributes of an object, is the opposite of construction. This assertion seems false in OOP.

Okay. Does it matter? Clearly Spam(a=1, b=2) does not necessarily result in an instance with attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be equivalent to (roughly): if (instance(obj, Spam) and getattr(obj, a) == 1 and getattr(obj, b) == 2) it doesn't imply that obj was *literally* created by a call to the constructor `Spam(a=1, b=2)`, or even that this call would be possible. I think that it will certainly be true that for many objects, there is a very close (possibly even exact) correspondence between the constructor parameters and the instance attributes, i.e. deconstruction via attribute access is the opposite of construction. But for the exceptions, why does it matter that they are exceptions? Let me be concrete for the sake of those who may not be following these abstract arguments. Suppose I have a class: class Car: def __init__(self, brand, model): self.brand = brand self.model = model and an instance: obj = Car("Suzuki", "Swift") For this class, deconstruction by attribute access is exactly the opposite of construction, and I can match any Suzuki like this: match obj: case Car(brand="Suzuki", model) which is roughly equivalent to: if isinstance(obj, Car) and getattr(obj, "brand") == "Suzuki": model = getattr(obj, "model") It's not actually asserting that the instance *was* constructed with a call to `Car(brand="Suzuki", model="Swift")`, only that for the purposes of deconstruction it might as well have been. If the constructor changes, leaving the internal structure the same: class Car: def __init__(self, manufacturer, variant): self.brand = manufacturer self.model = variant the case statement need not change. Remember that non-underscore attributes are public in Python, so a change to the internal structure: class Car: def __init__(self, brand, model): self.brand_name = brand self.model_id = model is already a breaking change, whether we have pattern matching or not.

When we added the "with" statement, there was no attempt to force existing code to support it. We made the standard library support it, and let the community add support as and when it suited them.

We should do the same with pattern matching.

That's a terrible analogy. Pattern matching is sugar for things that we can already do: - isinstance - getattr - sequence unpacking - equality - dict key testing etc. Pattern matching doesn't "force" objects to support anything they don't already support. As far as I can tell, the only thing that the community will need to add support for is the mapping between positional attributes and attribute names (the `__match_args__` protocol). **Everything** else needed by pattern matching is already supported. [...]

That `0|1` means something completely different to `0+1` in PEP 634 seems like an unnecessary trap.

If so, it's a trap we have managed since the earliest days of Python: x in sequence # It's a bool expression. for x in sequence: ....^^^^^^^^^^^^^ Syntax that looks exactly like an expression, but means something completely different in the context of a for loop. [...]

Modifying variables during matching, as you describe, is a serious flaw in PEP 634/642. You don't need a debugger for them to have surprising behavior, failed matches can change global state in an unspecified way.

I think you exaggerate the magnitude of the flaw. We can already write conditional code that has side effects, or that changes global state, and we could do that even before the walrus operator was introduced. The PEP doesn't *mandate* that variables are modified during matching, it only *allows* it. This is a clear "Quality of Implementation" issue. Quote: """ The implementation may choose to either make persistent bindings for those partial matches or not. User code including a match statement should not rely on the bindings being made for a failed match, but also shouldn't assume that variables are unchanged by a failed match. This part of the behavior is left intentionally unspecified so different implementations can add optimizations, and to prevent introducing semantic restrictions that could limit the extensibility of this feature. """ I have no problem with this being implementation-defined. -- Steve

If you don't want the overlapping with the existing syntax angle brackets can be used: match eggs: case Spam: ... On Sat, Nov 21, 2020 at 7:31 PM David Mertz wrote:

On Sat, Nov 21, 2020 at 12:23 PM Steven D'Aprano wrote:

...

Clearly Spam(a=1, b=2) does not necessarily result in an instance with attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be equivalent to (roughly):

if (instance(obj, Spam) and getattr(obj, a) == 1 and getattr(obj, b) == 2)

it doesn't imply that obj was *literally* created by a call to the constructor `Spam(a=1, b=2)`, or even that this call would be possible.

I think this explanation makes me not worry about the fact that `Spam(a=1, b=2)` in a pattern looks a lot like a constructor. Like some other commenters, I was vaguely bothered that the identical spelling might have these different meanings in different contexts. But I think a match case just clearly enough IS a different context that using slightly different intuitions is no real conceptual stretch for remembering or teaching it.

As a strawman, we could use different syntax for "match the thing of class Spam that has attributes with these values:

match eggs: case Spam[a=1, b=2]: ...

Or:

match eggs: case Spam{a=1, b=2}: ...

Well, the square brackets COULD mean something different if PEP 637 is adopted. But even supposing the curly braces could be fit into the grammar. Yes, it sort of suggests the connection between dictionaries and Spam.__dict__. But it still reads as "this is something special that I have to think about a little differently."

Even where there are capture variables, I think I'd be completely comfortable thinking about the different context for:

match eggs: case Spam(a=x, b=2): ...

-- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XSRVX2NT... Code of Conduct: http://python.org/psf/codeofconduct/

-- Thanks, Andrew Svetlov

Regarding the difficulty which some people have respecting class patterns and dictionary patterns, I would like to draw attention to a similar feature in JavaScript, object destructuring. JavaScript does not have pattern matching but object destructuring is closely related. Take the example of a function to compute the distance between two points. ``` function distance(p1, p2) { // This is an object destructuring assignment that extracts the x field of p1 into x1 and the y field of p1 into y1 const { x: x1, y: y1 } = p1; // This is an object destructuring assignment that extracts the x field of p2 into x2 and the y field of p2 into y2 const { x: x2, y: y2 } = p2; const dx = x2 - x1; const dy = y2 - y1; return Math.sqrt(dx*dx + dy*dy); } // An object literal is assigned to p1 here const p1 = { x: 0, y: 0 }; // An object literal is assigned to p2 here const p2 = { x: 1, y: 1 }; const d = distance(p1, p2); ``` Similarly to dictionary patterns in Python, object destructuring in JavaScript places the variable names that are the targets of the assignments in the same position where a value would be expected in an object literal. This feature has existed in JavaScript for several years now and I would like to draw attention to it as a counterpoint to the argument that pattern matching is not a good fit for languages that are not statically typed. Destructuring can also be found in Common Lisp, which is not statically typed. Pattern matching is also a core part of Erlang, which is not statically typed.. I would also like to draw attention to the fact that object destructuring was added to JavaScript years into it's development. I do not believe this to be a reasonable argument against adopting pattern matching in Python.

I think your changed constructor: class Car: def __init__(self, manufacturer, variant): self.brand = manufacturer self.model = variant is a particularly good example, and the PEP should specify whether: Car("Chrysler", "PT Cruiser") is matched by: Car(brand="Chrysler", mod:=model) or: Car(manufacturer="Chrysler", mod:=variant) or both, or possibly even Frankenstein combinations like: Car(brand="Chrysler", mod:=variant) -jJ

On Sun, Nov 22, 2020 at 5:03 PM Jim J. Jewett wrote:

I think your changed constructor:

class Car: def __init__(self, manufacturer, variant): self.brand = manufacturer self.model = variant

is a particularly good example, and the PEP should specify whether: Car("Chrysler", "PT Cruiser")

is matched by: Car(brand="Chrysler", mod:=model) or: Car(manufacturer="Chrysler", mod:=variant) or both, or possibly even Frankenstein combinations like: Car(brand="Chrysler", mod:=variant)

(a) We got rid of the walrus and now spell it using '<Pattern> as <variable>' instead of '<variable> := <Pattern>". (b) All your examples have a positional argument following a keyword argument and that's invalid syntax, in patterns as in expressions. (This was the only bug in Steven's example that I found, but I didn't bother to call it out.) (c) What Ethan said -- after fixing the syntax this does show that the keywords in class patterns correspond to attributes, not to constructor arguments (unless those are the same, as they often are, and always for dataclasses and named tuples). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...