PEP 642 v3: Explicit patterns for structured pattern matching
I’ve made a final round of updates to PEP 642 and submitted it to the Steering Council for consideration alongside PEP 634. As usual, the rendered version can be found here: https://www.python.org/dev/peps/pep-0642/ There's a Discourse thread at https://discuss.python.org/t/pep-642-v3-explicit-pattern-syntax-for-structur..., and the rest of the email covers the same points as the opening post in that thread. There are some pretty significant changes relative to v2, although I did already discuss most of them in the v2 thread at https://discuss.python.org/t/pep-642-constraint-pattern-syntax-for-structura... The PEP itself contains a list of major changes relative to PEP 634, so I won’t repeat that here: https://www.python.org/dev/peps/pep-0642/#appendix-c-summary-of-changes-rela... Instead I’ll summarise the parts that I consider most important: * ensuring that all “binding to the right” operations use the as keyword. This drove changes to both mapping patterns and class patterns. * explicitly qualifying both name bindings and value constraints with `as`, `==`, or `is`. This change makes it possible to make pattern matching available to users without having to resolve the thorny questions of what bare names and attribute references should do by default. It also opens up the possibility of potentially adding more value constraint options later (like `in` , `is not`, and `!=`) if those operations seem sufficiently compelling to be worth adding. * explicitly decoupling sequence pattern matching from iterable unpacking. The change to require qualification of name binding operations already breaks the alignment between the two, and that created an opportunity to simplify the grammar by only allowing square bracket based sequence patterns and eliminating both open sequence patterns and parenthesis based sequence patterns * changing class patterns to draw more of their syntactic inspiration from mapping patterns rather than from class instantiation * explicitly representing patterns in the AST, rather than treating patterns as pseudo-expressions all the way through to the code generation layer. Skipping this step makes the code fragile and hard to follow, as there isn’t actually any point in the AST that accepts both expressions and patterns, but with pattern parsing reusing expression nodes, you can’t tell from just looking at the AST which nodes expect subexpressions and which expect subpatterns. I’ll also quote the example match statement from the PEP abstract, which extracts “host” and “port” details from a 2 item sequence, a mapping with “host” and “port” keys, any object with “host” and “port” attributes, or a “host:port” string, treating the “port” as optional in the latter three cases: port = DEFAULT_PORT match expr: case [as host, as port]: pass case {"host" as host, "port" as port}: pass case {"host" as host}: pass case object{.host as host, .port as port}: pass case object{.host as host}: pass case str{} as addr: host, __, optional_port = addr.partition(":") if optional_port: port = optional_port case __ as m: raise TypeError(f"Unknown address format: {m!r:.200}") port = int(port) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 3 Jan 2021, at 15:21, Nick Coghlan <ncoghlan@gmail.com> wrote:
I’ve made a final round of updates to PEP 642 and submitted it to the Steering Council for consideration alongside PEP 634.
As usual, the rendered version can be found here: https://www.python.org/dev/peps/pep-0642/
There's a Discourse thread at https://discuss.python.org/t/pep-642-v3-explicit-pattern-syntax-for-structur..., and the rest of the email covers the same points as the opening post in that thread.
There are some pretty significant changes relative to v2, although I did already discuss most of them in the v2 thread at https://discuss.python.org/t/pep-642-constraint-pattern-syntax-for-structura...
The PEP itself contains a list of major changes relative to PEP 634, so I won’t repeat that here: https://www.python.org/dev/peps/pep-0642/#appendix-c-summary-of-changes-rela...
Instead I’ll summarise the parts that I consider most important:
* ensuring that all “binding to the right” operations use the as keyword. This drove changes to both mapping patterns and class patterns. * explicitly qualifying both name bindings and value constraints with `as`, `==`, or `is`. This change makes it possible to make pattern matching available to users without having to resolve the thorny questions of what bare names and attribute references should do by default. It also opens up the possibility of potentially adding more value constraint options later (like `in` , `is not`, and `!=`) if those operations seem sufficiently compelling to be worth adding. * explicitly decoupling sequence pattern matching from iterable unpacking. The change to require qualification of name binding operations already breaks the alignment between the two, and that created an opportunity to simplify the grammar by only allowing square bracket based sequence patterns and eliminating both open sequence patterns and parenthesis based sequence patterns * changing class patterns to draw more of their syntactic inspiration from mapping patterns rather than from class instantiation * explicitly representing patterns in the AST, rather than treating patterns as pseudo-expressions all the way through to the code generation layer. Skipping this step makes the code fragile and hard to follow, as there isn’t actually any point in the AST that accepts both expressions and patterns, but with pattern parsing reusing expression nodes, you can’t tell from just looking at the AST which nodes expect subexpressions and which expect subpatterns.
I’ll also quote the example match statement from the PEP abstract, which extracts “host” and “port” details from a 2 item sequence, a mapping with “host” and “port” keys, any object with “host” and “port” attributes, or a “host:port” string, treating the “port” as optional in the latter three cases:
port = DEFAULT_PORT match expr: case [as host, as port]: pass case {"host" as host, "port" as port}: pass case {"host" as host}: pass case object{.host as host, .port as port}: pass case object{.host as host}: pass case str{} as addr: host, __, optional_port = addr.partition(":") if optional_port: port = optional_port case __ as m: raise TypeError(f"Unknown address format: {m!r:.200}") port = int(port)
I read the above and believe I know what it meant without needing to read the PEP in detail. I like that a lot. I quickly read 642 v3 and missed an explanation about why the syntax to match a string object is str{} and not str. Are you saying that I MUST use {} so that when case is parsed its clear that its a class with no constraints? in the "Changes to class patterns" I read the BinaryOp example and I thought from the above that it would also use {} and not (). --- match expr: case BinaryOp(== '+', as left, as right): --- I was expecting to see: --- match expr: case BinaryOp{== '+', as left, as right}: --- Barry
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GQHKW5KH... Code of Conduct: http://python.org/psf/codeofconduct/
On Sun, 3 Jan 2021 16:19:01 +0000 Barry Scott <barry@barrys-emacs.org> wrote:
I’ll also quote the example match statement from the PEP abstract, which extracts “host” and “port” details from a 2 item sequence, a mapping with “host” and “port” keys, any object with “host” and “port” attributes, or a “host:port” string, treating the “port” as optional in the latter three cases:
port = DEFAULT_PORT match expr: case [as host, as port]: pass case {"host" as host, "port" as port}: pass case {"host" as host}: pass case object{.host as host, .port as port}: pass case object{.host as host}: pass case str{} as addr: host, __, optional_port = addr.partition(":") if optional_port: port = optional_port case __ as m: raise TypeError(f"Unknown address format: {m!r:.200}") port = int(port)
I read the above and believe I know what it meant without needing to read the PEP in detail. I like that a lot.
+1. Unlike the other PEP, there is no confusion with regular Python syntax such as function calls. Regards Antoine.
On Sun, 3 Jan 2021 at 16:26, Barry Scott <barry@barrys-emacs.org> wrote:
I read the above and believe I know what it meant without needing to read the PEP in detail. I like that a lot.
Personally, I read it and was horribly confused. I worked out most of it, but I would *not* count it as intuitive or natural. Specific examples: case {"host" as host, "port" as port}: pass case {"host" as host}: pass I assume that's dictionary unpacking? It doesn't really look like anything else in Python, though, and it took me a while to work out. case object{.host as host, .port as port}: pass I can only guess at this. I assume "subclass of object with host and port attributes/properties"? But why use {...} for object access? And does that extend to Foo{...} meaning "subclass of Foo? There's no reason to assume yes or no to that. Overall, the whole thing feels like an attempt to invent some sort of syntax in reaction to the PEP 634 form - not having a logic in its own right, but simply with a driving principle of "don't be like PEP 534". It abandons any idea of "make matching look like the thing that's being matched" and replaces it with a completely new set of syntax, which lacks the intuitive nature that I expect from Python (and yes, I know that can be read as "I'm not familiar with it, so I don't like it" - that may indeed be all that it is, but I feel that it's a bit more fundamental than just that). I have not read the full PEP, so take this as very much a "first impression" reaction, but I'd avoid using this syntax, both in my own projects and in any project I contribute to. Honestly, I feel like I'd rather see pattern matching rejected altogether than see this version accepted. Paul
On 1/3/21 8:50 AM, Paul Moore wrote:
Personally, I read it and was horribly confused.
case object{.host as host, .port as port}: pass
Leading periods is a big no-go for me, for all the reasons listed in the original thread.
I have not read the full PEP, so take this as very much a "first impression" reaction, but I'd avoid using this syntax, both in my own projects and in any project I contribute to. Honestly, I feel like I'd rather see pattern matching rejected altogether than see this version accepted.
Agreed. -- ~Ethan~
On Mon, 4 Jan 2021, 4:34 am Ethan Furman, <ethan@stoneleaf.us> wrote:
On 1/3/21 8:50 AM, Paul Moore wrote:
Personally, I read it and was horribly confused.
case object{.host as host, .port as port}: pass
Leading periods is a big no-go for me, for all the reasons listed in the original thread.
It gave me pause as well, but the differences I see relative to the proposed usage in the early iterations of PEP 622 are: 1. This dot isn't semantically significant to the parser, it's just a "this is an attribute name" hint for the human reader. If you forget it, you get a syntax error rather than the code meaning something else. 2. It only appears inside instance attribute mappings, not as part of arbitrary patterns, so the leading dot isn't the only "this is an attribute reference" hint. 3. It means the same thing as dots do in the rest of Python (indicating that the following identifier is an attribute name). Cheers, Nick.
Hello, On Sun, 3 Jan 2021 16:50:33 +0000 Paul Moore <p.f.moore@gmail.com> wrote:
On Sun, 3 Jan 2021 at 16:26, Barry Scott <barry@barrys-emacs.org> wrote:
I read the above and believe I know what it meant without needing to read the PEP in detail. I like that a lot.
Personally, I read it and was horribly confused. I worked out most of it, but I would *not* count it as intuitive or natural.
Specific examples:
case {"host" as host, "port" as port}: pass case {"host" as host}: pass
I assume that's dictionary unpacking? It doesn't really look like anything else in Python, though, and it took me a while to work out.
case object{.host as host, .port as port}: pass
I can only guess at this. I assume "subclass of object with host and port attributes/properties"? But why use {...} for object access? And does that extend to Foo{...} meaning "subclass of Foo? There's no reason to assume yes or no to that.
Overall, the whole thing feels like an attempt to invent some sort of syntax in reaction to the PEP 634 form - not having a logic in its own right, but simply with a driving principle of "don't be like PEP 534". It abandons any idea of "make matching look like the thing that's being matched" and replaces it with a completely new set of syntax, which lacks the intuitive nature that I expect from Python
+1 for that evaluation. And I'd like to remind with what the original discussion started, with very simple and focused questions: a) What if we *do* use sigils for value patterns? or, b) What if we *do* use sigils for capture patterns? Instead of focusing on one of the above questions, making fine, focused, pin-head sized adjustments to the original pattern matching PEPs, with PEP642 we've got a noticeable scope creep, with evermore extravagant syntax replacements of the large part of the original proposal. This once again shows that it's very easy to get carried away and how important for others to help to remind where to stop. []
I have not read the full PEP, so take this as very much a "first impression" reaction, but I'd avoid using this syntax, both in my own projects and in any project I contribute to. Honestly, I feel like I'd rather see pattern matching rejected altogether than see this version accepted.
Just the same, there's no need to go too far here either. The question of PEP642 rejection should be treated as orthogonal to acceptance / rejection of PEP634. Because, well, PEP634 misses to do some things better, but indeed at least some of them can be added later. But PEP642 twists matters so much, that it likely will be irreparable afterwards.
Paul
-- Best regards, Paul mailto:pmiscml@gmail.com
On Mon, 4 Jan 2021, 2:50 am Paul Moore, <p.f.moore@gmail.com> wrote:
On Sun, 3 Jan 2021 at 16:26, Barry Scott <barry@barrys-emacs.org> wrote:
I read the above and believe I know what it meant without needing to read the PEP in detail. I like that a lot.
Personally, I read it and was horribly confused. I worked out most of it, but I would *not* count it as intuitive or natural.
Specific examples:
case {"host" as host, "port" as port}: pass case {"host" as host}: pass
I assume that's dictionary unpacking? It doesn't really look like anything else in Python, though, and it took me a while to work out.
These *can* have the colon included, and it would be quite viable to have that as the only spelling in the initial design iteration: case {"host": as host, "port": as port}: pass case {"host": as host}: pass The submitted grammar just allows the colon to be left out for brevity (as even with it missing you were still able to correctly identify this as a mapping match).
case object{.host as host, .port as port}: pass
I can only guess at this. I assume "subclass of object with host and port attributes/properties"?
Correct :) But why use {...} for object access? And
does that extend to Foo{...} meaning "subclass of Foo? There's no reason to assume yes or no to that.
The instance attribute syntax arose from trying to deal with two problems from class patterns in PEP 634: * "ATTR=TARGET" using "=" to bind to the right instead of to the left * no subsequent path to ever offering a syntax for *retrieving* multiple attributes in ordinary expressions outside pattern matching The minimal fix for the first point would have been just "case object(host=as host, port=as port}:", but that couldn't ever be used to retrieve multiple attributes, as "object(host, port)" is already a function call. By contrast, "object{.host, .port}" is currently illegal syntax that could plausibly become a syntactic shorthand for "(object.host, object.port)" in both ordinary expressions and in assignment targets, even if it was initially only supported in pattern matching.
Overall, the whole thing feels like an attempt to invent some sort of syntax in reaction to the PEP 634 form - not having a logic in its own right, but simply with a driving principle of "don't be like PEP 534".
The design logic went as follows: 1. Start with PEP 634 2a. Ensure all "binding to the right" operations use 'as' 2b. Ensure all bare names outside subexpressions are qualified with a sigil or keyword 3. Ensure all deconstruction operations offer a potential future path to producing tuples in ordinary expressions using pattern matching inspired syntax It abandons any idea of "make matching look like the thing that's
being matched"
PEP 634 doesn't offer that for class patterns either. I like being able to leave out the colon for mapping name bindings, but wouldn't object to requiring that it be left in. and replaces it with a completely new set of syntax,
which lacks the intuitive nature that I expect from Python (and yes, I know that can be read as "I'm not familiar with it, so I don't like it" - that may indeed be all that it is, but I feel that it's a bit more fundamental than just that).
I have not read the full PEP, so take this as very much a "first impression" reaction,
I'd definitely like to hear your second impression after reviewing the PEP text, but getting first impressions like yours was a big part of my motivation for including the match statement example. but I'd avoid using this syntax, both in my own
projects and in any project I contribute to. Honestly, I feel like I'd rather see pattern matching rejected altogether than see this version accepted.
I've reached that point with PEP 634 - I think the problems with binding to the right in mapping and especially class patterns are serious enough that I'd prefer to see no pattern matching to *that* syntax for pattern matching. I also have concerns about AST ambiguity in the current PEP 634 implementation, but those could be fixed by defining separate AST nodes for patterns without having to change the surface syntax proposal (the AST I've defined for 642 should be usable for 634 as well). Cheers, Nick.
Paul
On Sun, 3 Jan 2021 at 23:38, Nick Coghlan <ncoghlan@gmail.com> wrote:
The instance attribute syntax arose from trying to deal with two problems from class patterns in PEP 634:
* "ATTR=TARGET" using "=" to bind to the right instead of to the left * no subsequent path to ever offering a syntax for *retrieving* multiple attributes in ordinary expressions outside pattern matching
The minimal fix for the first point would have been just "case object(host=as host, port=as port}:", but that couldn't ever be used to retrieve multiple attributes, as "object(host, port)" is already a function call.
OK, so there's our dispute. Neither of those seem to me to be problems with PEP 634. 1. I view ATTR=PLACEHOLDER as *equality* with a placeholder that gets filled in, not a binding that goes left to right. (And no, I don't have a problem with the rule that the placeholder must be on the right). 2. I don't see any immediate reason to assume we want to "retrieve multiple attributes in ordinary expressions". We can easily add a "match let" statement that did match-style destructuring, why do we need it to be built in to ordinary assignment (which is what I assume you mean)? So IMO you're producing a weird syntax to solve problems I don't believe exist in the first place. Whereas PEP 634 offers a feature that I think I would find useful, using a syntax that I find perfectly comfortable in the context in which it's defined. Paul
On Mon, 4 Jan 2021 at 18:38, Paul Moore <p.f.moore@gmail.com> wrote:
On Sun, 3 Jan 2021 at 23:38, Nick Coghlan <ncoghlan@gmail.com> wrote:
The instance attribute syntax arose from trying to deal with two problems from class patterns in PEP 634:
* "ATTR=TARGET" using "=" to bind to the right instead of to the left * no subsequent path to ever offering a syntax for *retrieving* multiple attributes in ordinary expressions outside pattern matching
The minimal fix for the first point would have been just "case object(host=as host, port=as port}:", but that couldn't ever be used to retrieve multiple attributes, as "object(host, port)" is already a function call.
OK, so there's our dispute. Neither of those seem to me to be problems with PEP 634.
1. I view ATTR=PLACEHOLDER as *equality* with a placeholder that gets filled in, not a binding that goes left to right. (And no, I don't have a problem with the rule that the placeholder must be on the right).
But why would you assume `=` means that in Python code? The only places we currently use `=` are in assignment statements and in keyword arguments, and in both of those cases, the target is on the left: TARGET = EXPR func(TARGET_PARAM=EXPR) However, you're right, if "SOURCE = DESTINATION" doesn't read strangely to you, then the switch to mapping-pattern inspired syntax for attribute patterns in PEP 642 isn't going to feel compelling. For the other point, consider the following examples (using "match SUBJECT as case PTRN" as a strawman one-shot pattern matching syntax): # One shot PEP 634 pattern match addr as case object(host=host, port=port) # One shot PEP 642 pattern match addr as case object{.host as host, .port as port} # Attribute tuple (PEP 642 based deferred idea) host, port = addr{.host, .port} One shot pattern matching turns out to have an inherent verbosity problem, as it forces the specification of the destructuring to be separated from the specification of the subject. That separation is desirable in match statements, where one subject is potentially destructured in different ways. It's far from clear that the separation would be desirable in one-shot matching - we might instead be better off with a model that only allows simple destructuring operations that produce a tuple, and then uses ordinary iterable unpacking for name binding. While that's a relatively minor point in the grand scheme of things, I do like having more future design options available to us. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hello, On Tue, 5 Jan 2021 20:37:27 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
object(host=as host, port=as port}:", but that couldn't ever be
I'd like to point out the weirdness of the "as" syntax when applied to positional arguments, e.g.: case [as x, as y]: case Cls(as x, as y): That feels unnatural, and the fact behind that intuitive feeling is that "as" *never* used like that in Python so far. Where it's used now, there's explicit "what" is available before "as": import org_name as alias_name with expr as var: So again, standalone "as" feels weird. To feel it natural, it would need to be written as "case [_ as x, _ as y]:", but that's exactly what PEP634 already allows, and nothing needs to be done! And I'd like to remind that one of the original alternatives raised for the syntax was something like: case [>x, >y]: Which is both concise and intuitive (for as long as you accept that ">" here represent an arrow, which should be one-time-in-life learning event). The argument for "lone as" can be "just let that syntax keep you hostages for some time (say, one week of regular usage) and you'll acquire Stockholm Syndrome for it." That may very well be true. But 1-or-2-character punctuation prefix sigil is still more concise and more intuitive. []
For the other point, consider the following examples (using "match SUBJECT as case PTRN" as a strawman one-shot pattern matching syntax):
# One shot PEP 634 pattern match addr as case object(host=host, port=port)
# One shot PEP 642 pattern match addr as case object{.host as host, .port as port}
# Attribute tuple (PEP 642 based deferred idea) host, port = addr{.host, .port}
One shot pattern matching turns out to have an inherent verbosity problem,
My bet that it has "inherent verbosity problem" only because there was no such proposal. And any such proposal would start with basics: What's syntax for sequence destructuring aka pattern matching against catch-all variables? Assignment to a tuple/list: (a, b) = tuply Ok, what happens if you assign to a dict syntax now: {a: b} = dicty Syntax error? Ok, that's new one-time dict matching. What happens if you assign to a function call syntax now (also object constructor syntax): Addr(ip, port) = addr_obj Syntax error? That's new one-time object matching. Not verbose at all: (a, b) = tuply {a: b} = dicty Addr(ip, port) = addr_obj Now, those throw ValueError, what if you want to get a boolean? Just wrap it in the "if" context: if (a, b) = tuply: if {a: b} = dicty: if Addr(ip, port) = addr_obj: From there, bikeshedding can start. E.g., there will be strong cry that "= and == are easy to mix up", so it'll become: if match (a, b) = tuply: if match {a: b} = dicty: if match Addr(ip, port) = addr_obj: That's a whole 1 token more verbose, but 6 chars more, so someone will come up with following alternatives: if_m (a, b) = tuply: if (a, b) m= tuply: Etc, the usual bikeshedding whirlwind. [] -- Best regards, Paul mailto:pmiscml@gmail.com
Paul Sokolovsky wrote:
object(host=as host, port=as port}:", but that couldn't ever be I'd like to point out the weirdness of the "as" syntax when applied to
Hello, On Tue, 5 Jan 2021 20:37:27 +1000 Nick Coghlan ncoghlan@gmail.com wrote: positional arguments, e.g.: case [as x, as y]: case Cls(as x, as y): That feels unnatural, and the fact behind that intuitive feeling is that "as" never used like that in Python so far. Where it's used now, there's explicit "what" is available before "as": import org_name as alias_name with expr as var: So again, standalone "as" feels weird.
It's a matter of taste, I like the compacity of the standalone as. It describe clearly the variable name, and the order matches the positional arguments so I don't find it surprising. If it's not the positional value, what else can it be ? What would be clearer by using an underscore, which itself corresponds to nothing ? The fact that it was never used like this it not an argument per se, because that's the point of new syntax...
On Sat, 9 Jan 2021, 7:07 am Joseph Martinot-Lagarde, <contrebasse@gmail.com> wrote:
Paul Sokolovsky wrote:
object(host=as host, port=as port}:", but that couldn't ever be I'd like to point out the weirdness of the "as" syntax when applied to
Hello, On Tue, 5 Jan 2021 20:37:27 +1000 Nick Coghlan ncoghlan@gmail.com wrote: positional arguments, e.g.: case [as x, as y]: case Cls(as x, as y): That feels unnatural, and the fact behind that intuitive feeling is that "as" never used like that in Python so far. Where it's used now, there's explicit "what" is available before "as": import org_name as alias_name with expr as var: So again, standalone "as" feels weird.
It's a matter of taste, I like the compacity of the standalone as. It describe clearly the variable name, and the order matches the positional arguments so I don't find it surprising. If it's not the positional value, what else can it be ? What would be clearer by using an underscore, which itself corresponds to nothing ? The fact that it was never used like this it not an argument per se, because that's the point of new syntax...
It's also a pure syntactic shortcut, so anyone that really dislikes it could favour the "__ as name" form. The extra leading "__" doesn't convey any information that the leading "as" doesn't convey on its own, but YMMV. The key difference relative to PEP 634 is that even when the code author uses the shorthand form, *readers* will still get at least the "as" keyword as a prompt, rather than having to just know that "name" appearing in a pattern means "__ as name", unless it's a class or attribute name in a class pattern, or an attribute name in a value pattern. While we know from experience that it's feasible to keep that in mind for iterable unpacking, it's far from clear that it will work as well for the more complex destructuring involved in class and mapping patterns. Cheers, Nick.
Hello, On Sat, 9 Jan 2021 12:27:45 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
On Sat, 9 Jan 2021, 7:07 am Joseph Martinot-Lagarde, <contrebasse@gmail.com> wrote:
Paul Sokolovsky wrote:
object(host=as host, port=as port}:", but that couldn't ever be I'd like to point out the weirdness of the "as" syntax when applied to
Hello, On Tue, 5 Jan 2021 20:37:27 +1000 Nick Coghlan ncoghlan@gmail.com wrote: positional arguments, e.g.: case [as x, as y]: case Cls(as x, as y): That feels unnatural, and the fact behind that intuitive feeling is that "as" never used like that in Python so far. Where it's used now, there's explicit "what" is available before "as": import org_name as alias_name with expr as var: So again, standalone "as" feels weird.
It's a matter of taste, I like the compacity of the standalone as. It describe clearly the variable name, and the order matches the positional arguments so I don't find it surprising. If it's not the positional value, what else can it be ? What would be clearer by using an underscore, which itself corresponds to nothing ? The fact that it was never used like this it not an argument per se, because that's the point of new syntax...
It's also a pure syntactic shortcut, so anyone that really dislikes it could favour the "__ as name" form. The extra leading "__" doesn't convey any information that the leading "as" doesn't convey on its own, but YMMV.
The key difference relative to PEP 634 is that even when the code author uses the shorthand form, *readers* will still get at least the "as" keyword as a prompt,
Ok, so let's summarize the alternatives: 1. PEP634, which says that "case Cls(a, b):", etc. is not worse than other languages. 2. PEP642, which says that taking existing "as" and using it (and it's also an English word with a specific meaning) with rather different meaning in pattern matching is ok. 3. Alternative proposal sounded on the list, which says that taking existing punctuational operator like ">" or "->" (and they're also graphemes depicturing arrows) and using them (still with rather different meaning) in pattern matching is ok. PEP642 never mentions 3rd alternative. And it actually starts its presentation with things like:
case {"host" as host, "port" as port}:
There're 2 obvious problems with it: a) In Python, {} with things inside it, but no ":" inside it, is a set, set. b) Everywhere else in Python, thing on the left of "as" gets into thing on the right of "as", behold: import foo as bar # original module "foo" gets into "bar". with Cls(a, b) as c: # original expression "Cls(a, b)" gets into "c" Then based on the existing Python syntax, the meaning of '{"host" as host, "port" as port}' is: a set, whose contained, constant in this case, values "host" and "port" get captured as variables host and port. In pattern matching context (with 'case' in front), it means: match using a set, check for presence on constant "host" and "port" elements, and capture those constants to variables host and port (that capturing doesn't make much sense, yeah. And yet syntax tells just that. You propose to assign to it completely different meaning.). So, the "dict pattern shortcut syntax" proposed by PEP642 is rather ambiguous and confusing, and represent climax of PEP642's "fixing too many things at once", where it goes over the summit and onto the dark side. And even looking at: case {"host": as host, "port": as port} I'm personally tripped by the meaning of "as" in English, and think that it does something about thing on the left of it, i.e. the dictionary *key*, not the *value*, as it really does. To get around that effect, I'd need to write it as: case {"host": (as host), "port": (as port)} That finally looks pretty unambiguous to me. Again, an alternative is: case {"host": ->host), "port": ->port} Which is pretty self-expressive and unambiguous right away IMHO. (Of course, there's a fine distinction between "arrow points from A to B" vs "arrow points into B", I bet on the last meaning). Also, the distinction between "as" vs "->" in patterns is of the same nature as distinction between "or" vs "|" in them (there're reasons why "|" is used for alternatives, and not "or", right?)
rather than having to just know that "name" appearing in a pattern means "__ as name",
But PEP634 doesn't have no "__ as name"! It has "_ as name". And that's another case of PEP642's "too much at once". While that change is pretty clearly presented in the PEP642, I find that discussion here rather downplays it. Specifically, you use "__" even in the context talking about PEP634, as if you take it for granted. Instead, each time you mention "__", you'd rather say "which is another alternative syntax PEP642 propose". Because you see, I'm almost sure that readers of just this thread don't even pay enough attention that you use double-underscore instead of the original underscore. That leads us to the obvious concern: 1. The difference between "__" and "_" isn't visible enough. People should start saying "OMG" and "horror!" now, not when PEP642 gets implemented and they finally notice that they need to type _ twice. Which leads us to: 2. Both PEP642 and discussion here should elaborate explicitly what happens when people still use "_" in patterns. Beyond that: 3. The PEP642 attributes the idea to "a Stackoverflow answer", but discloses that the author of the PEP is also the author of the Stackoverflow answer. There's nothing wrong with that. But, given that the answer is from 2011, how much adoption that idea got in the meantime? If not enough, what would be the reasons for that? How those reasons might affect the usage in PEP642? The Stackoverflow answer specifically mentions:
(many folks prefer a double-underscore, __, as their throwaway variable for exactly this reason).
Never saw that, sorry. (Maybe I saw, but didn't pay enough attention that it's double-underscore instead of single, exactly the problem we discuss).
Linters often recognize this use case. [...] The fix, if day is truly not needed, is to write year, month, _ = date().
So, you see, it says "many people prefer __", and then immediately says "linters recognize _ as a special case". So, 10 years later, how many linters recognize double-underscore as a special case too? Other general comments on PEP642v3 text:
* allow an initial form of pattern matching to be developed and released without needing to decide up front on the best default options for handling bare names, attribute lookups, and literal values
Can this be made more explicit with appending: " (that's why it proposes to prefix both capture patterns and value constraints, even though prefixing just one would be enough to avoid ambiguity)".
MatchValue(matchop op, expr value) matchop = EqCheck | IdCheck
Why bloat ASDL with duplicates? Why not "matchop = Eq | Is" ?
whereas that path is blocked for "_" by i18n related use cases. ... This deprecation path couldn't be followed for _, as there's no way for the interpreter to distinguish between attempts to read back _ when nominally used as a "don't care" marker, and legitimate reads of _ as either an i18n text translation function or as the last statement result at the interactive prompt.
A way to deprecate lagacy use of "_" would be something like "from __future__ import _". Sources which use legacy i18n naming can't use pattern matching, might be considered a fair tradeoff. Interactive prompt is also a separate usecase.
this PEP restricts sequence patterns specifically to the square bracket form.
That's relatively minor, but sad choice, because [...] signifies a mutable list, whereas a language should promote use of immutable values, like tuple (...).
This means that only types that actually define __match_args__ will be usable in class defined patterns. Types that don't define __match_args__ will still be usable in instance attribute patterns.
All this is neat, smart construction, but not intuitive, sufficiently irregular, complicated, and confusing. I grokked the idea after reading the PEP642, it's smart, again. But I don't think it's ok to subject innocent to study these complex conceptual hierarchies of "objects designed to be used with Cls(as foo, as bar) matching syntax, and those which weren't, and must use unheard of, and NIH syntax of Cls{.attr1 as foo, .attr2 as bar} instead". People should just learn pattern matching as it's presented in other languages indeed. To help them with "may bind to the right" matter, a very simple, focused change was proposed - to add explicit arrow pointing straight at the bind/capture target. Instead of that simple measure, PEP642 builds whole parallel hierarchy of concepts, which is very artificial and complex response ("seeing complication, rehash everything, and add significantly more overall complexity to deal with originally simple case"). Again, I pray for PEP642 rejection, based on its "attempting to do too much, and overdoing it to "remedy is worse than the problem" situation, and trying to build complex hierarchies to artificially separate concepts in pattern matching, instead of treating pattern matching as "cartesian product" of simple concepts (and see how to make these simple concepts more explicit to humans, rather than adding more constraints for humans to be aware of). Deep and unobvious constraints, how on earth I know if some class defines __match_args__ or not, so I can select appropriate syntax? I simply will use the syntax which is usable for all cases (and that's the ugly one), and the other syntax will be used only for builtin types, and whoever will see it used for a non-builtin class will be shocked and confused. [] -- Best regards, Paul mailto:pmiscml@gmail.com
On Sat, 9 Jan 2021 at 10:52, Paul Sokolovsky <pmiscml@gmail.com> wrote:
case {"host" as host, "port" as port}:
There're 2 obvious problems with it:
a) In Python, {} with things inside it, but no ":" inside it, is a set, set. b) Everywhere else in Python, thing on the left of "as" gets into thing on the right of "as", behold:
import foo as bar # original module "foo" gets into "bar". with Cls(a, b) as c: # original expression "Cls(a, b)" gets into "c"
Then based on the existing Python syntax, the meaning of '{"host" as host, "port" as port}' is: a set, whose contained, constant in this case, values "host" and "port" get captured as variables host and port. In pattern matching context (with 'case' in front), it means: match using a set, check for presence on constant "host" and "port" elements, and capture those constants to variables host and port (that capturing doesn't make much sense, yeah. And yet syntax tells just that. You propose to assign to it completely different meaning.).
This analysis nicely captures my reservations with the proposal here. It claims to be addressing the problems with PEP 634, where the syntax proposed there is specialised to matching, but it then proceeds to introduce another syntax, which is *also* unlike existing usage, just in different ways. You may say that the PEP 642 syntax is "better", but that's a matter of opinion. Having {...} without a colon be a dict-style unpacking is very uncomfortable to me. Having "as" not assign the result of what's on the left of it to the right hand name is uncomfortable to me. The PEP 634 equivalents do *not* feel uncomfortable in the same way. I'm mostly just repeating myself here, though. I'm on record already as saying that I feel PEP 642 is strictly worse than PEP 634. My main worry is that the two competing PEPs turn the question into "let's assume pattern matching is a good thing, which is the better syntax" - which is *not* the question. Each PEP should be considered on its own merits, and people who like pattern matching but don't like the PEP 634 syntax should be *at least* as strict in their analysis of PEP 642 (possibly even more so, because PEP 642 makes a bunch of unverified - and in my view, unjustified - assumptions about the desirability of potential adoption of the syntax in wider contexts). To be honest, though, I think that we're now at the point where we have to rely on the SC for a decision. Pretty much everyone has made their views clear, so the final decision is going to be down to "design instinct". Ultimately, that's the basis on which we all voted for SC members, after all. Paul PS This situation is particularly special, of course, because it's a direct comparison of Guido's design instincts (PEP 634) and the SC's. So whatever the result, it'll be interesting!
On Sat, 9 Jan 2021 12:17:32 +0000 Paul Moore <p.f.moore@gmail.com> wrote:
On Sat, 9 Jan 2021 at 10:52, Paul Sokolovsky <pmiscml@gmail.com> wrote:
case {"host" as host, "port" as port}:
There're 2 obvious problems with it:
a) In Python, {} with things inside it, but no ":" inside it, is a set, set. b) Everywhere else in Python, thing on the left of "as" gets into thing on the right of "as", behold:
import foo as bar # original module "foo" gets into "bar". with Cls(a, b) as c: # original expression "Cls(a, b)" gets into "c"
Then based on the existing Python syntax, the meaning of '{"host" as host, "port" as port}' is: a set, whose contained, constant in this case, values "host" and "port" get captured as variables host and port. In pattern matching context (with 'case' in front), it means: match using a set, check for presence on constant "host" and "port" elements, and capture those constants to variables host and port (that capturing doesn't make much sense, yeah. And yet syntax tells just that. You propose to assign to it completely different meaning.).
This analysis nicely captures my reservations with the proposal here. It claims to be addressing the problems with PEP 634, where the syntax proposed there is specialised to matching, but it then proceeds to introduce another syntax, which is *also* unlike existing usage, just in different ways.
Introducing a new syntax is not a problem, it's the solution. The problem with PEP 634 is precisely that it reuses existing syntax and gives it a different meaning, thereby producing confusion. It's erroneous to claim that `{"host" as host, "port" as port}` is a set. It's currently invalid syntax due to the `as`:
{"host" as host, "port" as port} File "<stdin>", line 1 {"host" as host, "port" as port} ^ SyntaxError: invalid syntax
So, opposing Nick's proposal on the basis that it "looks like a set" is just like opposing set literals on the basis they they "look like a dict". Regards Antoine.
On Sat, 9 Jan 2021 at 13:53, Antoine Pitrou <antoine@python.org> wrote:
So, opposing Nick's proposal on the basis that it "looks like a set" is just like opposing set literals on the basis they they "look like a dict".
That's not what I was doing (I can't comment on what Paul S intended, though). My position is that the lack of a colon makes this not look like a dictionary destructuring to me. The "as" keyword doesn't have the same feeling of describing the *form* of a data structure, possibly because structure in fundamental data types is more often indicated by punctuation than keywords. As a larger point, I'm saying that Nick's proposal has just as many of these "doesn't feel quite right" compromises as the original proposal (actually, *to me* it has far *more* of them, but there's a level of subjectivity here). As a result, I'm unmoved by the arguments that "it's pattern matching, but with a nicer syntax". The dictionary destructuring can act as an example. We know Nick's position: case {"text": message, "color": c}: 1. There's nothing (other than the fact that it's in a case clause) to indicate that message and c are assigned to. 2. It "binds to the right without using as", (as far as I can see, based on the premise that "as" is the only valid way of binding names where the name is on the right, which was asserted without any justification). But the PEP 642 form: case {"text" as message, "color" as c}: is essentially identical except for using "as" rather than a colon. My view is: 1. Nowhere else in Python does "as" indicate a dictionary, and braces alone don't (because sets use them too). 2. It loses the "match looks like the input" aspect, while only gaining some sort of theoretical "as is how we bind to the right" property that's never been a design principle in Python before now. 3. It's entirely new syntax, where the PEP 634 form is similar to existing Python syntax for dictionaries, and to other languages' matching constructs. What I'm saying here is simply that looked at (relatively) objectively, there are similar numbers of debatable points on both sides of the argument. Paul
Le 09/01/2021 à 15:18, Paul Moore a écrit :
But the PEP 642 form:
case {"text" as message, "color" as c}:
is essentially identical except for using "as" rather than a colon. My view is:
1. Nowhere else in Python does "as" indicate a dictionary, and braces alone don't (because sets use them too).
Admittedly. But *something* has to be found, right?
2. It loses the "match looks like the input" aspect, while only gaining some sort of theoretical "as is how we bind to the right" property that's never been a design principle in Python before now.
Is there an official catalog of Python design principles? "... as y" is already (optionally) used in `import` and `with` statements, so this is not an innovation in Nick's PEP.
3. It's entirely new syntax, where the PEP 634 form is similar to existing Python syntax for dictionaries, and to other languages' matching constructs.
As I said, using new syntax to denote a new semantics seems like the right thing to do. Regards Antoine.
On Sun, 10 Jan 2021, 12:22 am Paul Moore, <p.f.moore@gmail.com> wrote:
On Sat, 9 Jan 2021 at 13:53, Antoine Pitrou <antoine@python.org> wrote:
So, opposing Nick's proposal on the basis that it "looks like a set" is just like opposing set literals on the basis they they "look like a dict".
That's not what I was doing (I can't comment on what Paul S intended, though). My position is that the lack of a colon makes this not look like a dictionary destructuring to me. The "as" keyword doesn't have the same feeling of describing the *form* of a data structure, possibly because structure in fundamental data types is more often indicated by punctuation than keywords.
And I've already said I'd be fine with making the colon mandatory if the SC share that view.
As a larger point, I'm saying that Nick's proposal has just as many of these "doesn't feel quite right" compromises as the original proposal (actually, *to me* it has far *more* of them, but there's a level of subjectivity here). As a result, I'm unmoved by the arguments that "it's pattern matching, but with a nicer syntax".
The dictionary destructuring can act as an example. We know Nick's position:
case {"text": message, "color": c}:
1. There's nothing (other than the fact that it's in a case clause) to indicate that message and c are assigned to. 2. It "binds to the right without using as", (as far as I can see, based on the premise that "as" is the only valid way of binding names where the name is on the right, which was asserted without any justification).
What other syntax do we have that binds to the right without "as"? Allowing it at all is only as old as with statements, but that's still more precedence than ":" and "=" have for being used that way.
But the PEP 642 form:
case {"text" as message, "color" as c}:
is essentially identical except for using "as" rather than a colon. My view is:
1. Nowhere else in Python does "as" indicate a dictionary, and braces alone don't (because sets use them too). 2. It loses the "match looks like the input" aspect, while only gaining some sort of theoretical "as is how we bind to the right" property that's never been a design principle in Python before now. 3. It's entirely new syntax, where the PEP 634 form is similar to existing Python syntax for dictionaries, and to other languages' matching constructs.
What I'm saying here is simply that looked at (relatively) objectively, there are similar numbers of debatable points on both sides of the argument.
Except that many of the PEP 642 ones are optional shorthand that the SC can selectively reject or defer if they want to - for example, always requiring the use of ":" in both mapping patterns and instance attribute patterns would only increase verbosity, it wouldn't reduce expressivity. PEP 634 is more constrained than that. For example, mapping keys have to be restricted to just literals and attributes, because allowing bare names or more complex expressions would make the pattern name bindings too hard to pick out. Cheers, Nick.
Paul _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CIFLROSU... Code of Conduct: http://python.org/psf/codeofconduct/
On Sun, 10 Jan 2021, 12:54 am Nick Coghlan, <ncoghlan@gmail.com> wrote:
On Sun, 10 Jan 2021, 12:22 am Paul Moore, <p.f.moore@gmail.com> wrote:
On Sat, 9 Jan 2021 at 13:53, Antoine Pitrou <antoine@python.org> wrote:
The dictionary destructuring can act as an example. We know Nick's position:
case {"text": message, "color": c}:
1. There's nothing (other than the fact that it's in a case clause) to indicate that message and c are assigned to. 2. It "binds to the right without using as", (as far as I can see, based on the premise that "as" is the only valid way of binding names where the name is on the right, which was asserted without any justification).
What other syntax do we have that binds to the right without "as"? Allowing it at all is only as old as with statements, but that's still more precedence than ":" and "=" have for being used that way.
Correcting myself: the import statement usage has been around much longer than that. With statements were just the case that took 'as name' from being an import specific syntax to being a bit more general than that. Cheers, Nick.
On Sat, 9 Jan 2021 at 14:54, Nick Coghlan <ncoghlan@gmail.com> wrote: [...]
And I've already said I'd be fine with making the colon mandatory if the SC share that view.
So the response to my comment that omitting the colon makes it hard to see that it's a dictionary unpacking is either "don't use that form if you don't like it" (which as we all know doesn't account for the problem of dealing with other people's code) or "the SC can make the colon mandatory" which seems to me to be extending the SC's role to designing the syntax rather than just approving the proposal. And means that the PEP we're debating might not be what gets accepted. Which isn't how I understood the process to work, but OK. And if the colon is mandatory, the syntax is {"text": as message, "color": as c}. Which is essentially the same as PEP 634 with an additional "as". Which basically makes this a somewhat disguised version of "add a marker to indicate the variable to be bound". Meh. We've been down this route, some people want bindings to be explicitly marked, other people don't. I don't feel compelled to have explicit markers, I like the "match pattern looks like what's being matched" feature of PEP 634, and don't have a huge difficulty understanding what's bound and what's matched in the sort of real-world cases that have been presented. The nearest to a problem I've seen is "case CONSTANT", which I acknowledge is a potential bug magnet, but I think the "solution" proposed by PEP 642 is far worse than the problem.
What other syntax do we have that binds to the right without "as"? Allowing it at all is only as old as with statements, but that's still more precedence than ":" and "=" have for being used that way.
None at the moment, but that's not my point. My point is that patterns are more like expressions than statements, and all current uses of "as" for binding are in statements. Expressions typically use punctuation rather than keywords, so there's no compelling argument that the syntax for "binding the RHS" *in expressions* has to be "as". You say "we use as in statements, so let's re-use that", I say "we use punctuation in expressions, and a keyword looks clumsy and verbose". I don't believe that point has ever been addressed directly, which is why I get frustrated that arguments about "as" seem to be getting redirected into arguments about binding to the right all the time.
Except that many of the PEP 642 ones are optional shorthand that the SC can selectively reject or defer if they want to - for example, always requiring the use of ":" in both mapping patterns and instance attribute patterns would only increase verbosity, it wouldn't reduce expressivity.
So based on that comment, PEP 642 isn't a proposal so much as a selection of options the SC can pick between. And attempts like this to discuss specifics of the proposal are hard to have, because "well, the SC can drop that bit if they don't like it" is always an option. I don't like that, and I'm not sure it's the right way to use the PEP process. I'm inclined to suggest the SC should reject the proposal simply because it leaves too much undecided, and doesn't take a firm position ;-) But whatever. To restate my position (which hasn't changed much) - I like the idea of pattern matching in Python. PEP 634 seems like a "natural" syntax to me, even though it's somewhat "do what I mean" in terms of the exact details (in other words, I find it a good practical approach even if it frustrates the rulebook lawyer in me). I dislike PEP 642, although I can't honestly put my finger on why - mainly, it seems like it prioritises rules (ease of implementation, keeping the spec concise) over usability. I don't think pattern matching is important enough to accept the usability compromises that I feel PEP 642 requires, but I *do* think it's sufficiently important to mean that I'm comfortable accepting the edge cases (specification compromises) in PEP 634. If PEP 642 were accepted, I'd use pattern matching a *lot* less than if PEP 634 were accepted (and I'd argue against its use more often on projects I contribute to). I'm glad I'm not on the SC and having to make this decision. Paul
On Sun, 10 Jan 2021, 2:55 am Paul Moore, <p.f.moore@gmail.com> wrote:
On Sat, 9 Jan 2021 at 14:54, Nick Coghlan <ncoghlan@gmail.com> wrote:
[...]
And I've already said I'd be fine with making the colon mandatory if the SC share that view.
So the response to my comment that omitting the colon makes it hard to see that it's a dictionary unpacking is either "don't use that form if you don't like it" (which as we all know doesn't account for the problem of dealing with other people's code) or "the SC can make the colon mandatory" which seems to me to be extending the SC's role to designing the syntax rather than just approving the proposal. And means that the PEP we're debating might not be what gets accepted. Which isn't how I understood the process to work, but OK.
It's the same process any PEP goes through: comments from the PEP delegate carry more weight than general review comments, as ignoring them risks getting the PEP rejected.
The SC haven't provided any feedback on the submitted version of 642 yet, so I don't know at this point if they share the concern that allowing omitting the key/value separating colon to be omitted pushes mapping & instance attribute patterns too close to looking like a variation on set syntax instead of mapping syntax. I do accept that it's a legitimate concern, it's just also a feature that's easy to drop if the SC agrees it is problematic.
And if the colon is mandatory, the syntax is {"text": as message, "color": as c}. Which is essentially the same as PEP 634 with an additional "as". Which basically makes this a somewhat disguised version of "add a marker to indicate the variable to be bound".
It isn't disguised at all, as I state it in the PEP's Design Discussion section: """... instead requiring that all uses of bare simple names for anything other than a variable lookup be qualified by a preceding sigil or keyword""" (from https://www.python.org/dev/peps/pep-0642/#requiring-explicit-qualification-o... ) Meh.
We've been down this route, some people want bindings to be explicitly marked, other people don't. I don't feel compelled to have explicit markers, I like the "match pattern looks like what's being matched" feature of PEP 634,
PEP 634 doesn't have that feature for class patterns in general, only for classes like data classes, where the constructor signature is carefully aligned with the match arguments. For positional args, PEP 642 would also align in those carefully constructed cases, so it's only the new syntax for attribute matching where things differ. For mapping patterns, I think you've made a reasonable case that the colon should be required even when unconditionally binding names, so it wouldn't surprise me at all if the shorthand to omit them gets rejected by the SC (or at least moved to the "Deferred Ideas" section).
What other syntax do we have that binds to the right without "as"? Allowing it at all is only as old as with statements, but that's still more precedence than ":" and "=" have for being used that way.
None at the moment, but that's not my point. My point is that patterns are more like expressions than statements, and all current uses of "as" for binding are in statements. Expressions typically use punctuation rather than keywords, so there's no compelling argument that the syntax for "binding the RHS" *in expressions* has to be "as".
But PEP 634 uses "PTRN as NAME" as well. The only difference in this regard between the two PEPs is that 634 allows "_ as NAME" to be shortened to just "NAME" (omitting both the pattern and the keyword), while 642 only allows "__ as NAME" to be shortened to "as NAME" (omitting only the pattern). You say "we use as in statements, so let's re-use that", I say "we use
punctuation in expressions, and a keyword looks clumsy and verbose". I don't believe that point has ever been addressed directly, which is why I get frustrated that arguments about "as" seem to be getting redirected into arguments about binding to the right all the time.
I didn't realise it was an argument you were trying to make. Cheers, Nick.
Hello, On Sun, 10 Jan 2021 12:08:05 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote: []
PEP 634 doesn't have that feature for class patterns in general, only for classes like data classes, where the constructor signature is carefully aligned with the match arguments.
You see, if PEP622/PEP634 contained clause like: "class patterns may be applied only to namedtuple types (and their subclasses) and dataclasses (and their subclasses)", then I personally would be happy. But if PEP622 draft would have contained such a clause, it would be removed before publishing anyway, because: 1. Enforcing such a check is more trouble/overhead. 2. There's no need to police users such deep, e.g. what if someone writes a class which has API *like* namedtuple/dataclasses, without formally being one? And that's the point. If you have a class: class MyCls: def __init__(self, val, nm): self.val = val; self.name = nm , and you normally use it like: MyCls(nm="foo", val=123), then when you'll try to use it with pattern matching, you'll immediately see what's wrong with your class: you need to use the same name for constructor arg and a related attribute (if you don't want to see funky discrepancies). To come to that, you don't need to read anything, it'll come naturally. In full accordance with Python approach of "gentle, but exciting (not depressing) and eventually high, learning curve", where next step is naturally guided by your previous steps. And of course, for people who prefer to read, PEP634/etc. in "informational" part should have a recommendation like "Class pattern matching syntax best works with namedtuples and dataclasses, or classes which follow their design pattern of "no data hiding", e.g. names of attributes should match names of constructor arguments, as both are public interface of such a class".
For positional args, PEP 642 would also align in those carefully constructed cases, so it's only the new syntax for attribute matching where things differ.
And that's the biggest trouble with PEP642. "Conflating set syntax with dict semantics" is an easy case (to both see and fix), that's why I talked about it first. Introducing 2nd syntax to match against objects, so, there're two syntaxes to use against objects, and one of them is completely new to Python - that's the real problem of PEP642. And I patiently continue this thread, hoping that people whose argument would be along the lines of "I teach Python, and I don't want to teach my students 2 ways of doing the same thing, and which way use when. Why, if PEP634 offers just one way?" -- Best regards, Paul mailto:pmiscml@gmail.com
On Sun, 10 Jan 2021, 7:37 pm Paul Sokolovsky, <pmiscml@gmail.com> wrote:
And I patiently continue this thread, hoping that people whose argument would be along the lines of "I teach Python, and I don't want to teach my students 2 ways of doing the same thing, and which way use when. Why, if PEP634 offers just one way?"
They don't do the same thing, though. One does traditional duck typing (checking for the presence of a specified set of attributes on an object), while the other matches a sequence of attributes specified by the class. PEP 634 just conflates the two tasks into a single call-like syntax that may or may not bare any resemblance to the type's constructor signature. Cheers, Nick.
Hello, On Tue, 12 Jan 2021 00:11:33 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
On Sun, 10 Jan 2021, 7:37 pm Paul Sokolovsky, <pmiscml@gmail.com> wrote:
And I patiently continue this thread, hoping that people whose argument would be along the lines of "I teach Python, and I don't want to teach my students 2 ways of doing the same thing, and which way use when. Why, if PEP634 offers just one way?"
They don't do the same thing, though. One does traditional duck typing (checking for the presence of a specified set of attributes on an object), while the other matches a sequence of attributes specified by the class.
Yes, as I mentioned, I grokked the "construction" you made in your PEP, and under given premises, it's a neat construction. But I don't think that premises under which you did that are "right". And from point of view of an average Python programmer, they both apply to objects, so from that PoV, they "do the same thing".
PEP 634 just conflates the two tasks into a single call-like syntax that may or may not bare any resemblance to the type's constructor signature.
But pattern matching concept in the first place conflates "instance-of" checks and usual one-by-one comparisons of contained values with some references, and extraction of other contained values. That's how we treated the matter for decades - we knew that on the other side, the esoteric functional languages use "pattern matching". And we smiled at that, and told ourselves that our imperative languages are more expressive - by combining our individual "isinstance" and scalar comparisons we can easily achieve the same effect as pattern matching, but also tons of other effects. Fast forward, and we have slightly adjusted definition of "expressivity" - it's not "when you can do anything with low-level means, combining them verbosely", it's "when you can do practically useful things concisely/without extra verbosity". In that regard, PEP642: 1. Steals some generality/expressivity from PEP634, limiting "Cls(a, b)" style patterns to only classes which define __match_args__. 2. Hastily adds new pattern matching syntax, "obj{.a, .b}". But PEP634 forecasts that beyond patterns which it describes, it's easy to imagine that over time, more of useful patterns can be added (up to user-defined patterns). It just says that it doesn't want to haste with trying to defining those right away, instead tries to lay the grounds for the pattern matching per se.
Cheers, Nick.
-- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Sat, 9 Jan 2021 14:49:19 +0100 Antoine Pitrou <antoine@python.org> wrote:
On Sat, 9 Jan 2021 12:17:32 +0000 Paul Moore <p.f.moore@gmail.com> wrote:
On Sat, 9 Jan 2021 at 10:52, Paul Sokolovsky <pmiscml@gmail.com> wrote:
case {"host" as host, "port" as port}:
There're 2 obvious problems with it:
a) In Python, {} with things inside it, but no ":" inside it, is a set, set. b) Everywhere else in Python, thing on the left of "as" gets into thing on the right of "as", behold:
import foo as bar # original module "foo" gets into "bar". with Cls(a, b) as c: # original expression "Cls(a, b)" gets into "c"
Then based on the existing Python syntax, the meaning of '{"host" as host, "port" as port}' is: a set, whose contained, constant in this case, values "host" and "port" get captured as variables host and port. In pattern matching context (with 'case' in front), it means: match using a set, check for presence on constant "host" and "port" elements, and capture those constants to variables host and port (that capturing doesn't make much sense, yeah. And yet syntax tells just that. You propose to assign to it completely different meaning.).
This analysis nicely captures my reservations with the proposal here. It claims to be addressing the problems with PEP 634, where the syntax proposed there is specialised to matching, but it then proceeds to introduce another syntax, which is *also* unlike existing usage, just in different ways.
Introducing a new syntax is not a problem, it's the solution. The problem with PEP 634 is precisely that it reuses existing syntax and gives it a different meaning, thereby producing confusion.
It's erroneous to claim that `{"host" as host, "port" as port}` is a set. It's currently invalid syntax due to the `as`:
{"host" as host, "port" as port} File "<stdin>", line 1 {"host" as host, "port" as port} ^ SyntaxError: invalid syntax
So, opposing Nick's proposal on the basis that it "looks like a set" is just like opposing set literals on the basis they they "look like a dict".
No, it's different kind of opposition. Consider for example the syntax "2+2". One can say that it doesn't make sense, everyone knows that two plus two is four, so everyone would write "4" right away. So, let's take "2+2" and assign to it a different meaning, let's say that it means "5". Or let's say that it means a dict. That's exactly what PEP642 does - it takes set *syntax* and appropriates it for dict *semantics*, hence the opposition. Compare that to what PEP634 does: it takes tuple syntax, and defines sequence pattern matching based on it; it also takes dict syntax and defines mapping pattern matching based on it; finally, it takes object construction syntax, and defines object pattern matching syntax based on it. In all 3 cases, PEP634 defines *new semantics* based on *existing syntax* (by putting it into new higher-level syntactic construct, the "case" clause), and does so in consistent way. PEP634 later could be extended with set patterns, e.g.: match [1, 2]: case {1, 2}: # matches match [1, 2]: case {2, 1}: # also matches match {1, 2}: case {2, _ as another}: # matches and assigns 1 to 'another' Whereas again, PEP642 tries to appropriate such set pattern syntax for completely different purpose, only because its author thought that it's ok in the mapping pattern syntax to suddenly drop ":" operator. -- Best regards, Paul mailto:pmiscml@gmail.com
On Sat, 9 Jan 2021, 8:50 pm Paul Sokolovsky, <pmiscml@gmail.com> wrote:
The key difference relative to PEP 634 is that even when the code author uses the shorthand form, *readers* will still get at least the "as" keyword as a prompt,
Ok, so let's summarize the alternatives:
1. PEP634, which says that "case Cls(a, b):", etc. is not worse than other languages. 2. PEP642, which says that taking existing "as" and using it (and it's also an English word with a specific meaning) with rather different meaning in pattern matching is ok.
I don't follow this objection. It's being used the same way it is used elsewhere in Python: binding a name given on the right based on information given on the left. The shorthand to omit "__" is the only novelty that doesn't strictly follow that precedent. 3. Alternative proposal sounded on the list, which says that taking
existing punctuational operator like ">" or "->" (and they're also graphemes depicturing arrows) and using them (still with rather different meaning) in pattern matching is ok.
PEP642 never mentions 3rd alternative.
Because PEP 634 doesn't propose it either. And it actually starts its
presentation with things like:
case {"host" as host, "port" as port}:
There're 2 obvious problems with it:
a) In Python, {} with things inside it, but no ":" inside it, is a set, set. b) Everywhere else in Python, thing on the left of "as" gets into thing on the right of "as", behold:
import foo as bar # original module "foo" gets into "bar". with Cls(a, b) as c: # original expression "Cls(a, b)" gets into "c"
Then based on the existing Python syntax, the meaning of '{"host" as host, "port" as port}' is: a set, whose contained, constant in this case, values "host" and "port" get captured as variables host and port. In pattern matching context (with 'case' in front), it means: match using a set, check for presence on constant "host" and "port" elements, and capture those constants to variables host and port (that capturing doesn't make much sense, yeah. And yet syntax tells just that. You propose to assign to it completely different meaning.).
Yep, this is why I'd be entirely OK with the SC saying they'd prefer the colon to be mandatory, even when parsing is unambiguous without it. I personally like the idea of being able to leave it out as redundant, but I definitely agree that the omission has the potential to make mapping patterns harder to interpret for readers.
So, the "dict pattern shortcut syntax" proposed by PEP642 is rather ambiguous and confusing, and represent climax of PEP642's "fixing too many things at once", where it goes over the summit and onto the dark side.
And even looking at:
case {"host": as host, "port": as port}
I'm personally tripped by the meaning of "as" in English, and think that it does something about thing on the left of it, i.e. the dictionary *key*, not the *value*, as it really does.
It is as closely related to the key as the binding in with statements is to the context manager expression (in mapping patterns, it would bind the result of looking up the key, in with statements it binds the result of the __enter__ method).
rather than having to just know that "name" appearing in a pattern means "__ as name",
But PEP634 doesn't have no "__ as name"! It has "_ as name". And that's another case of PEP642's "too much at once". While that change is pretty clearly presented in the PEP642, I find that discussion here rather downplays it. Specifically, you use "__" even in the context talking about PEP634, as if you take it for granted. Instead, each time you mention "__", you'd rather say "which is another alternative syntax PEP642 propose".
Because you see, I'm almost sure that readers of just this thread don't even pay enough attention that you use double-underscore instead of the original underscore. That leads us to the obvious concern:
1. The difference between "__" and "_" isn't visible enough.
People should start saying "OMG" and "horror!" now, not when PEP642 gets implemented and they finally notice that they need to type _ twice.
Which leads us to:
2. Both PEP642 and discussion here should elaborate explicitly what happens when people still use "_" in patterns.
It's a syntax error, because PEP 642 patterns don't allow the use of bare names. So, you see, it says "many people prefer __", and then immediately says
"linters recognize _ as a special case". So, 10 years later, how many linters recognize double-underscore as a special case too?
pylint does. I haven't checked the others, as pylint is the one I typically use.
Other general comments on PEP642v3 text:
* allow an initial form of pattern matching to be developed and released without needing to decide up front on the best default options for handling bare names, attribute lookups, and literal values
Can this be made more explicit with appending: " (that's why it proposes to prefix both capture patterns and value constraints, even though prefixing just one would be enough to avoid ambiguity)".
I'd use "parsing ambiguity" at the end, but I agree that would be a useful clarification.
MatchValue(matchop op, expr value) matchop = EqCheck | IdCheck
Why bloat ASDL with duplicates? Why not "matchop = Eq | Is" ?
Because that would result in a name conflict in the generated C code.
People should just learn pattern matching as it's presented in other languages indeed. To help them with "may bind to the right" matter, a very simple, focused change was proposed - to add explicit arrow pointing straight at the bind/capture target.
I never liked that proposal, and it has absolutely zero to do with the origins of PEP 642 (remember, the PEP started out *not* qualifying name bindings, the same as PEP 634). If you'd like to present specifically that proposal to the SC, then you're going to have to prepare your own PEP. Instead of that simple
measure, PEP642 builds whole parallel hierarchy of concepts, which is very artificial and complex response ("seeing complication, rehash everything, and add significantly more overall complexity to deal with originally simple case").
Again, I pray for PEP642 rejection, based on its "attempting to do too much, and overdoing it to "remedy is worse than the problem" situation, and trying to build complex hierarchies to artificially separate concepts in pattern matching, instead of treating pattern matching as "cartesian product" of simple concepts (and see how to make these simple concepts more explicit to humans, rather than adding more constraints for humans to be aware of).
Deep and unobvious constraints, how on earth I know if some class defines __match_args__ or not, so I can select appropriate syntax?
Instance attribute matching would work on any class. It's only positional matching that classes would have to opt in to by telling the interpreter what it means for instances of that class. As for how you know whether a class supports custom matching, you'd find out the same way you find out if it can be called or used as a context manager: documentation, examples, and runtime introspection tools (e.g. class help in IDEs) (Note: PEP 634 has the same distinction, where only classes that define __match_args__ can accept positional patterns in cases) I
simply will use the syntax which is usable for all cases (and that's the ugly one), and the other syntax will be used only for builtin types, and whoever will see it used for a non-builtin class will be shocked and confused.
Why would they be any more shocked and confused by that than they would by classes defining custom constructors that take positional arguments, or implementing the context management protocol? Cheers, Nick.
[]
-- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Sun, 10 Jan 2021 01:42:25 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
On Sat, 9 Jan 2021, 8:50 pm Paul Sokolovsky, <pmiscml@gmail.com> wrote:
The key difference relative to PEP 634 is that even when the code author uses the shorthand form, *readers* will still get at least the "as" keyword as a prompt,
Ok, so let's summarize the alternatives:
1. PEP634, which says that "case Cls(a, b):", etc. is not worse than other languages. 2. PEP642, which says that taking existing "as" and using it (and it's also an English word with a specific meaning) with rather different meaning in pattern matching is ok.
I don't follow this objection. It's being used the same way it is used elsewhere in Python: binding a name given on the right based on information given on the left.
Where's "left" in the case of your proposed syntax: case [as a, as b]: ? There's no "left". And that's the whole point, in your PEP642 proposal, "as" applies to implicitly given term, encoded by position in the match pattern. That's ... unusual. And ... confusing, at least at first. As I said, maybe we could get used to it, but I always bring that matter in comparison to the alternative syntax, which IMHO avoids confusion implied by both English and current Python meaning of "as": case [>a, >b]:
The shorthand to omit "__" is the only novelty that doesn't strictly follow that precedent.
3. Alternative proposal sounded on the list, which says that taking
existing punctuational operator like ">" or "->" (and they're also graphemes depicturing arrows) and using them (still with rather different meaning) in pattern matching is ok.
PEP642 never mentions 3rd alternative.
Because PEP 634 doesn't propose it either.
Well, PEP622 did have it: https://www.python.org/dev/peps/pep-0622/#alternatives-for-constant-value-pa... , quoting (last bullet point there):
There was also an idea to make lookup semantics the default, and require $ or ? to be used in capture patterns:
Then we know the issues with PEP622/PEP634, that requests/proposals for various clarifications to them were denied. That's how PEP642 was born, according to its own text. The more strange that it continues that cycle of hushing/avoiding mentioning alternative ideas.
And it actually starts its
presentation with things like:
case {"host" as host, "port" as port}:
There're 2 obvious problems with it:
a) In Python, {} with things inside it, but no ":" inside it, is a set, set. b) Everywhere else in Python, thing on the left of "as" gets into thing on the right of "as", behold:
import foo as bar # original module "foo" gets into "bar". with Cls(a, b) as c: # original expression "Cls(a, b)" gets into "c"
Then based on the existing Python syntax, the meaning of '{"host" as host, "port" as port}' is: a set, whose contained, constant in this case, values "host" and "port" get captured as variables host and port. In pattern matching context (with 'case' in front), it means: match using a set, check for presence on constant "host" and "port" elements, and capture those constants to variables host and port (that capturing doesn't make much sense, yeah. And yet syntax tells just that. You propose to assign to it completely different meaning.).
Yep, this is why I'd be entirely OK with the SC saying they'd prefer the colon to be mandatory, even when parsing is unambiguous without it.
That's great, thanks for considering it! But in the formal review process what we have is: 1) It's v3 of PEP642, which still has such an issue. 2) Its author doesn't admit that there's objective ambiguity between mapping pattern syntax in PEP642 and would-be set patterns, and treat that issue as subjective, saying that he'd make a change if SC asks for it, not because he has seen the issue with it, based on the community feedback.
I personally like the idea of being able to leave it out as redundant, but I definitely agree that the omission has the potential to make mapping patterns harder to interpret for readers.
So, the "dict pattern shortcut syntax" proposed by PEP642 is rather ambiguous and confusing, and represent climax of PEP642's "fixing too many things at once", where it goes over the summit and onto the dark side.
And even looking at:
case {"host": as host, "port": as port}
I'm personally tripped by the meaning of "as" in English, and think that it does something about thing on the left of it, i.e. the dictionary *key*, not the *value*, as it really does.
It is as closely related to the key as the binding in with statements is to the context manager expression (in mapping patterns, it would bind the result of looking up the key, in with statements it binds the result of the __enter__ method).
Sorry, we talk about surface syntax here, not underlying semantics. And syntax of: {"host" as host} isn't consistent with with open("foo") as f: If you wanted to expose deeper semantics with the first syntax, you would probably come up with something like {["host"] as host} - that's truly "expliticize" the fact "it would bind the result of looking the key", ain't it? But all that is just going deeper down the rabbit whole. Because the right syntax *and* meaning, consistent with sequence patterns is: {"host": as host} where "as host" stands in for the *value* in dict entry corresponding to a key, and the fact which you bring up - that it would be looked up by that key - is irrelevant implementation detail, as far as *syntax* is concerned.
rather than having to just know that "name" appearing in a pattern means "__ as name",
But PEP634 doesn't have no "__ as name"! It has "_ as name". And that's another case of PEP642's "too much at once". While that change is pretty clearly presented in the PEP642, I find that discussion here rather downplays it. Specifically, you use "__" even in the context talking about PEP634, as if you take it for granted. Instead, each time you mention "__", you'd rather say "which is another alternative syntax PEP642 propose".
Because you see, I'm almost sure that readers of just this thread don't even pay enough attention that you use double-underscore instead of the original underscore. That leads us to the obvious concern:
1. The difference between "__" and "_" isn't visible enough.
People should start saying "OMG" and "horror!" now, not when PEP642 gets implemented and they finally notice that they need to type _ twice.
Which leads us to:
2. Both PEP642 and discussion here should elaborate explicitly what happens when people still use "_" in patterns.
It's a syntax error, because PEP 642 patterns don't allow the use of bare names.
Good. Please consider adding explicit paragraph describing that (in detail whenever possible, hence paragraph.) That would be important argument for the people wary of confusion "__" may bring.
So, you see, it says "many people prefer __", and then immediately says
"linters recognize _ as a special case". So, 10 years later, how many linters recognize double-underscore as a special case too?
pylint does. I haven't checked the others, as pylint is the one I typically use.
I'd recommend to add that info to the PEP. Again, it strengthens the cause, showing that it has some prior art, not just adhoc "original design" made up on spot. []
People should just learn pattern matching as it's presented in other languages indeed. To help them with "may bind to the right" matter, a very simple, focused change was proposed - to add explicit arrow pointing straight at the bind/capture target.
I never liked that proposal, and it has absolutely zero to do with the origins of PEP 642 (remember, the PEP started out *not* qualifying name bindings, the same as PEP 634).
I have my conceptual model of origins of PEP642: there were calls on the list from various people to consider "sigil" alternatives to PEP634: a) to add sigils for value patterns; alternatively b) to add sigils to capture patterns. You picked the idea "a)", which was great rejoice. Sadly, nobody paid enough attention to the idea b), though the whole [my] idea was to contrast the *two*.
If you'd like to present specifically that proposal to the SC, then you're going to have to prepare your own PEP.
I don't think I'm qualified to write a PEP. How I treat it is that if I'm the only one who finds it an interesting choice, it's already lost. You ended up going the same "sigil for capture patterns" way, except you chose word "as" as a sigil, instead of punctuation like ">" or "->". In that regard, it would be instructive if your PEP elaborated why "you never liked punctuation proposal", how is it much different from "word as sigil" up to you *never* liking it.
Instead of that simple
measure, PEP642 builds whole parallel hierarchy of concepts, which is very artificial and complex response ("seeing complication, rehash everything, and add significantly more overall complexity to deal with originally simple case").
Again, I pray for PEP642 rejection, based on its "attempting to do too much, and overdoing it to "remedy is worse than the problem" situation, and trying to build complex hierarchies to artificially separate concepts in pattern matching, instead of treating pattern matching as "cartesian product" of simple concepts (and see how to make these simple concepts more explicit to humans, rather than adding more constraints for humans to be aware of).
Deep and unobvious constraints, how on earth I know if some class defines __match_args__ or not, so I can select appropriate syntax?
Instance attribute matching would work on any class. It's only positional matching that classes would have to opt in to by telling the interpreter what it means for instances of that class.
That's what I was saying, yes.
As for how you know whether a class supports custom matching, you'd find out the same way you find out if it can be called or used as a context manager: documentation, examples, and runtime introspection tools (e.g. class help in IDEs)
The matter, most of us don't write *new* context managers that often. I do write them, but mostly as wrappers for existing context managers. I don't remember signature for __exit__. How I write it is: def __exit__(self, *args): return self.wrapper.__exit__(*args) That all is supposed to be different for matter matching, as it's supposed to be used with multitude of classes we have to represent data objects. And most people won't bother to lookup/remember __match_args__, they will [have to] use obj{.attr} syntax. But they will see Cls(attr) syntax in other people's code and in regard to objects coming from 3rd-party libs, and will be in permanent state of confusion regarding what chasm lies between their own classes and those libs'. If you says that will cause them to learn the difference, then yes, on a free day of Xmas holidays they will look it up, tweet their amazement that someone did such a thing to the language, and then safely forgot about it, just swearing about foo{.bar} vs foo(bar) pattern matching dichotomy once in a while.
(Note: PEP 634 has the same distinction, where only classes that define __match_args__ can accept positional patterns in cases)
But classes which don't define it, still use the same syntax, just with "keywords for attributes instead of positional args", instead of completely unheard syntax of obj{.attr1, .attr2} which you propose! And all that because you insist that: host, port = addr{.host, .port} is anyhow better than: Addr(host = >host, port = >port) = addr
I
simply will use the syntax which is usable for all cases (and that's the ugly one), and the other syntax will be used only for builtin types, and whoever will see it used for a non-builtin class will be shocked and confused.
Why would they be any more shocked and confused by that than they would by classes defining custom constructors that take positional arguments, or implementing the context management protocol?
Cheers, Nick.
-- Best regards, Paul mailto:pmiscml@gmail.com
On Mon, 4 Jan 2021, 2:19 am Barry Scott, <barry@barrys-emacs.org> wrote:
I quickly read 642 v3 and missed an explanation about why the syntax to match a string object is str{} and not str. Are you saying that I MUST use {} so that when case is parsed its clear that its a class with no constraints?
Yes. "str{}" would give a pure subclass check, "str()" would also be allowed for classes that define "__match_args__".
in the "Changes to class patterns" I read the BinaryOp example and I thought from the above that it would also use {} and not ().
---
match expr:
case BinaryOp(== '+', as left, as right):
---
I was expecting to see:
---
match expr:
case BinaryOp{== '+', as left, as right}:
---
The example in the abstract doesn't show a class defined pattern that relies on __match_args__, only an instance attributes pattern. Cheers, Nick.
participants (8)
-
Antoine Pitrou -
Antoine Pitrou -
Barry Scott -
Ethan Furman -
Joseph Martinot-Lagarde -
Nick Coghlan -
Paul Moore -
Paul Sokolovsky