Hi,
Since a decision on PEP 634 is imminent, I'd like to reiterate some
concerns that I voiced last year.
I am worried about the semantics and implementation of PEP 634.
I don't want to comment on the merits of pattern matching in general, or
the proposed syntax in PEP 634 (or PEP 640 or PEP 642).
Semantics
---------
1. PEP 634 eschews the object model, in favour of adhoc instance checks,
length checks and attribute accesses.
This is in contrast to almost all of the the rest of the language, which
uses special (dunder) methods:
All operators,
subscripting,
attribute lookup,
iteration,
calls,
tests,
object creation,
conversions,
and the with statement
AFAICT, no justification is given for this.
Why can't pattern matching use the object model?
No one has said that "we can't". It's just that "we don't have to". The underlying mechanisms used by pattern matching (instance check, length, attribute access) already have their defined protocols and support within the object model. It's analogous as the way in which iterable unpacking didn't need to define it's own object model special methods, because the existing iteration mechanism used in for loops was sufficient.
PEP 343 (the "with" statement) added the __enter__ and __exit__ methods
to the object model, and that works very well.
2. PEP 634 deliberately introduces a large area of undefined behaviour
into Python.
https://www.python.org/dev/peps/pep-0634/#side-effects-and-undefined-behavior
Python has, in general, been strict about not having undefined behaviour.
Having defined semantics means that you can reason about programs, even
non-idiomatic ones.
[This is not unique to Python, it is really only C and C++ that have
areas of undefined behaviour]
The C standard uses a very peculiar definition of "undefined behaviour" (I'm not familiar with the C++ one to assert anything, I'll assume it's the same), where for certain set of programs, any resulting behaviour is valid, even at compile time (so a compiler that deletes all your files when trying to compile "void f() {int a[10]; a[10]=0;}" is standard compliant). Comparing that with the use of the term "undefined behaviour" in the PEP is not very useful, because even if they are the same words, they don't have the same meaning
If you want to compare it with the C standards, the term we'd use would be "implementation defined behaviour". Python has a lot of those. For example, the output of all these python programs can change between implementations (And some of these even change between versions of cpython):
- print({3,2,1})
- print(hash("hello"))
- if id(1) == id(1): print("hello")
- import sys; print(sys.getsizeof([]))
Some order of operations is also implementation dependent for example in
def foo():
print(open("/etc/passwd")).read()
foo()
The moment where file closing happens is implementation dependent.
The section you linked to introduces behaviour in similar lines to all of the above.
I can see no good reason for adding undefined behaviour. It doesn't help
anyone.
It helps for the point 3 that you mention (See below)
The lack of precise semantics makes programs harder to understand, and
it makes the language harder to implement.
If the semantics aren't specified, then the implementation becomes the
specification.
This bakes bugs into the language and makes it harder to maintain,
as bug-for-bug compatibility must be maintained.
3. Performance
PEP 634 says that each pattern must be checked in turn.
That means that multiple redundant checks must be performed on (for
example) a sequence if there are several mapping patterns.
This is unnecessarily slow.
What PEP 634 says about this(unless we're reading different sections, a quote could help here) is that the semantics are the one of selecting the first case block whose patterns succeeds matching it and whose guard condition (if present) is "truthy".
As long as the semantics are respected, a smart compiler could reduce some of the redundant checks (and that's *precisely* why the order of the checks is left as implementation dependent).
It's important here to separate "sequential semantics" vs "implemented as sequential checks". For an analogy, look at the "serializable" concept in databases, where the outcome of multiple statements must be equal to the outcome of operations executed serially, but in practice databases can find more efficient non-serial executions with the same outcome. The PEP specifies semantics, not how the implementation must behave.
Implementation
--------------
My main concern with the implementation is that it does too much work
into the interpreter.
Much of that work can and should be done in the compiler.
For example, deep in the implementation of the MATCH_CLASS instruction
is the following comment:
https://github.com/brandtbucher/cpython/blob/patma/Python/ceval.c#L981
Such complex control flow should be handled during compilation, rather
than in the interpreter.
Giant instructions like MATCH_CLASS are likely to have odd corner cases,
and may well have a negative impact on the performance of the rest of
the language.
It is a lot easier to reason about a sequence of simple bytecodes, than
one giant one with context-dependent behaviour.
We have spent quite a lot of effort over the last few years streamlining
the interpreter.
Adding these extremely complex instructions would be a big backward step.
What you say about this sounds very reasonable to me, although I don't understand who it is a "Concern with PEP 634". The implementation is not part of the PEP, and I'm pretty sure all parties here (the PEP authors, the PEP critics, the SC) would be more than happy to see this implementation improving.
I'm pretty sure we both have presented these ideas before so I'm not expecting any of us to change our minds at this point, but I'm leaving this reply here for future reference in case other people want to have the different positions on this discussion
Best,
D.