Hi Paul,

Thank you for your comments on the DLS'20 paper.  I am glad to hear that it helps paint a clear(er) picture of pattern matching in Python.  However, please let me set the record straight in a few regards.

First, there is no 'shoehorning' or 'deception' in our pattern matching PEPs.  This notion of our team standing against the Python community or trying to outsmart it is quite absurd and unnecessary, really.  At the end of the day, you might find that the PEPs are a boiled-down version of months of intense work and discussions (as Brandt has already pointed out elsewhere) and thus necessarily terse in some parts.  Pattern matching is a complex subject and the PEPs have been written with a sincere effort to convey the big picture and give a tour of this feature in an accessible manner.  Our entire discussion is all openly available to anyone interested.

Second, our work on pattern matching is based on a very simple premise/idea: *bring pattern matching to Python*.  In particular, it is *not* to 'bring Python to pattern matching'.  We want to introduce pattern matching as a new feature to Python as Python stands right now, without the need to first modify Python and make it more static, C-like, functional or whatever.  As I have mentioned before, pattern matching is an opt-in feature, something with absolutely no influence on the behaviour of any Python program you have ever written before.  Not introducing constants or a completely new kind of scopes to Python first is certainly not a lack of academic rigour, oversight on our part because we are not smart enough, or trick to coerce people into accepting pattern matching.  It just means that we propose to introduce pattern matching to Python as it is right now.

Third, the academic paper has a different scope than the PEPs.  In case you missed it: after the first version of PEP 622, we listened to the reactions and suggestions from the community and went back to overhaul the entire design and build as much as possible on the raised concerns and cummulative experience of the community.  One major outcome of this was to reduce the overall proposal to the absolute core features of what we need for pattern matching (another was to put more effort into writing an accessible account of what we propose and why we think the various design features are a good choice, leading to three PEPs with different aims).  The academic paper outlines a much larger scope of possibilities than the PEPs, whereas the PEPs are more grounded in the pragmatic and taking one step at a time.  In short, the academic paper lays out an entire journey, a vision, whereas the PEPs propose a first step forward.

Finally, the reason why the academic paper has not been referred to before is simply that it will officially be published coming (!) week.  Since the process of peer review is now complete, we can already provide a preprint.  It was clear from the outset that the PEPs will contain a link to the paper as soon as it becomes publicly available.

Of course, we encourage anyone interested in it to read the academic paper.  Since its focus is somewhat complementary to the PEPs, it might highlight some ideas behind our design that might not be made clear enough in the PEPs.  But while doing so, please keep in mind that the paper is a vision of a bigger picture and as such distinct from the PEPs!

Kind regards,
Tobias



Quoting Paul Sokolovsky <pmiscml@gmail.com>:

Hello,

As was mentioned many times on the list, PEP634-PEP636 are thoroughly
prepared and good materials, many thanks to their authors. PEP635
"Motivation and Rationale" (https://www.python.org/dev/peps/pep-0635/)
stands out among the 3 however: while reading it, chances that you'll
get a feeling of "residue", accumulating a section over section. By the
end of reading, you may get a well-formed feeling that you've read a
half-technical, half-marketing material, which is intended to "sell" a
particular idea among many other very viable ideas, by shoehorning some
concepts, downplaying other ideas, and at the same time, light-heartedly
presenting drawbacks of its subject one.

Just to give one example, literally at the very beginning, at the
"Pattern Matching and OO" section (3rd heading) it says:

Pattern matching is complimentary to the object-oriented paradigm.

It's not until the very end of document, in the "History and Context" it
tells the whole truth:

With its emphasis on abstraction and encapsulation, object-oriented
programming posed a serious challenge to pattern matching.

You may wonder how "complimentary" and "posed a serious challenge"
relate to each other. While they're definitely not contradictory,
starting the document with light-hearted "complimentary" can be seen as
trying to set the stage where readers don't pay enough attention to the
problem. And it kinda worked: only now [1] wider community discovers the
implications of "Class Patterns" choices. (As a note, PEP635 does well
on explaining them, and I'm personally sold on that, but it's *tough*
choice, not the *obvious* choice).

There're many more examples like that in the PEP635, would take too
much space to list them all. However, PEP635 refers to the paper:

Kohn et al., Dynamic Pattern Matching with Python
https://doi.org/10.1145/3426422.3426983 (Accepted by DLS 2020. The
link will go live after Nov. 17; a preview PDF can be obtained from
the first author.)

As that citation suggests, the paper is not directly linked from the
PEP635. But the preprint is now accessible from the conference page,
https://conf.researchr.org/home/dls-2020?#event-overview (direct link
as of now: https://gvanrossum.github.io//docs/PyPatternMatching.pdf).

That paper is written at much higher academic standard, and a pleasure
to read. I recommend it to everyone who read PEP635 (note that it was
written with PEP622 in mind, but conceptual differences are minor). With
it, I noticed just 2 obvious issues:


Section 4.3. Named Constants

It would clearly be desirable to allow named constants in patterns
as a replacement and extension of literals. However, Python has no
concept of a constant, i.e. all variables are mutable (even where
the values themselves are immutable).

So, unlike PEP635, the paper pinpoints right the root of PEP634's
problems: lack of constants in Python (on the language level). This is
just the point which was raised on the mailing list either
(https://mail.python.org/archives/list/python-dev@python.org/message/WV2UA4AKXN5PCDCSTWIUHID25QWZTGMS/).

Under strict academic peer review, the paper would have been returned
for elaboration, with a note like: "Given that nowadays many dynamic
languages (PHP, JavaScript, Lua, etc.) have support for constants, and
lack of constants poses a serious usability challenge to your proposal,
please explain why you chose to proceed anyway (and apply workarounds),
instead of first introducing the concept of constants to the language.
(Given that amount of work to implement pattern matching is certainly
an order of magnitude larger than to introduce constants)."

But the paper wasn't returned for elaboration, so we'll keep wondering
why the authors chose such a backward process.


Section 6.1. Scope

The granularity of the scope of local variables is at the level of
functions and frames. [...]
The only way to restrict the scope of a variable to part of a
function’s body (such as a case clause) would be to actively delete
the variable when leaving the block. This would, however, not restore
any previous value of a local variable in the function’s scope.

This is a misconception ("the only way") which is repeated almost one
to one on PEP635 either. If anything, the above describes how
pseudo-scoping is currently implemented for exception vars in "except
Exception as e:" clause (more info:
https://mail.python.org/pipermail/python-dev/2019-January/155991.html),
which is hardly a "best practice", and definitely not the only way.

How to support multiple variable scopes in one stack frame is not a
rocket science at all. One just need to remember how C did that since
~forever. And that's: (for unoptimized programs) variables in different
scopes live in disjoint subsections of a frame. (For optimized
programs, variables with disjoint liveness can share the same
locations in a frame).

The only reasons for not implementing the same solution in Python would
be intertia of thought and "but it's not done anywhere else in
Python". Yeah, but nobody del'eted local variables behind users' backs
either, before somebody started to do that for exception clause
variables. And the whole of pattern matching is such that either one
thing, or another, but will be done for the first time in Python. For
example, nobody before could imagine that one can write "Point(x, y)",
and get x and y assigned, and now we're facing just that [1]. (Again,
I personally love it, though think that "Point(>x, >y)" is an
interesting option to address the tensions).


In that regard, the current PEP634 and friends miss too many interesting
and useful opportunities (constants in language core and true scoping
for match'es, to name a few). Well, that happens. But they try to
shoehorn too much of "we didn't do" into "it's not possible" or "it
doesn't make sense", or "let's workaround it in adhoc ways" and that
raises eyebrows, leading to concerns of whether the proposals are
actually "raw" as of yet.



[1] People expressing surprise at "Class Patterns" syntax:
https://mail.python.org/archives/list/python-dev@python.org/message/F66J72JUEAUKBM5VDSMG4HRHUEQBWI5M/
https://mail.python.org/archives/list/python-dev@python.org/message/Q2ARJULLJG6HRCDXR4SSA7K6NLTOPUL7/


--
Best regards,
Paul                          mailto:pmiscml@gmail.com
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EQX6GK7CZVW5NS42VYJKQ3WOPEXPU5CD/Code of Conduct: http://python.org/psf/codeofconduct/