On Mon., 28 Sep. 2020, 1:22 am Marco Sulla, wrote:

I like this, but IMHO adding a character at the end of the macro name (the exclamation mark), lowers readability. I'd prefer a character at the start of the macro, a character that is not used as an unary operator.

For usage, whether something is a macro or not should either be irrelevant (when they're used as a more powerful function call or decorator), or else entirely obvious from the way you use it (when they're defining a new pseudo-statement), so it doesn't make sense to emphasize the syntactic marker too much. Putting the marker gives the compiler the info and reader the info they need without being too obtrusive (and also matches the way Rust macros are used). Using "!" as a prefix operator is also effectively already claimed by IPython for shell command execution, so I'd be surprised if we ever used that spelling for anything else. For the PEP itself, I'd like to see the bijection macro presented in the abstract to give context for the specification section. While I've had my doubts about macros in the past due to the inevitable evolution of different "dialects" of Python, I think we already hit that point years ago through sophisticated use of metaclasses and other features (Django code looks very different from NumPy code, for example). Judicious use of macros should offer opportunities to make domain specific dialects *easier* for new users to pick up, rather than harder. For the specification section, my main question/suggestion would be around sibling macros: how about calling those "decorator macros" and making the invocation syntax "@name!" rather than the bare "name!"? Cheers, Nick.

On Mon, Sep 28, 2020 at 7:23 AM Marco Sulla wrote:

...

For usage, whether something is a macro or not should either be irrelevant (when they're used as a more powerful function call or decorator), or else entirely obvious from the way you use it (when

On Mon, 28 Sep 2020 at 13:56, Nick Coghlan wrote: they're defining a new pseudo-statement), so it doesn't make sense to

...

emphasize the syntactic marker too much.

IMHO `import!` can be easily confused with `import`, even with a monospace font. An alternative is that macro must have a name different from keywords (so you will have macro_import!, for example), even if I'd prefer my first proposal (maybe something like %import).

I think the whole point is that import! won't be obvious unless you're looking for it, and those macros won't be obvious unless you're looking for them. They're meant to blend in rather than stand out.

My 2c as a Python user (mostly) and someone who dabbled in ES2020: The shouting syntax! does not sit well with me. The $hygenic is also cumbersome. To contrast, babel macros: * looks like regular code, without special syntax: existing tooling works, less mental strain * have access to call site environment, so not strictly hygienic(?): allow for greater expressive power I these the two points above really helped adopt babel macros in the js community and should, at the very least be seriously considered by the py community. Cheers, d. On Sat, 26 Sep 2020 at 21:16, Mark Shannon wrote:

Hi everyone,

I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/

All comments and suggestions are welcome.

Cheers, Mark _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U4C4XHNR... Code of Conduct: http://python.org/psf/codeofconduct/

On Sat, Sep 26, 2020 at 5:11 AM Mark Shannon wrote:

Hi everyone,

I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/

Speaking as a former C developer, why do "We need to let the community develop their own extensions"? What's insufficient about Python's current extensibility? The complexity of a language varies with the square of its feature count, and adding macros intended for creation of domain-specific language features balloons the feature count. Granted, they don't much increase the complexity of CPython, the language implementation, but the de facto language then becomes more than CPython - potentially a lot more. That is, making it easier to extend python means a proliferation of extensions with little thought given to their long term viability or cross-domain compatibility. It's kind of like zsh vs. bash. zsh has a smaller implementation, but a larger _language_. For this reason, I'm not terribly interested in zsh, but I like bash. On the other hand, bash has seen a proliferation of unnecessary extensions in recent years, so I may jump ship to something else someday - something with a smaller language, that isn't afraid of fork+exec. I've used m4 before as a macro system for Python+Cython, but I would never consider suggesting that m4 should become part of Python itself. IMO Perl is dying because of its exuberant design. One of the most important things a language designer has to do is say "no" sometimes.

On Fri, 16 Oct 2020 at 23:22, Guido van Rossum wrote:

Dima,

Do you have a link to "babel macros"? Searching for that brought up several different things; not being a frequent JS user I don't know how to filter these.

These links should help: https://babeljs.io/blog/2017/09/11/zero-config-with-babel-macros https://github.com/kentcdodds/babel-plugin-macros https://github.com/jgierer12/awesome-babel-macros That's a general intro, the code repo for the macro plugin, and a repo that lists implemented macros. Martin

I'm excited about the potential introduction of Lisp-style syntactic macros into Python. 😁 It's unclear to me whether PEP 638 ("Syntactic Macros") is still being actively developed, since the last activity I see on it is over a year ago (Sep & Oct 2020), but I thought I'd leave some initial comments I have anyway. Quoting from the PEP text... (1)

Lexical analysis ~~~~~~~~~~~~~~~~

Any sequence of identifier characters followed by an exclamation point (exclamation mark, UK English) will be tokenized as a ``MACRO_NAME``.

+1 to using the ! character to mark macros explicitly and loudly, since they are very powerful and can completely customize the syntax used inside them. I like the postfix syntax myself (as proposed) - so `macro_name!` is cool. I wouldn't be a fan of prefix syntax; `!macro_name` looks ugly. (2)

Statement form ~~~~~~~~~~~~~~

macro_stmt = MACRO_NAME testlist [ "import" NAME ] [ "as" NAME ] [ ":" NEWLINE suite ]

(2.1) What is `testlist`? It is not defined in this PEP, nor is it defined in the Python Grammar Specification [1]. Also, the `[ "import" NAME ]` and `[ "as" NAME ]` parts appear to be special-purpose syntax only useful for supporting the `from!` and `with!` macros mentioned later in the PEP. It feels odd that this syntax isn't more general-purpose. Perhaps the grammar could be made more general with something like:

macro_stmt = MACRO_NAME macro_expr_parameters ( NEWLINE | ":" NEWLINE suite ) macro_expr_parameters = ( expression | <keyword> )*

That would allow a statement macro to take some number of expressions and keywords as arguments, in addition to a suite. (2.2) +0 to the idea of prefixing a @ to a MACRO_NAME that is intended to be used as a sibling-macro. That would look like:

@do_nothing_marker! def foo(...): ...

(3)

Expression form ~~~~~~~~~~~~~~~

macro_expr = MACRO_NAME "(" testlist ")"

Again, what is `testlist`? Perhaps you're looking for something more like:

macro_expr = MACRO_NAME "(" macro_parameters ")" macro_parameters = ( expression ( "," expression )* ","? )?

(4)

Resolving ambiguity ~~~~~~~~~~~~~~~~~~~

The statement form of a macro takes precedence, so that the code ``macro_name!(x)`` will be parsed as a macro statement, not as an expression statement containing a macro expression.

It seems to me that if you were to define an expression macro - like `cast!(T, expression)` - that users would expect to be able to call such an expression macro on a line by itself. The rule stated here suggests that the parser would get confused by such a call, incorrectly treating the call as a call of a macro *statement* rather than a macro *expression*. (5)

Compilation ~~~~~~~~~~~

For macros with multiple names, [...]

Nit: "Multiple names"? I think you meant macros with "additional names", as described later in the PEP. (6)

Defining macro processors ~~~~~~~~~~~~~~~~~~~~~~~~~

A macro processor is defined by a four-tuple, consisting of ``(func, kind, version, additional_names)``:

* [...] * ``additional_names`` are the names of the additional parts of the macro, and must be a tuple of strings.

Seems to me that "additional_names" might make more sense to call "additional clause names" or just "clause names". That would be consistent with the following "anatomy of a statement-macro" sketch: try_!: # begin try_ statement; is try_ clause header ... # is try_ clause body finally_!: # is finally_ clause header ... # is finally_ clause body; end of try_ statement (7)

Hygiene and debugging ~~~~~~~~~~~~~~~~~~~~~

[...] No rules for naming will be enforced, but to ensure hygiene and help debugging, the following naming scheme is recommended: [...]

This appears to imply that it is up to the macro processor author to take special care that their macro is hygenic. That is, *unhygenic* macros are the default. Wouldn't it be safer to define a system where *hygenic* macros would be the default instead? Seems like if we can head off the introduction of Command Injection by default at the design level then we should do so. (8)

Examples ''''''''

Not a single example in this section defines an actual macro processor function. Recommend implementing at least a toy macro end-to-end (including the macro processor function) for full clarity, for each type of macro (i.e. statements, sibling, and expression). (9)

Backwards Compatibility =======================

This PEP is fully backwards compatible.

Nit: In the §"Implementation" section below, it is mentioned that nodes in the`_ast` module would be made immutable. That sounds like a backward-incompatible change to me. (10)

Currently, all AST nodes are allocated using an arena allocator. Changing to use the standard allocator might slow compilation down a little, but has advantages in terms of maintenance, as much code can be deleted.

I presume the arena allocator was introduced in the first place for a reason. Perhaps to improve performance? By removing the arena allocator are there potential downsides other than a performance regression? (11)

Reference Implementation ''''''''''''''''''''''''

None as yet.

Seems like you could get a prototype off the ground by implementing an initial version as a fake Python source file text encoding. Then you could put something like `# coding=macros` at the top of a source file to have it preprocessed by a prototype macro system. (12) Thanks for taking the time to read my comments. -- David Foster | Seattle, WA, USA Contributor to mypy, TypedDict, and Python's type system [1]: https://docs.python.org/3/reference/grammar.html

I’ve used lisp and scheme and one reason why you wouldn’t want a syntactic macro is because there should be one and only one way to do a task. Sure we have deviated that in the ecosystem but allowing syntactic macros can have the side effect of many programs or projects to have multiple ways to do something because they would have the feature . I think that this would be better as a library in my opinion. Sent from my iPhone

On Jan 29, 2023, at 2:20 PM, cdp49@cam.ac.uk wrote:

It looks like this hasn't gone anywhere in the past few years, which is a shame. Syntactic macros are one of the 2 or 3 "Killer features" that pushed me out of Python and into Julia (along with JITting inferred types and multiple dispatch). Math+data science code written in Julia is a lot more readable because of this. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PPCXZJYP... Code of Conduct: http://python.org/psf/codeofconduct/

Unfortunately, it's no longer being maintained.

cdp49@cam.ac.uk writes:

I think that's exactly the problem with a lack of Python macros. The full quote, of course, goes: "There should be one-- and preferably only one --*obvious* way to do it."

You understand that the Zen is humorous? Most of the Zen, if taken universally and seriously, advocates the impossible. And as a whole, it's definitely impossible even in the limited scope it's intended to apply to -- it's internally inconsistent.

Often, there's a mathematical notation for something, and *this* is the only obvious way to write anything out.

But that's not the way Python looks at it, you see. First, "a" mathematical notation doesn't exclude multiple notations, and for most mathematical concepts there are indeed multiple common notations (eg, for multiplication, juxtaposition of the factors, *, ・, and × are all in common use depending on the kind of multiplication). I'm pretty sure Tim Peters was well aware of such. Frequently the most commonly used expressions are really ugly (at least in my experience in mathematical applications to game theory ;-). Now, you can recover your position from that issue by appealing to other more or less Zen desiderata (readability counts, for example), but they're not quite as strong arguments here. The second objection is more serious: the Zen is intended to be restricted to Python. "There should be one-- and preferably only one --obvious way to do it [*in Python*]." Guido (and the other OG core devs) wanted consistent and obvious ways to do it *in Python*. The consistent and obvious way to write "X ~ N(0,1)" (oops, not so obvious after all!) in Python is "X = Normal(0,1)", where presumably the Normal class provides facilities such as CDF and PDF as well as the PRNG of random.normalvariate.

Not to mention, DSLs are forced to adopt all kinds of weird syntax

This is more or less intentional, though, as is the restriction to a predefined set of operator symbols (you can't define ・ as an operator symbol when you need two kinds of multiplication for example). Python has consistently refused to be turned into a platform for DSLs for almost 3 decades. Ruby and Lisps are better for that. You don't have to like that (quite a few people don't, of course that's why MacroPy was written), but it's really not un-Pythonic. There is method to this madness: Python aims for readability and flexibility for the community of Python programmers who might encounter the code, rather than catering to authors and their domain specialist community. The fact that Python adoption is still growing should tell you something about the preferences and needs of the general Python community.

I don't want to be forced to learn lots of weird little functions like `np.matmul(x1, x2)` when there's already one obvious syntax I'm very familiar with: `x1 * x2`.

I don't recall ever writing matrix multiplication that way in mathematics though. That's universally written as juxtaposition in my experience. And the obvious way to write it in Python (and np) has been "x1 @ x2" for some years now. In np, "*" means element-wise multiplication, I believe. Perhaps some BDFL will arise to merge the benefits of Python with those of Julia, but for the near term we're all going to have to choose one or the other. Steve

On Thu, Feb 2, 2023 at 12:46 PM Stéfane Fermigier wrote:

https://en.wikipedia.org/wiki/Mathematical_operators_and_symbols_in_Unicode https://oeis.org/wiki/List_of_LaTeX_mathematical_symbols NB: on a very basic level, I remember trying, a few years ago, to use the Unicode "empty set" symbol as a synonym for set(), and it didn't end well, for several reasons, including the fact that Python didn't like it as a variable name.

I use the `vim` conceal plugin to make some of these appear on screen while I'm editing. So, for example, when I type `set()` I see `∅`. When I type `all(...)` I see `∀(...)`. Much to the chagrin of Moshe Zadka, when I type `None` I see `ℵ` ... because I argue that ℵ_0 really should be considered an inaccessible cardinal :-). However, it never causes me problems because the files on disk are just plain old ASCII (for the most part), and the special symbols are just what my screen shows, not overloads to underlying operators. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.

" On Fri, Feb 3, 2023 at 12:28 PM Stephen J. Turnbull < stephenjturnbull@gmail.com> wrote:

Stéfane Fermigier writes:

...

NB: on a very basic level, I remember trying, a few years ago, to use the Unicode "empty set" symbol as a synonym for set(), and it didn't end well, for several reasons, including the fact that Python didn't like it as a variable name.

I know about the issue that '∅' is not valid in identifiers, but I'm curious about these other "several reasons".

IIRC (this was a few years back): '∅' (aka EMPTY SET) was not accepted by Python, so I used instead ''Ø" (aka "LATIN CAPITAL LETTER O WITH STROKE"). Python was OK with it, but some of the tools I use (one of flake8, isort, black... or maybe one of my IDE) barfed on it. S. -- Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/ Co-Founder & Co-Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/ Co-Founder & Chairman, Association Professionnelle Européenne du Logiciel Libre (APELL) - https://www.apell.info/ Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) - https://www.euclidia.eu/ "I wish you were as accurate, & as much to be relied on, as I am myself" - Lady Ada Lovelace

Python has consistently refused to be turned into a platform for DSLs for almost 3 decades.

I think SymPy, PyMC, Pyomo, Pyro, and many more packages would all be very surprised to hear they're no longer welcome in Python. Still, it seems like it would be quite hard to kick them out, and would probably make the scientific programming community pretty angry. If you don't like having DSLs in Python, I think you're trying to close the barn door after the horse has bolted; you'd have to go back in time to the creation of NumPy. Syntactic macros aren't necessary for DSLs; it just makes them better. Without syntactic macros, DSLs are forced to use clunky, complicated, and error-prone string manipulation, rather than cleaner syntactic transformations. For instance, here's NumPy's einsum, effectively behaving like a string macro: ``` X = np.einsum('ij,jk->ik', A, B, optimize='optimal') ``` And now here's the same thing in Julia: ``` @einsum X[i, k] := A[i, j] * B[j, k] ``` Which is more readable? Which is more Pythonic? It's not that Python doesn't have DSLs (NumPy is effectively a DSL for linear algebra). It's just that their syntax is sufficiently obscure that it's not at all clear that's what they're doing.