On Tue, Sep 15, 2020, at 20:10, Greg Ewing wrote:
> Maybe the PEP should propose an AST of its own, which would initially
> be a third thing separate from either of the existing ones, with
> the possibility of adopting it as the ast module representation
> some time in the future.
The rust approach to this problem to pass only a token stream to the macro. Then, the macro writer brings their own parser.
And (I looked this up [1]) it returns another token stream which is then re-parsed by the standard parser. I'm not quite sure who is responsible for determining where the macro ends -- the standard parser or the macro parser.
A nice side effect of this would be that (hopefully) the macro syntax doesn't need to have the ugly special cases needed to allow `import! foo as bar` and `from! mod import foo as bar` (though these still need to be predefined macros).
I'm not entirely sure how this would preserve the hygiene of the macros though, since the macro's parser could just do a token-by-token substitution without checking the context, effectively what a macro in C does. Forcing an AST on the macros restricts macros somewhat (e.g. it requires the input arguments to look like well-formed expressions) but that's probably a good thing.
(Hm, or not. How would you implement something like PEP 505 using macros? It introduces new tokens like `?.` and `??`.