On 20 July 2015 at 18:01, Andrew Barnert <abarnert@yahoo.com> wrote:
On Jul 19, 2015, at 22:43, Nick Coghlan <ncoghlan@gmail.com> wrote:
I see nothing explicit about https://pypi.python.org/pypi/MacroPy or the examples at https://github.com/lihaoyi/macropy#macropy, as it looks just like normal Python code to me, with no indication that compile time modifications are taking place.
I suppose what I meant by explicit is things like using [] instead of () for macro calls and quick lambda definitions, s[""] for string interpolation, etc. Once you get used to it, it's usually obvious at a glance where code is using MacroPy.
Is this code using MacroPy for compile time transformations? data = source[lookup] You have no idea, and neither do I. Instead, we'd be relying on our rote memory to recognise certain *names* as being typical MacroPy operations - if someone defines a new transformation, our pattern recognition isn't going to trigger properly. It doesn't help that my rote memory is awful, so I *detest* APIs that expect me to have a good one and hence "just know" when certain operations are special and don't work the same way as other operations. My suggested "!(expr)" notation is based on the idea of providing an inline syntactic marker to say "magic happening here" (with the default anonymous transformation being to return the AST object itself).
That's not MacroPy's fault - it *can't* readily be explicit the way I would want it to be if it's going to reuse the existing AST compiler to do the heavy lifting.
However, I agree the MacroPy approach to tree transformations could be a good backend concept. I'd previously wondered how you'd go about embedding third party syntax like shell expressions or format strings, but eventually realised that combining an AST transformation syntax with string quoting works just fine there.
I think you want to be able to hook the tokenizer here as well. If you want f"..." of !f"...", that's hard to do at the tree level or the text level; you'd have to do something like f("...") or "f..." instead.
I figured out that AST->AST is fine, as anything else can be handled as quoted string transformations, which then gets you all the nice benefits of strings literals (choice of single or double quotes, triple-quoting for multi-line strings, escape sequences with the option of raw strings, etc), plus a clear inline marker alerting the reader to the fact that you've dropped out of Python's normal syntactic restrictions. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia