[Python-ideas] Macros for Python

Mart Sõmermaa mrts.pydev at gmail.com
Mon May 13 19:02:58 CEST 2013 

MacroPy is awesome both conceptually and feature-demo-wise - thanks for
working on and sharing this!
Simple, elegant things are hardest to come by - to utilize module loading
hooks for AST transformation seems really natural from hindsight, except
no-one did that before.


On Wed, May 8, 2013 at 11:04 PM, Haoyi Li <haoyi.sg at gmail.com> wrote:

> Just an update for people who are interested,
>
> The project (https://github.com/lihaoyi/macropy) is more or less done for
> now, in its current state as a proof of concept/demo. Almost all of it runs
> perfectly on both CPython and PyPy, except for the pattern matcher which
> has some bugs on PyPy we haven't ironed out yet. Jython doesn't work at
> all: it seems to handle a number of things about the ast module pretty
> differently from either PyPy or CPython.
>
> We've got a pretty impressive list of feature demos:
>
>    - Quasiquotes <https://github.com/lihaoyi/macropy#quasiquotes>, a
>    quick way to manipulate fragments of a program
>    - String Interpolation<https://github.com/lihaoyi/macropy#string-interpolation>,
>    a common feature in many languages
>    - Pyxl <https://github.com/lihaoyi/macropy#pyxl-integration>,
>    integrating XML markup into a Python program
>    - Tracing <https://github.com/lihaoyi/macropy#tracing> and Smart
>    Asserts <https://github.com/lihaoyi/macropy#smart-asserts>
>    - Case Classes <https://github.com/lihaoyi/macropy#case-classes>, easy Algebraic
>    Data Types <https://en.wikipedia.org/wiki/Algebraic_data_type> from
>    Scala
>    - Pattern Matching<https://github.com/lihaoyi/macropy#pattern-matching> from
>    the Functional Programming world
>    - LINQ to SQL <https://github.com/lihaoyi/macropy#linq-to-sql> from C#
>    - Quick Lambdas <https://github.com/lihaoyi/macropy#quick-lambdas> from
>    Scala and Groovy,
>    - Parser Combinators<https://github.com/lihaoyi/macropy#parser-combinators>,
>    inspired by Scala's<http://www.suryasuravarapu.com/2011/04/scala-parser-combinators-win.html>
>    .
>
> And have pushed a release to PyPI (https://pypi.python.org/pypi/MacroPy),
> to make it easier for people to download it and mess around. Hopefully
> somebody will find this useful in messing around with the Python language!
>
> Thanks!
> -Haoyi
>
>
> On Sat, Apr 27, 2013 at 11:05 PM, Haoyi Li <haoyi.sg at gmail.com> wrote:
>
>> I pushed a simple implementation of case classes<https://github.com/lihaoyi/macropy#case-classes> using
>> Macros, as well as a really nice to use parser combinator library<https://github.com/lihaoyi/macropy#parser-combinators>.
>> The case classes are interesting because they overlap a lot with
>> enumerations: auto-generated __str__, __repr__, inheritence via nesting,
>> they can have members and methods, etc.
>>
>> They also show off pretty well how far Python's syntax (and semantic!)
>> can be stretched using macros, so if anyone still has some crazy ideas for
>> enumerations and wants to prototype them without hacking the CPython
>> interpreter, this is your chance!
>>
>> Thanks!
>> -Haoyi
>>
>>
>> On Wed, Apr 24, 2013 at 3:15 PM, Haoyi Li <haoyi.sg at gmail.com> wrote:
>>
>>> @Jonathan: That would be possible, although I can't say I know how to do
>>> it. A naive macro that wraps everything and has a "substitute awaits for
>>> yields, wrap them in inlineCallbacks(), and substitute returns for
>>> returnValue()s" may work, but I'm guessing it would run into a forest of
>>> edge cases where the code isn't so simple (what if you *want* a return?
>>> etc.).
>>>
>>> pdb *should* show the code after macro expansion. Without source maps,
>>> I'm not sure there's any way around that, so debugging may be hard.
>>>
>>> Of course, if the alternative is macros of forking the interpreter,
>>> maybe macros is the easier way to do it =) Debugging a buggy custom-forked
>>> interpreter probably isn't easy either!
>>>
>>>
>>> On Wed, Apr 24, 2013 at 5:48 PM, Jonathan Slenders <jonathan at slenders.be
>>> > wrote:
>>>
>>>> One use case I have is for Twisted's inlineCallbacks. I forked the
>>>> pypy project to implement the await-keyword. Basically it transforms:
>>>>
>>>> def async_function(deferred_param):
>>>>     a = await deferred_param
>>>>     b = await some_call(a)
>>>>     return b
>>>>
>>>> into:
>>>>
>>>> @defer.inlineCallbacks
>>>> def async_function(deferred_param):
>>>>     a = yield deferred_param
>>>>     b = yield some_call(a)
>>>>     yield defer.returnValue(b)
>>>>
>>>>
>>>> Are such things possible? And if so, what lines of code would pdb show
>>>> during introspection of the code?
>>>>
>>>> It's interesting, but when macros become more complicated, the
>>>> debugging of these things can turn out to be really hard, I think.
>>>>
>>>>
>>>> 2013/4/24 Haoyi Li <haoyi.sg at gmail.com>:
>>>> > I haven't tested in on various platforms, so hard to say for sure.
>>>> MacroPy
>>>> > basically relies on a few things:
>>>> >
>>>> > - exec/eval
>>>> > - PEP 302
>>>> > - the ast module
>>>> >
>>>> > All of these are pretty old pieces of python (almost 10 years old!)
>>>> so it's
>>>> > not some new-and-fancy functionality. Jython seems to have all of
>>>> them, I
>>>> > couldn't find any information about PyPy.
>>>> >
>>>> > When the project is more mature and I have some time, I'll see if I
>>>> can get
>>>> > it to work cross platform. If anyone wants to fork the repo and try
>>>> it out,
>>>> > that'd be great too!
>>>> >
>>>> > -Haoyi
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Apr 24, 2013 at 11:55 AM, Andrew Barnert <abarnert at yahoo.com>
>>>> wrote:
>>>> >>
>>>> >> On Apr 24, 2013, at 8:05, Haoyi Li <haoyi.sg at gmail.com> wrote:
>>>> >>
>>>> >> You actually can get a syntax like that without macros, using
>>>> >> stack-introspection, locals-trickery and lots of `eval`. The
>>>> question is
>>>> >> whether you consider macros more "extreme" than stack-introspection,
>>>> >> locals-trickery and `eval`! A JIT compiler will probably be much
>>>> happier
>>>> >> with macros.
>>>> >>
>>>> >>
>>>> >> That last point makes this approach seem particularly interesting to
>>>> me,
>>>> >> which makes me wonder: Is your code CPython specific, or does it
>>>> also work
>>>> >> with PyPy (or Jython or Iron)? While PyPy is obviously a whole lot
>>>> easier to
>>>> >> mess with in the first place than CPython, having macros at the same
>>>> >> language level as your code is just as interesting in both
>>>> implementations.
>>>> >>
>>>> >>
>>>> >> On Wed, Apr 24, 2013 at 10:35 AM, Terry Jan Reedy <tjreedy at udel.edu>
>>>> >> wrote:
>>>> >>>
>>>> >>> On 4/23/2013 11:49 PM, Haoyi Li wrote:
>>>> >>>>
>>>> >>>> I thought this may be of interest to some people on this list,
>>>> even if
>>>> >>>> not strictly an "idea".
>>>> >>>>
>>>> >>>> I'm working on MacroPy <https://github.com/lihaoyi/macropy>, a
>>>> little
>>>> >>>>
>>>> >>>> pure-python library that allows user-defined AST rewrites as part
>>>> of the
>>>> >>>> import process (using PEP 302).
>>>> >>>
>>>> >>>
>>>> >>> From the readme
>>>> >>> '''
>>>> >>> String Interpolation
>>>> >>>
>>>> >>> a, b = 1, 2
>>>> >>> c = s%"%{a} apple and %{b} bananas"
>>>> >>> print c
>>>> >>> #1 apple and 2 bananas
>>>> >>> '''
>>>> >>> I am a little surprised that you would base a cutting edge
>>>> extension on
>>>> >>> Py 2. Do you have it working with 3.3 also?
>>>> >>>
>>>> >>> '''Unlike the normal string interpolation in Python, MacroPy's
>>>> string
>>>> >>> interpolation allows the programmer to specify the variables to be
>>>> >>> interpolated inline inside the string.'''
>>>> >>>
>>>> >>> Not true as I read that.
>>>> >>>
>>>> >>> a, b = 1, 2
>>>> >>> print("{a} apple and {b} bananas".format(**locals()))
>>>> >>> print("%(a)s apple and %(b)s bananas" % locals())
>>>> >>> #1 apple and 2 bananas
>>>> >>> #1 apple and 2 bananas
>>>> >>>
>>>> >>> I rather like the anon funcs with anon params. That only works when
>>>> each
>>>> >>> param is only used once in the expression, but that restriction is
>>>> the
>>>> >>> normal case.
>>>> >>>
>>>> >>> I am interested to see what you do with pattern matching.
>>>> >>>
>>>> >>> tjr
>>>> >>>
>>>> >>> _______________________________________________
>>>> >>> Python-ideas mailing list
>>>> >>> Python-ideas at python.org
>>>> >>> http://mail.python.org/mailman/listinfo/python-ideas
>>>> >>
>>>> >>
>>>> >> _______________________________________________
>>>> >> Python-ideas mailing list
>>>> >> Python-ideas at python.org
>>>> >> http://mail.python.org/mailman/listinfo/python-ideas
>>>> >
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Python-ideas mailing list
>>>> > Python-ideas at python.org
>>>> > http://mail.python.org/mailman/listinfo/python-ideas
>>>> >
>>>>
>>>
>>>
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130513/68fbe39b/attachment.html>

More information about the Python-ideas mailing list