[Python-ideas] Macros for Python

Haoyi Li haoyi.sg at gmail.com
Wed May 8 22:04:33 CEST 2013 

Just an update for people who are interested,

The project (https://github.com/lihaoyi/macropy) is more or less done for
now, in its current state as a proof of concept/demo. Almost all of it runs
perfectly on both CPython and PyPy, except for the pattern matcher which
has some bugs on PyPy we haven't ironed out yet. Jython doesn't work at
all: it seems to handle a number of things about the ast module pretty
differently from either PyPy or CPython.

We've got a pretty impressive list of feature demos:

   - Quasiquotes <https://github.com/lihaoyi/macropy#quasiquotes>, a quick
   way to manipulate fragments of a program
   - String Interpolation<https://github.com/lihaoyi/macropy#string-interpolation>,
   a common feature in many languages
   - Pyxl <https://github.com/lihaoyi/macropy#pyxl-integration>,
   integrating XML markup into a Python program
   - Tracing <https://github.com/lihaoyi/macropy#tracing> and Smart
Asserts<https://github.com/lihaoyi/macropy#smart-asserts>
   - Case Classes <https://github.com/lihaoyi/macropy#case-classes>,
easy Algebraic
   Data Types <https://en.wikipedia.org/wiki/Algebraic_data_type> from Scala
   - Pattern Matching
<https://github.com/lihaoyi/macropy#pattern-matching> from
   the Functional Programming world
   - LINQ to SQL <https://github.com/lihaoyi/macropy#linq-to-sql> from C#
   - Quick Lambdas <https://github.com/lihaoyi/macropy#quick-lambdas> from
   Scala and Groovy,
   - Parser Combinators<https://github.com/lihaoyi/macropy#parser-combinators>,
   inspired by Scala's<http://www.suryasuravarapu.com/2011/04/scala-parser-combinators-win.html>
   .

And have pushed a release to PyPI (https://pypi.python.org/pypi/MacroPy),
to make it easier for people to download it and mess around. Hopefully
somebody will find this useful in messing around with the Python language!

Thanks!
-Haoyi


On Sat, Apr 27, 2013 at 11:05 PM, Haoyi Li <haoyi.sg at gmail.com> wrote:

> I pushed a simple implementation of case classes<https://github.com/lihaoyi/macropy#case-classes> using
> Macros, as well as a really nice to use parser combinator library<https://github.com/lihaoyi/macropy#parser-combinators>.
> The case classes are interesting because they overlap a lot with
> enumerations: auto-generated __str__, __repr__, inheritence via nesting,
> they can have members and methods, etc.
>
> They also show off pretty well how far Python's syntax (and semantic!) can
> be stretched using macros, so if anyone still has some crazy ideas for
> enumerations and wants to prototype them without hacking the CPython
> interpreter, this is your chance!
>
> Thanks!
> -Haoyi
>
>
> On Wed, Apr 24, 2013 at 3:15 PM, Haoyi Li <haoyi.sg at gmail.com> wrote:
>
>> @Jonathan: That would be possible, although I can't say I know how to do
>> it. A naive macro that wraps everything and has a "substitute awaits for
>> yields, wrap them in inlineCallbacks(), and substitute returns for
>> returnValue()s" may work, but I'm guessing it would run into a forest of
>> edge cases where the code isn't so simple (what if you *want* a return?
>> etc.).
>>
>> pdb *should* show the code after macro expansion. Without source maps,
>> I'm not sure there's any way around that, so debugging may be hard.
>>
>> Of course, if the alternative is macros of forking the interpreter, maybe
>> macros is the easier way to do it =) Debugging a buggy custom-forked
>> interpreter probably isn't easy either!
>>
>>
>> On Wed, Apr 24, 2013 at 5:48 PM, Jonathan Slenders <jonathan at slenders.be>wrote:
>>
>>> One use case I have is for Twisted's inlineCallbacks. I forked the
>>> pypy project to implement the await-keyword. Basically it transforms:
>>>
>>> def async_function(deferred_param):
>>>     a = await deferred_param
>>>     b = await some_call(a)
>>>     return b
>>>
>>> into:
>>>
>>> @defer.inlineCallbacks
>>> def async_function(deferred_param):
>>>     a = yield deferred_param
>>>     b = yield some_call(a)
>>>     yield defer.returnValue(b)
>>>
>>>
>>> Are such things possible? And if so, what lines of code would pdb show
>>> during introspection of the code?
>>>
>>> It's interesting, but when macros become more complicated, the
>>> debugging of these things can turn out to be really hard, I think.
>>>
>>>
>>> 2013/4/24 Haoyi Li <haoyi.sg at gmail.com>:
>>> > I haven't tested in on various platforms, so hard to say for sure.
>>> MacroPy
>>> > basically relies on a few things:
>>> >
>>> > - exec/eval
>>> > - PEP 302
>>> > - the ast module
>>> >
>>> > All of these are pretty old pieces of python (almost 10 years old!) so
>>> it's
>>> > not some new-and-fancy functionality. Jython seems to have all of
>>> them, I
>>> > couldn't find any information about PyPy.
>>> >
>>> > When the project is more mature and I have some time, I'll see if I
>>> can get
>>> > it to work cross platform. If anyone wants to fork the repo and try it
>>> out,
>>> > that'd be great too!
>>> >
>>> > -Haoyi
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Apr 24, 2013 at 11:55 AM, Andrew Barnert <abarnert at yahoo.com>
>>> wrote:
>>> >>
>>> >> On Apr 24, 2013, at 8:05, Haoyi Li <haoyi.sg at gmail.com> wrote:
>>> >>
>>> >> You actually can get a syntax like that without macros, using
>>> >> stack-introspection, locals-trickery and lots of `eval`. The question
>>> is
>>> >> whether you consider macros more "extreme" than stack-introspection,
>>> >> locals-trickery and `eval`! A JIT compiler will probably be much
>>> happier
>>> >> with macros.
>>> >>
>>> >>
>>> >> That last point makes this approach seem particularly interesting to
>>> me,
>>> >> which makes me wonder: Is your code CPython specific, or does it also
>>> work
>>> >> with PyPy (or Jython or Iron)? While PyPy is obviously a whole lot
>>> easier to
>>> >> mess with in the first place than CPython, having macros at the same
>>> >> language level as your code is just as interesting in both
>>> implementations.
>>> >>
>>> >>
>>> >> On Wed, Apr 24, 2013 at 10:35 AM, Terry Jan Reedy <tjreedy at udel.edu>
>>> >> wrote:
>>> >>>
>>> >>> On 4/23/2013 11:49 PM, Haoyi Li wrote:
>>> >>>>
>>> >>>> I thought this may be of interest to some people on this list, even
>>> if
>>> >>>> not strictly an "idea".
>>> >>>>
>>> >>>> I'm working on MacroPy <https://github.com/lihaoyi/macropy>, a
>>> little
>>> >>>>
>>> >>>> pure-python library that allows user-defined AST rewrites as part
>>> of the
>>> >>>> import process (using PEP 302).
>>> >>>
>>> >>>
>>> >>> From the readme
>>> >>> '''
>>> >>> String Interpolation
>>> >>>
>>> >>> a, b = 1, 2
>>> >>> c = s%"%{a} apple and %{b} bananas"
>>> >>> print c
>>> >>> #1 apple and 2 bananas
>>> >>> '''
>>> >>> I am a little surprised that you would base a cutting edge extension
>>> on
>>> >>> Py 2. Do you have it working with 3.3 also?
>>> >>>
>>> >>> '''Unlike the normal string interpolation in Python, MacroPy's string
>>> >>> interpolation allows the programmer to specify the variables to be
>>> >>> interpolated inline inside the string.'''
>>> >>>
>>> >>> Not true as I read that.
>>> >>>
>>> >>> a, b = 1, 2
>>> >>> print("{a} apple and {b} bananas".format(**locals()))
>>> >>> print("%(a)s apple and %(b)s bananas" % locals())
>>> >>> #1 apple and 2 bananas
>>> >>> #1 apple and 2 bananas
>>> >>>
>>> >>> I rather like the anon funcs with anon params. That only works when
>>> each
>>> >>> param is only used once in the expression, but that restriction is
>>> the
>>> >>> normal case.
>>> >>>
>>> >>> I am interested to see what you do with pattern matching.
>>> >>>
>>> >>> tjr
>>> >>>
>>> >>> _______________________________________________
>>> >>> Python-ideas mailing list
>>> >>> Python-ideas at python.org
>>> >>> http://mail.python.org/mailman/listinfo/python-ideas
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Python-ideas mailing list
>>> >> Python-ideas at python.org
>>> >> http://mail.python.org/mailman/listinfo/python-ideas
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > Python-ideas mailing list
>>> > Python-ideas at python.org
>>> > http://mail.python.org/mailman/listinfo/python-ideas
>>> >
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130508/d97db9e8/attachment.html>

More information about the Python-ideas mailing list