<div dir="ltr">MacroPy is awesome both conceptually and feature-demo-wise - thanks for working on and sharing this!<div style>Simple, elegant things are hardest to come by - to utilize module loading hooks for AST transformation seems really natural from hindsight, except no-one did that before.</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, May 8, 2013 at 11:04 PM, Haoyi Li <span dir="ltr"><<a href="mailto:haoyi.sg@gmail.com" target="_blank">haoyi.sg@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Just an update for people who are interested,<div><br></div><div>The project (<a href="https://github.com/lihaoyi/macropy" target="_blank">https://github.com/lihaoyi/macropy</a>) is more or less done for now, in its current state as a proof of concept/demo. Almost all of it runs perfectly on both CPython and PyPy, except for the pattern matcher which has some bugs on PyPy we haven't ironed out yet. Jython doesn't work at all: it seems to handle a number of things about the ast module pretty differently from either PyPy or CPython.</div>
<div><br></div><div>We've got a pretty impressive list of feature demos:</div><div><ul style="margin:15px 0px;padding:0px 0px 0px 30px;border:0px;color:rgb(51,51,51);font-family:Helvetica,arial,freesans,clean,sans-serif;font-size:15px;line-height:25px">
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#quasiquotes" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Quasiquotes</a>, a quick way to manipulate fragments of a program</li>
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#string-interpolation" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">String Interpolation</a>, a common feature in many languages</li>
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#pyxl-integration" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Pyxl</a>, integrating XML markup into a Python program</li>
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#tracing" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Tracing</a> and <a href="https://github.com/lihaoyi/macropy#smart-asserts" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Smart Asserts</a></li>
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#case-classes" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Case Classes</a>, easy <a href="https://en.wikipedia.org/wiki/Algebraic_data_type" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Algebraic Data Types</a> from Scala</li>
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#pattern-matching" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Pattern Matching</a> from the Functional Programming world</li>
<li style="margin:0px;padding:0px;border:0px"><a href="https://github.com/lihaoyi/macropy#linq-to-sql" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">LINQ to SQL</a> from C#</li>
<li style="margin:0px;padding:0px;border:0px">
<a href="https://github.com/lihaoyi/macropy#quick-lambdas" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Quick Lambdas</a> from Scala and Groovy,</li><li style="margin:0px;padding:0px;border:0px">
<a href="https://github.com/lihaoyi/macropy#parser-combinators" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Parser Combinators</a>, inspired by <a href="http://www.suryasuravarapu.com/2011/04/scala-parser-combinators-win.html" style="margin:0px;padding:0px;border:0px;color:rgb(65,131,196);text-decoration:none" target="_blank">Scala's</a>.</li>
</ul></div><div>And have pushed a release to PyPI (<a href="https://pypi.python.org/pypi/MacroPy" target="_blank">https://pypi.python.org/pypi/MacroPy</a>), to make it easier for people to download it and mess around. Hopefully somebody will find this useful in messing around with the Python language!</div>
<div><br></div><div>Thanks!</div><span class="HOEnZb"><font color="#888888"><div>-Haoyi</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Apr 27, 2013 at 11:05 PM, Haoyi Li <span dir="ltr"><<a href="mailto:haoyi.sg@gmail.com" target="_blank">haoyi.sg@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I pushed a simple implementation of <a href="https://github.com/lihaoyi/macropy#case-classes" target="_blank">case classes</a> using Macros, as well as a really nice to use parser <a href="https://github.com/lihaoyi/macropy#parser-combinators" target="_blank">combinator library</a>. The case classes are interesting because they overlap a lot with enumerations: auto-generated __str__, __repr__, inheritence via nesting, they can have members and methods, etc. <br>
</div><div><br></div><div>They also show off pretty well how far Python's syntax (and semantic!) can be stretched using macros, so if anyone still has some crazy ideas for enumerations and wants to prototype them without hacking the CPython interpreter, this is your chance!</div>
<div><br></div><div>Thanks!</div><span><font color="#888888"><div>-Haoyi</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 24, 2013 at 3:15 PM, Haoyi Li <span dir="ltr"><<a href="mailto:haoyi.sg@gmail.com" target="_blank">haoyi.sg@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">@Jonathan: That would be possible, although I can't say I know how to do it. A naive macro that wraps everything and has a "substitute awaits for yields, wrap them in inlineCallbacks(), and substitute returns for returnValue()s" may work, but I'm guessing it would run into a forest of edge cases where the code isn't so simple (what if you *want* a return? etc.).<div>
<br></div><div>pdb *should* show the code after macro expansion. Without source maps, I'm not sure there's any way around that, so debugging may be hard.</div><div><br></div><div>Of course, if the alternative is macros of forking the interpreter, maybe macros is the easier way to do it =) Debugging a buggy custom-forked interpreter probably isn't easy either!</div>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 24, 2013 at 5:48 PM, Jonathan Slenders <span dir="ltr"><<a href="mailto:jonathan@slenders.be" target="_blank">jonathan@slenders.be</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">One use case I have is for Twisted's inlineCallbacks. I forked the<br>
pypy project to implement the await-keyword. Basically it transforms:<br>
<br>
def async_function(deferred_param):<br>
a = await deferred_param<br>
b = await some_call(a)<br>
return b<br>
<br>
into:<br>
<br>
@defer.inlineCallbacks<br>
def async_function(deferred_param):<br>
a = yield deferred_param<br>
b = yield some_call(a)<br>
yield defer.returnValue(b)<br>
<br>
<br>
Are such things possible? And if so, what lines of code would pdb show<br>
during introspection of the code?<br>
<br>
It's interesting, but when macros become more complicated, the<br>
debugging of these things can turn out to be really hard, I think.<br>
<br>
<br>
2013/4/24 Haoyi Li <<a href="mailto:haoyi.sg@gmail.com" target="_blank">haoyi.sg@gmail.com</a>>:<br>
<div><div>> I haven't tested in on various platforms, so hard to say for sure. MacroPy<br>
> basically relies on a few things:<br>
><br>
> - exec/eval<br>
> - PEP 302<br>
> - the ast module<br>
><br>
> All of these are pretty old pieces of python (almost 10 years old!) so it's<br>
> not some new-and-fancy functionality. Jython seems to have all of them, I<br>
> couldn't find any information about PyPy.<br>
><br>
> When the project is more mature and I have some time, I'll see if I can get<br>
> it to work cross platform. If anyone wants to fork the repo and try it out,<br>
> that'd be great too!<br>
><br>
> -Haoyi<br>
><br>
><br>
><br>
><br>
><br>
> On Wed, Apr 24, 2013 at 11:55 AM, Andrew Barnert <<a href="mailto:abarnert@yahoo.com" target="_blank">abarnert@yahoo.com</a>> wrote:<br>
>><br>
>> On Apr 24, 2013, at 8:05, Haoyi Li <<a href="mailto:haoyi.sg@gmail.com" target="_blank">haoyi.sg@gmail.com</a>> wrote:<br>
>><br>
>> You actually can get a syntax like that without macros, using<br>
>> stack-introspection, locals-trickery and lots of `eval`. The question is<br>
>> whether you consider macros more "extreme" than stack-introspection,<br>
>> locals-trickery and `eval`! A JIT compiler will probably be much happier<br>
>> with macros.<br>
>><br>
>><br>
>> That last point makes this approach seem particularly interesting to me,<br>
>> which makes me wonder: Is your code CPython specific, or does it also work<br>
>> with PyPy (or Jython or Iron)? While PyPy is obviously a whole lot easier to<br>
>> mess with in the first place than CPython, having macros at the same<br>
>> language level as your code is just as interesting in both implementations.<br>
>><br>
>><br>
>> On Wed, Apr 24, 2013 at 10:35 AM, Terry Jan Reedy <<a href="mailto:tjreedy@udel.edu" target="_blank">tjreedy@udel.edu</a>><br>
>> wrote:<br>
>>><br>
>>> On 4/23/2013 11:49 PM, Haoyi Li wrote:<br>
>>>><br>
>>>> I thought this may be of interest to some people on this list, even if<br>
>>>> not strictly an "idea".<br>
>>>><br>
>>>> I'm working on MacroPy <<a href="https://github.com/lihaoyi/macropy" target="_blank">https://github.com/lihaoyi/macropy</a>>, a little<br>
>>>><br>
>>>> pure-python library that allows user-defined AST rewrites as part of the<br>
>>>> import process (using PEP 302).<br>
>>><br>
>>><br>
>>> From the readme<br>
>>> '''<br>
>>> String Interpolation<br>
>>><br>
>>> a, b = 1, 2<br>
>>> c = s%"%{a} apple and %{b} bananas"<br>
>>> print c<br>
>>> #1 apple and 2 bananas<br>
>>> '''<br>
>>> I am a little surprised that you would base a cutting edge extension on<br>
>>> Py 2. Do you have it working with 3.3 also?<br>
>>><br>
>>> '''Unlike the normal string interpolation in Python, MacroPy's string<br>
>>> interpolation allows the programmer to specify the variables to be<br>
>>> interpolated inline inside the string.'''<br>
>>><br>
>>> Not true as I read that.<br>
>>><br>
>>> a, b = 1, 2<br>
>>> print("{a} apple and {b} bananas".format(**locals()))<br>
>>> print("%(a)s apple and %(b)s bananas" % locals())<br>
>>> #1 apple and 2 bananas<br>
>>> #1 apple and 2 bananas<br>
>>><br>
>>> I rather like the anon funcs with anon params. That only works when each<br>
>>> param is only used once in the expression, but that restriction is the<br>
>>> normal case.<br>
>>><br>
>>> I am interested to see what you do with pattern matching.<br>
>>><br>
>>> tjr<br>
>>><br>
>>> _______________________________________________<br>
>>> Python-ideas mailing list<br>
>>> <a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
>>> <a href="http://mail.python.org/mailman/listinfo/python-ideas" target="_blank">http://mail.python.org/mailman/listinfo/python-ideas</a><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Python-ideas mailing list<br>
>> <a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
>> <a href="http://mail.python.org/mailman/listinfo/python-ideas" target="_blank">http://mail.python.org/mailman/listinfo/python-ideas</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Python-ideas mailing list<br>
> <a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
> <a href="http://mail.python.org/mailman/listinfo/python-ideas" target="_blank">http://mail.python.org/mailman/listinfo/python-ideas</a><br>
><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/python-ideas" target="_blank">http://mail.python.org/mailman/listinfo/python-ideas</a><br>
<br></blockquote></div><br></div>