<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.kwd
{mso-style-name:kwd;}
span.pln
{mso-style-name:pln;}
span.typ
{mso-style-name:typ;}
span.pun
{mso-style-name:pun;}
span.str
{mso-style-name:str;}
span.lit
{mso-style-name:lit;}
span.EmailStyle24
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:952715507;
mso-list-template-ids:-1032176380;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">PTVS’s parser (which is just a branch of the IronPython parser) supports this so that we could implement the refactoring features. The nodes have a ToCodeString
method on them which you can call to get back the original code. You’ll need to parse in verbatim mode like so:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas">
<span style="color:blue">var</span> parser = <span style="color:#2B91AF">Parser</span>.CreateParser(<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas">
<span style="color:blue">someTextReader</span>,<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas"> Project.LanguageVersion,<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas">
<span style="color:blue">new</span> <span style="color:#2B91AF">ParserOptions</span>() { Verbatim =
<span style="color:blue">true</span>, BindReferences = <span style="color:blue">true</span> }<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas"> );<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas"> var ast = parser.ParseFile();<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:9.5pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">And you’ll need to pass the PythonAst that you get back into the ToCodeString as all of the verbatim round-trip info is stored on the top-level Ast node.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">This will preserve comments, whitespace, etc… You can modify the AST in any way you want and still call ToCodeString – the nodes which you insert will all
get some default amount of formatting. You’ll want to pass in the original PythonAst that you used to parse the nodes with. There’s some random helper methods like CopyLeadingWhiteSpace and CopyTrailingNewLine on the Node classes for manipulating some of
the round trip info – it’s not exhaustive, just what we’ve needed for PTVS’s refactoring support. You can always download the source and add more or contribute back changes if you need some specific additions.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The parser is in Microsoft.PythonTools.Analysis.dll which is a stand-alone DLL (no dependencies on anything outside of the .NET framework). If you want to
get a copy of the DLL you can just download and install PTVS (<a href="http://pytools.codeplex.com/">http://pytools.codeplex.com/</a>) and then the DLL will either be in:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">VSINSTALLDIR\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\1.0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> or<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">%LOCALAPPDATA%\Microsoft\VisualStudio\10.0\Extensions\Microsoft\Python Tools for Visual Studio\1.0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Depending on if you did the default (all-users) install or the per-user install.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The only thing that’s possibly missing is that we don’t have a good AST re-writer, so you’ll need to do that by hand or implement something like the DLR ExpressionVisitor
class.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> ironpython-users-bounces+dinov=exchange.microsoft.com@python.org [mailto:ironpython-users-bounces+dinov=exchange.microsoft.com@python.org]
<b>On Behalf Of </b>Jay Riley<br>
<b>Sent:</b> Friday, October 28, 2011 8:39 AM<br>
<b>To:</b> ironpython-users@python.org<br>
<b>Subject:</b> [Ironpython-users] Manipulating/Reversing the Ironpython AST<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Hi all,<br>
<br>
I was wondering if IronPython has the ability to reverse/unparse the AST back into a ource file? Or if someone has perhaps built something to do so?<br>
<br>
Here's what I'm trying to do:<br>
<br>
I have a game I'm working on, and I'm currently drafting up tools for it. The tools are being written in C# and are meant to make changing and editing game files easier. Several of the game files are written in python and are used to extend objects from the
main game source. For instace, I have an Items.py file that contains the following (minimalized for example):<br>
<br>
<span class="kwd"><span style="color:darkblue">from</span></span><span class="pln">
</span><span class="typ"><span style="color:#2B91AF">ItemModule</span></span><span class="pln">
</span><span class="kwd"><span style="color:darkblue">import</span></span><span class="pln">
</span><span class="pun">*</span><span class="pln"> </span><br>
<span class="pln"> </span><br>
<span class="kwd"><span style="color:darkblue">import</span></span><span class="pln"> copy </span><br>
<span class="pln"> </span><br>
<span class="kwd"><span style="color:darkblue">class</span></span><span class="pln">
</span><span class="typ"><span style="color:#2B91AF">ScriptedItem</span></span><span class="pun">(</span><span class="typ"><span style="color:#2B91AF">Item</span></span><span class="pun">):</span><span class="pln"> </span><br>
<span class="pln"> def __init__</span><span class="pun">(</span><span class="pln">self</span><span class="pun">,</span><span class="pln"> name</span><span class="pun">,</span><span class="pln"> description</span><span class="pun">,</span><span class="pln">
itemtypes</span><span class="pun">,</span><span class="pln"> primarytype</span><span class="pun">,</span><span class="pln"> flags</span><span class="pun">,</span><span class="pln"> usability</span><span class="pun">,</span><span class="pln"> value</span><span class="pun">,</span><span class="pln">
throwpower</span><span class="pun">):</span><span class="pln"> </span><br>
<span class="pln"> </span><span class="typ"><span style="color:#2B91AF">Item</span></span><span class="pun">.</span><span class="pln">__init__</span><span class="pun">(</span><span class="pln">self</span><span class="pun">,</span><span class="pln"> name</span><span class="pun">,</span><span class="pln">
description</span><span class="pun">,</span><span class="pln"> itemtypes</span><span class="pun">,</span><span class="pln"> primarytype</span><span class="pun">,</span><span class="pln"> flags</span><span class="pun">,</span><span class="pln"> usability</span><span class="pun">,</span><span class="pln">
value</span><span class="pun">,</span><span class="pln"> throwpower</span><span class="pun">,</span><span class="pln">
</span><span class="typ"><span style="color:#2B91AF">Item</span></span><span class="pun">.</span><span class="typ"><span style="color:#2B91AF">GetNextItemID</span></span><span class="pun">())</span><span class="pln"> </span><br>
<span class="pln"> def </span><span class="typ"><span style="color:#2B91AF">Clone</span></span><span class="pun">(</span><span class="pln">self</span><span class="pun">):</span><span class="pln"> </span><br>
<span class="pln"> </span><span class="kwd"><span style="color:darkblue">return</span></span><span class="pln"> copy</span><span class="pun">.</span><span class="pln">deepcopy</span><span class="pun">(</span><span class="pln">self</span><span class="pun">)</span><span class="pln"> </span><br>
<span class="pln"> </span><br>
<span class="typ"><span style="color:#2B91AF">ItemLibrary</span></span><span class="pun">.</span><span class="typ"><span style="color:#2B91AF">AddItem</span></span><span class="pun">(</span><span class="typ"><span style="color:#2B91AF">ScriptedItem</span></span><span class="pun">(</span><span class="str"><span style="color:maroon">"Abounding
Crystal"</span></span><span class="pun">,</span><span class="pln"> </span><span class="str"><span style="color:maroon">"A colourful crystal composed of many smaller crystals. It gives off a warm glow."</span></span><span class="pun">,</span><span class="pln">
</span><span class="typ"><span style="color:#2B91AF">ItemType</span></span><span class="pun">.</span><span class="typ"><span style="color:#2B91AF">SynthesisMaterial</span></span><span class="pun">,</span><span class="pln">
</span><span class="typ"><span style="color:#2B91AF">ItemType</span></span><span class="pun">.</span><span class="typ"><span style="color:#2B91AF">SynthesisMaterial</span></span><span class="pun">,</span><span class="pln">
</span><span class="lit"><span style="color:maroon">0</span></span><span class="pun">,</span><span class="pln">
</span><span class="typ"><span style="color:#2B91AF">ItemUsage</span></span><span class="pun">.</span><span class="typ"><span style="color:#2B91AF">Unusable</span></span><span class="pun">,</span><span class="pln">
</span><span class="lit"><span style="color:maroon">0</span></span><span class="pun">,</span><span class="pln">
</span><span class="lit"><span style="color:maroon">50</span></span><span class="pun">))</span><span class="pln"> </span><br>
<br>
<span class="pln">In this case, I'd like to provide a convenient front-end to allow editors to modify/add/delete items from the list. To do this, my editor need to be able to:</span><br>
<span class="pln"><o:p></o:p></span></span></p>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Find and list all the class types (in this example, it'd be only Scripted Item)</span><o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Find and list all created items (in this case there'd only be one, Abounding Crystal). I'd need to find the type (in this caseScriptedItem) and all the parameter values<o:p></o:p></span></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Allow editing of parameters and the creation/removal of items.<o:p></o:p></span></li></ol>
<p class="MsoNormal"><span class="pln"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> </span></span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><br>
<span class="pln">I tried writing my own parser, but that became increasingly difficult as the complexity of the Item constructors went up. When I found IronPython and its ability to generate a walkable AST, I thought I'd found my solution, and indeed the AST
makes it easy to accomplish 1 and 2 of my requirements. However, I'm currently stuck on how to write back changes made in my editor to the source file. My initial idea was to preserve the AST and modify values on the existing nodes for edited items and inject
new nodes when new items were created. However even if I could get this to work correctly, I have no idea how to reconvert the AST back into a source file. When I asked on stackoverflow, I was told this is usually done using "prettyprinting" and had some suggestions
to use python's "inspect" property.</span><br>
<span class="pln"> </span><br>
<span class="pln">I'm not sure how to use inspect to do what I want, and have some concerns over the amount of effort require to get "prettyprinting" correct, so i wanted to ask if anyone here has written a prettyprinter for ironpython or perhaps knows some
other way to accomplish my three goals? Any help would be appreciated,</span><br>
<span class="pln"> </span><br>
<span class="pln">Thanks</span><o:p></o:p></span></p>
</div>
</div>
</body>
</html>