[IronPython] IronPython 2.6 CodePlex Source Update

[merllab at microsoft.com]

This is an automated email letting you know that sources 
have recently been pushed out.  You can download these newer 
sources directly from http://ironpython.codeplex.com/SourceControl/changeset/view/66296.

ADDED SOURCES
	$/IronPython/IronPython_Main/Runtime/Microsoft.Scripting/IndexSpan.cs

MODIFIED SOURCES
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/OrExpression.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/ListComprehensionFor.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/SuiteStatement.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/SerializedScopeStatement.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/ForStatement.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/IfStatementTest.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/ClassDefinition.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/BinaryExpression.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/AndExpression.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/Node.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/IfStatement.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/PythonAst.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/PythonNameBinder.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/FunctionDefinition.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/WithStatement.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Parser.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Runtime/FunctionCode.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/TryStatement.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Tokenizer.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Token.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/UnaryExpression.cs
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Compiler/Ast/WhileStatement.cs
	$/IronPython/IronPython_Main/Runtime/Microsoft.Scripting/IndexSpan.cs
	$/IronPython/IronPython_Main/Runtime/Microsoft.Scripting/Microsoft.Scripting.csproj
	$/IronPython/IronPython_Main/Languages/IronPython/Tests/hosting/editor_svcs/tokencategorizer.py
	$/IronPython/IronPython_Main/Languages/IronPython/IronPython/Runtime/Operations/PythonOps.cs
	$/IronPython/IronPython_Main/Runtime/Microsoft.Scripting/Runtime/TokenInfo.cs
	$/IronPython/IronPython_Main/Runtime/Microsoft.Scripting/SourceUnit.cs

CHECKIN COMMENTS
--------------------------------------------------------------------------------
Changeset Id: 1829946
Date: 5/24/2010 4:29:18 PM

Reduces memory usage of AST nodes.  This replaces our current storing of index, line, column with just storing an index.  To resolve the line and column we store the line start information in the PythonAst which we can access via the parent chain.

Lots of places are touched but it’s a pretty straight forward transformation.  The interesting changes are:
                We no longer use TokenizerBuffer, it’s purpose was to track the line information
                We now need to construct the PythonAst before constructing the child nodes.  This is because we need to assign some parent node as we set the indexes otherwise you can’t access line information from unbound trees.
                We need to accommodate updating the line mapping information.  We currently just do a pass at the end of parsing to transform all of the line info.  For error reporting we use Tokenizer’s IndexToLocation which also goes to the source unit for line updates.


Adds IndexSpan to the DLR and also adds HasLineMapping to source unit.




(Shelveset: AstMemoryFinal5;REDMOND\dinov | SNAP CheckinId: 10826)