Tweaking AST lineno and col_offset
Hi all, Off and on I have been directly comparing Jython's AST with Python's AST and generally working towards making them as close to identical as possible. There are a couple of places where I haven't "fixed" Jython because it looks to me like Jython has slightly better offsets. One example: for a,b in c: pass The Tuple node "a,b" ends up with a col_offset of 0 (the position of the "for") where Jython has the col_offset as 4 (the position of "a"). Jython's result is more consistent with other Tuple node col_offset results. I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match). Also, would this be a change that would be considered a backwards incompatibility? In other words, would patches like this be allowed in 2.6/3.1 or only in 2.7/3.2. Regards, -Frank
2009/8/14 Frank Wierzbicki <fwierzbicki@gmail.com>:
Hi all,
Off and on I have been directly comparing Jython's AST with Python's AST and generally working towards making them as close to identical as possible. There are a couple of places where I haven't "fixed" Jython because it looks to me like Jython has slightly better offsets. One example:
for a,b in c: pass
The Tuple node "a,b" ends up with a col_offset of 0 (the position of the "for") where Jython has the col_offset as 4 (the position of "a"). Jython's result is more consistent with other Tuple node col_offset results.
I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match).
Yes, please submit it.
Also, would this be a change that would be considered a backwards incompatibility? In other words, would patches like this be allowed in 2.6/3.1 or only in 2.7/3.2.
While I don't see a problem in backporting it to maintence branches, I would personally only apply it to the current development branches. It doesn't seem to fix a "bug", just make a nice improvement. -- Regards, Benjamin
On Fri, Aug 14, 2009 at 12:16 PM, Benjamin Peterson<benjamin@python.org> wrote:
I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match).
Yes, please submit it. Great, the patch is here: http://bugs.python.org/issue6704
BTW - I would have added a test to test_ast.py, but above the test data it notes: #### EVERYTHING BELOW IS GENERATED ##### and I couldn't find the tool used for the generation. Does anyone know where that is? -Frank
2009/8/14 Frank Wierzbicki <fwierzbicki@gmail.com>:
On Fri, Aug 14, 2009 at 12:16 PM, Benjamin Peterson<benjamin@python.org> wrote:
I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match).
Yes, please submit it. Great, the patch is here: http://bugs.python.org/issue6704
I'll take a look.
BTW - I would have added a test to test_ast.py, but above the test data it notes: #### EVERYTHING BELOW IS GENERATED ##### and I couldn't find the tool used for the generation. Does anyone know where that is?
It's at the bottom of the test file. :) You can add a handwritten test above that, though. -- Regards, Benjamin
On Fri, Aug 14, 2009 at 3:11 PM, Benjamin Peterson<benjamin@python.org> wrote:
2009/8/14 Frank Wierzbicki <fwierzbicki@gmail.com>:
On Fri, Aug 14, 2009 at 12:16 PM, Benjamin Peterson<benjamin@python.org> wrote:
I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match).
Yes, please submit it. Great, the patch is here: http://bugs.python.org/issue6704
I'll take a look.
BTW - I would have added a test to test_ast.py, but above the test data it notes: #### EVERYTHING BELOW IS GENERATED ##### and I couldn't find the tool used for the generation. Does anyone know where that is?
It's at the bottom of the test file. :) You can add a handwritten test above that, though. Heh -- how did I miss that :) ? -- I'll resubmit the patch with tests.
-Frank
On Fri, Aug 14, 2009 at 3:41 PM, Frank Wierzbicki<fwierzbicki@gmail.com> wrote:
It's at the bottom of the test file. :) You can add a handwritten test above that, though. Heh -- how did I miss that :) ? -- I'll resubmit the patch with tests. Resubmitted http://bugs.python.org/issue6704 with tests.
-Frank
On Fri, Aug 14, 2009 at 09:16, Benjamin Peterson <benjamin@python.org>wrote:
2009/8/14 Frank Wierzbicki <fwierzbicki@gmail.com>:
Hi all,
Off and on I have been directly comparing Jython's AST with Python's AST and generally working towards making them as close to identical as possible. There are a couple of places where I haven't "fixed" Jython because it looks to me like Jython has slightly better offsets. One example:
for a,b in c: pass
The Tuple node "a,b" ends up with a col_offset of 0 (the position of the "for") where Jython has the col_offset as 4 (the position of "a"). Jython's result is more consistent with other Tuple node col_offset results.
I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match).
Yes, please submit it.
Also, would this be a change that would be considered a backwards incompatibility? In other words, would patches like this be allowed in 2.6/3.1 or only in 2.7/3.2.
While I don't see a problem in backporting it to maintence branches, I would personally only apply it to the current development branches. It doesn't seem to fix a "bug", just make a nice improvement.
I like the improvement, but I disagree it should be considered for backporting as it changes semantics for something that could be considered a bug, but that feels like a stretch. -Brett
On Fri, Aug 14, 2009 at 5:57 PM, Brett Cannon<brett@python.org> wrote:
I like the improvement, but I disagree it should be considered for backporting as it changes semantics for something that could be considered a bug, but that feels like a stretch. Just thought I'd ask -- I'm perfectly ok with the change being 2.7/3.2 only.
-Frank
participants (3)
-
Benjamin Peterson -
Brett Cannon -
Frank Wierzbicki