Recent changes to PyCodeObject
When I added Python 3.6 support to coverage.py, I posted a Mac wheel to PyPI: https://pypi.python.org/pypi/coverage/ That wheel was built against 3.6a3, the latest version at the time. When I use it now on 3.6b3, it doesn't work right. The reason is that the co_firstlineno field in PyCodeObject moved in September, as part of the PEP 523 work: https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d...
The docs say that PyCodeObject can change at any time, but I don't see why the field had to move in the first place. Was this needed?
Am I doing the wrong thing by using PyCodeObject fields directly in the coverage.py C trace function? It seems like this was an unnecessary breaking change, even if it is a non-guaranteed interface.
BTW, there are some comments that should be updated based on that commit, I can submit a patch to fix them.
--Ned.
On 16Nov2016 1618, Ned Batchelder wrote:
When I added Python 3.6 support to coverage.py, I posted a Mac wheel to PyPI: https://pypi.python.org/pypi/coverage/ That wheel was built against 3.6a3, the latest version at the time. When I use it now on 3.6b3, it doesn't work right. The reason is that the co_firstlineno field in PyCodeObject moved in September, as part of the PEP 523 work: https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d...
The docs say that PyCodeObject can change at any time, but I don't see why the field had to move in the first place. Was this needed?
IIRC, by reordering the fields we saved enough memory in padding that the new field did not take up any extra space (assuming 32-bit int and 64-bit pointers).
Am I doing the wrong thing by using PyCodeObject fields directly in the coverage.py C trace function? It seems like this was an unnecessary breaking change, even if it is a non-guaranteed interface.
I suspect the "wrong thing" here is expecting to not see any changes between alpha/beta versions. It certainly should not change within 3.6.x, so your wheel build will be fine, but changing during the prerelease versions is a possibility for anything not in the limited ABI.
Cheers, Steve
On 17 November 2016 at 10:44, Steve Dower steve.dower@python.org wrote:
On 16Nov2016 1618, Ned Batchelder wrote:
Am I doing the wrong thing by using PyCodeObject fields directly in the coverage.py C trace function? It seems like this was an unnecessary breaking change, even if it is a non-guaranteed interface.
I suspect the "wrong thing" here is expecting to not see any changes between alpha/beta versions. It certainly should not change within 3.6.x, so your wheel build will be fine, but changing during the prerelease versions is a possibility for anything not in the limited ABI.
Right, and we don't have an explicit policy on when the ABI gets locked prior to the x.y.0 release other than the general rule of "minimise change after the first release candidate".
This question also came up on the wheel-builders list in the context of "When can Python 3.6 support be added to the published manylinux1 build environments without risking ABI incompatible wheels on final release?".
Perhaps it would make sense to declare the final beta the last opportunity for public ABI changes? That would give folks a few weeks to start preparing launch binaries if they want to do so, rather than the single week planned between the rc and the final release.
Cheers, Nick.
Didn't 3.5 have to roll an extra last minute RC for an emergency abi-breaking bug fix, though? (Thinking of the windows runtime stuff.)
On Nov 16, 2016 7:51 PM, "Nick Coghlan" ncoghlan@gmail.com wrote:
On 17 November 2016 at 10:44, Steve Dower steve.dower@python.org wrote:
On 16Nov2016 1618, Ned Batchelder wrote:
Am I doing the wrong thing by using PyCodeObject fields directly in the coverage.py C trace function? It seems like this was an unnecessary breaking change, even if it is a non-guaranteed interface.
I suspect the "wrong thing" here is expecting to not see any changes
between
alpha/beta versions. It certainly should not change within 3.6.x, so your wheel build will be fine, but changing during the prerelease versions is
a
possibility for anything not in the limited ABI.
Right, and we don't have an explicit policy on when the ABI gets locked prior to the x.y.0 release other than the general rule of "minimise change after the first release candidate".
This question also came up on the wheel-builders list in the context of "When can Python 3.6 support be added to the published manylinux1 build environments without risking ABI incompatible wheels on final release?".
Perhaps it would make sense to declare the final beta the last opportunity for public ABI changes? That would give folks a few weeks to start preparing launch binaries if they want to do so, rather than the single week planned between the rc and the final release.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ njs%40pobox.com
Yeah, we did. That issue had been around since the betas though, it just wasn't discovered until late.
This is why we've been encouraging people to treat the betas like they used to treat the RCs (at least, I've been doing what I can). They're ready for testing against, just not ready for general distribution (or for producing wheels for general distribution).
Cheers, Steve
Top-posted from my Windows Phone
-----Original Message----- From: "Nathaniel Smith" njs@pobox.com Sent: 11/16/2016 20:19 To: "Nick Coghlan" ncoghlan@gmail.com Cc: "Python Dev" python-dev@python.org; "Steve Dower" steve.dower@python.org Subject: Re: [Python-Dev] Recent changes to PyCodeObject
Didn't 3.5 have to roll an extra last minute RC for an emergency abi-breaking bug fix, though? (Thinking of the windows runtime stuff.)
On Nov 16, 2016 7:51 PM, "Nick Coghlan" ncoghlan@gmail.com wrote:
On 17 November 2016 at 10:44, Steve Dower steve.dower@python.org wrote:
On 16Nov2016 1618, Ned Batchelder wrote:
Am I doing the wrong thing by using PyCodeObject fields directly in the coverage.py C trace function? It seems like this was an unnecessary breaking change, even if it is a non-guaranteed interface.
I suspect the "wrong thing" here is expecting to not see any changes between alpha/beta versions. It certainly should not change within 3.6.x, so your wheel build will be fine, but changing during the prerelease versions is a possibility for anything not in the limited ABI.
Right, and we don't have an explicit policy on when the ABI gets locked prior to the x.y.0 release other than the general rule of "minimise change after the first release candidate".
This question also came up on the wheel-builders list in the context of "When can Python 3.6 support be added to the published manylinux1 build environments without risking ABI incompatible wheels on final release?".
Perhaps it would make sense to declare the final beta the last opportunity for public ABI changes? That would give folks a few weeks to start preparing launch binaries if they want to do so, rather than the single week planned between the rc and the final release.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/njs%40pobox.com
On Wed, Nov 16, 2016 at 6:18 PM, Ned Batchelder ned@nedbatchelder.com wrote:
When I added Python 3.6 support to coverage.py, I posted a Mac wheel to PyPI: https://pypi.python.org/pypi/coverage/ That wheel was built against 3.6a3, the latest version at the time. When I use it now on 3.6b3, it doesn't work right. The reason is that the co_firstlineno field in PyCodeObject moved in September, as part of the PEP 523 work: https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d...
The docs say that PyCodeObject can change at any time, but I don't see why the field had to move in the first place. Was this needed?
Christian Heimes had the idea so that memory consumption wouldn't be impacted by adding co_extra to the struct: https://mail.python.org/pipermail/python-dev/2016-August/145961.html
Cody
On 11/17/16 10:09 AM, Cody Piersall wrote:
On Wed, Nov 16, 2016 at 6:18 PM, Ned Batchelder ned@nedbatchelder.com wrote:
When I added Python 3.6 support to coverage.py, I posted a Mac wheel to PyPI: https://pypi.python.org/pypi/coverage/ That wheel was built against 3.6a3, the latest version at the time. When I use it now on 3.6b3, it doesn't work right. The reason is that the co_firstlineno field in PyCodeObject moved in September, as part of the PEP 523 work: https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d...
The docs say that PyCodeObject can change at any time, but I don't see why the field had to move in the first place. Was this needed?
Christian Heimes had the idea so that memory consumption wouldn't be impacted by adding co_extra to the struct: https://mail.python.org/pipermail/python-dev/2016-August/145961.html
Thanks, I should have searched :)
--Ned.
I know you got your answer to why the change occurred, but a patch to update any outdated comments would be appreciated. :)
On Thu, 17 Nov 2016 at 07:55 Ned Batchelder ned@nedbatchelder.com wrote:
On 11/17/16 10:09 AM, Cody Piersall wrote:
On Wed, Nov 16, 2016 at 6:18 PM, Ned Batchelder ned@nedbatchelder.com
wrote:
When I added Python 3.6 support to coverage.py, I posted a Mac wheel to PyPI: https://pypi.python.org/pypi/coverage/ That wheel was built against 3.6a3, the latest version at the time. When I use it now on 3.6b3, it doesn't work right. The reason is that the co_firstlineno field in PyCodeObject moved in September, as part of the PEP 523 work:
https://github.com/python/cpython/commit/f1e6d88b3ca2b56d51d87b6b38ea1870c5d...
The docs say that PyCodeObject can change at any time, but I don't see why the field had to move in the first place. Was this needed?
Christian Heimes had the idea so that memory consumption wouldn't be impacted by adding co_extra to the struct: https://mail.python.org/pipermail/python-dev/2016-August/145961.html
Thanks, I should have searched :)
--Ned. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org
participants (6)
-
Brett Cannon -
Cody Piersall -
Nathaniel Smith -
Ned Batchelder -
Nick Coghlan -
Steve Dower