Need to rebuild wheels for every pypy minor version
Hi guys. Right now pypy wheels names include both the major and minor pypy version in them: e.g. uWSGI-2.0.14.0.*pp257*-pypy_41-linux_x86_64.whl This means that if we want to upgrade pypy from 5.7.1 to 5.8 we'd need to rebuild all our wheels and this is not scalable since there are new pypy releases every 3/4 months. Wouldn't it be enough to only include the major version in the wheel name? Are minor pypy versions really incompatible between them? Thanks, Daniele
Hi, note that this is not because of PyPy: it's the wheel package which chooses what to include in the wheel filename: https://bitbucket.org/pypa/wheel/src/5d49f8cf18679d1bc6f3e1e414a5df3a1e492644/wheel/pep425tags.py?at=default&fileviewer=file-view-default#pep425tags.py-39 PyPy reports only the ABI version, which is pypy_41. This is probably wrong for the opposite reasons, i.e. it claims it's backward compatible even when it's not: https://bitbucket.org/pypy/pypy/issues/2613/fix-the-abi-tag ciao, Antonio On Tue, Jul 25, 2017 at 8:47 PM, Daniele Rolando <danielerolando90@gmail.com
wrote:
Both the ABI version identifier pypy_41 and the python identifier pp257 need overhauling. I have submitted issues to pip and wheel for the Python Tag pp257 (i.e. pypy python 2 version 5.7) which IMO should be pp27 (i.e. pypy implementing python 2.7). https://github.com/pypa/pip/issues/4631 https://bitbucket.org/pypa/wheel/issues/187 Matti On 25/07/17 22:15, Antonio Cuni wrote:
I think that the intended idea is: The first field says what variant of the Python language the wheel uses. So it can be py2, py33, etc. for generic "python 2", "python 3.3 or better". Or it can be cp33 to mean "we specifically need cpython 3.3, other language variants won't do". And then the second field says what variant of the C ABI the wheel uses. So it can be cp35dm to mean "CPython, 3.5, debug, pymalloc". In practice this distinction doesn't make a *ton* of sense for CPython wheels, because it's redundant -- pretty much the only reason why you might need the cpython implementation specifically is if you're using the C ABI, and if you need the cp35dm ABI then obviously only CPython can give you that. bdist_wheel by default uses both, because, uh... belt *and* suspenders, I guess? But in principle they could vary independently. For example, if some package is python 2, and depends on some API in __pypy__, but doesn't use cpyext, then it might make sense to tag that as 'pp2_57-none' or something to mean "I need the pypy 2 v5.7-or-better variant of the language, but I don't care about the ABI". In general pypy can come up with any system that makes sense here. A given release can tell pip that it's compatible with whatever mix of python tags and ABI tags that make sense, and bdist_wheel can have whatever defaults make sense as well (and in general users can override if they have a weird case). You could even rethink it a bit and make extension modules generally use tags like 'py27-pp2_58' ("the python part is vanilla python 2.7, but it also needs the pypy2 v5.8 ABI"), and you could have a single release of PyPy declare that it supports multiple ABIs (e.g. you could have a pp2_cffi${X} ABI that you keep relatively stable and increment occasionally, as well as a pp2_cpyext${X} ABI that gets revved more often.) I guess one thing is that the python tags and ABI tags should probably be distinct for the pypy2 and pypy3 branches. And a "pp27" tag doesn't make much sense to me, because the difference between pp27 and py27 would be that the former declares that it specifically needs some kind of pypy extension... but what version of pypy added that extension? There's no way to say. Anyway, I guess the main thing is to think ahead about what kind of ABI/stability/evolution strategy you actually want to use :-). -n On Tue, Jul 25, 2017 at 2:30 PM, Matti Picus <matti.picus@gmail.com> wrote:
-- Nathaniel J. Smith -- https://vorpus.org
Hi, note that this is not because of PyPy: it's the wheel package which chooses what to include in the wheel filename: https://bitbucket.org/pypa/wheel/src/5d49f8cf18679d1bc6f3e1e414a5df3a1e492644/wheel/pep425tags.py?at=default&fileviewer=file-view-default#pep425tags.py-39 PyPy reports only the ABI version, which is pypy_41. This is probably wrong for the opposite reasons, i.e. it claims it's backward compatible even when it's not: https://bitbucket.org/pypy/pypy/issues/2613/fix-the-abi-tag ciao, Antonio On Tue, Jul 25, 2017 at 8:47 PM, Daniele Rolando <danielerolando90@gmail.com
wrote:
Both the ABI version identifier pypy_41 and the python identifier pp257 need overhauling. I have submitted issues to pip and wheel for the Python Tag pp257 (i.e. pypy python 2 version 5.7) which IMO should be pp27 (i.e. pypy implementing python 2.7). https://github.com/pypa/pip/issues/4631 https://bitbucket.org/pypa/wheel/issues/187 Matti On 25/07/17 22:15, Antonio Cuni wrote:
I think that the intended idea is: The first field says what variant of the Python language the wheel uses. So it can be py2, py33, etc. for generic "python 2", "python 3.3 or better". Or it can be cp33 to mean "we specifically need cpython 3.3, other language variants won't do". And then the second field says what variant of the C ABI the wheel uses. So it can be cp35dm to mean "CPython, 3.5, debug, pymalloc". In practice this distinction doesn't make a *ton* of sense for CPython wheels, because it's redundant -- pretty much the only reason why you might need the cpython implementation specifically is if you're using the C ABI, and if you need the cp35dm ABI then obviously only CPython can give you that. bdist_wheel by default uses both, because, uh... belt *and* suspenders, I guess? But in principle they could vary independently. For example, if some package is python 2, and depends on some API in __pypy__, but doesn't use cpyext, then it might make sense to tag that as 'pp2_57-none' or something to mean "I need the pypy 2 v5.7-or-better variant of the language, but I don't care about the ABI". In general pypy can come up with any system that makes sense here. A given release can tell pip that it's compatible with whatever mix of python tags and ABI tags that make sense, and bdist_wheel can have whatever defaults make sense as well (and in general users can override if they have a weird case). You could even rethink it a bit and make extension modules generally use tags like 'py27-pp2_58' ("the python part is vanilla python 2.7, but it also needs the pypy2 v5.8 ABI"), and you could have a single release of PyPy declare that it supports multiple ABIs (e.g. you could have a pp2_cffi${X} ABI that you keep relatively stable and increment occasionally, as well as a pp2_cpyext${X} ABI that gets revved more often.) I guess one thing is that the python tags and ABI tags should probably be distinct for the pypy2 and pypy3 branches. And a "pp27" tag doesn't make much sense to me, because the difference between pp27 and py27 would be that the former declares that it specifically needs some kind of pypy extension... but what version of pypy added that extension? There's no way to say. Anyway, I guess the main thing is to think ahead about what kind of ABI/stability/evolution strategy you actually want to use :-). -n On Tue, Jul 25, 2017 at 2:30 PM, Matti Picus <matti.picus@gmail.com> wrote:
-- Nathaniel J. Smith -- https://vorpus.org
participants (4)
-
Antonio Cuni -
Daniele Rolando -
Matti Picus -
Nathaniel Smith