Hi all, The cpyext and cffi extension modules are created with the extension ".pypy-XY.so", were X and Y are the current PyPy version. I believe that there is little point any more: as far as I'm aware, nothing in the API changed since a long time, and all these different versions are identical to each other. Should we simply freeze the ".pypy-26.so" name, so that future PyPy's will also use it, at least until some really incompatible change is made? This would resolve for example issues like https://github.com/pypa/pip/issues/2882 . A bientôt, Armin.
On June 13, 2015 at 3:10:39 AM, Armin Rigo (arigo@tunes.org) wrote:
Hi all,
The cpyext and cffi extension modules are created with the extension ".pypy-XY.so", were X and Y are the current PyPy version. I believe that there is little point any more: as far as I'm aware, nothing in the API changed since a long time, and all these different versions are identical to each other. Should we simply freeze the ".pypy-26.so" name, so that future PyPy's will also use it, at least until some really incompatible change is made?
This would resolve for example issues like https://github.com/pypa/pip/issues/2882 .
A bientôt,
Armin. _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
For the uses of pip, it would be preferable if there was an exported stable ABI that we could leverage to tag wheels with. For the people *making* Wheels, the less often that stable ABI changed the better, since a single wheel would be useful for more versions of PyPy then. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Hi Donald, On 13 June 2015 at 09:13, Donald Stufft <donald@stufft.io> wrote:
For the uses of pip, it would be preferable if there was an exported stable ABI that we could leverage to tag wheels with. For the people *making* Wheels, the less often that stable ABI changed the better, since a single wheel would be useful for more versions of PyPy then.
Sorry, I meant that nothing changed in the ABI, in addition to the API. You can just rename an existing "foo.pypy-25.so" to "foo.pypy-26.so" and it should work. Note that you have to assume that I don't know the problems related to making wheels, and give some more details in your answer. I'm ready to hear about any suggestion if fixing "imp.get_suffixes()" is either not a good idea, or ok but the same information needs to be exported some other way too. The point is that using sys.version_info[:2] is too vague, given that it is always (2, 7), and using sys.pypy_version_info[:2] is (or will become) too precise. A bientôt, Armin.
On June 13, 2015 at 3:23:21 AM, Armin Rigo (arigo@tunes.org) wrote:
Hi Donald,
On 13 June 2015 at 09:13, Donald Stufft wrote:
For the uses of pip, it would be preferable if there was an exported stable ABI that we could leverage to tag wheels with. For the people *making* Wheels, the less often that stable ABI changed the better, since a single wheel would be useful for more versions of PyPy then.
Sorry, I meant that nothing changed in the ABI, in addition to the API. You can just rename an existing "foo.pypy-25.so" to "foo.pypy-26.so" and it should work.
Note that you have to assume that I don't know the problems related to making wheels, and give some more details in your answer. I'm ready to hear about any suggestion if fixing "imp.get_suffixes()" is either not a good idea, or ok but the same information needs to be exported some other way too. The point is that using sys.version_info[:2] is too vague, given that it is always (2, 7), and using sys.pypy_version_info[:2] is (or will become) too precise.
A bientôt,
Armin.
Sorry, to be clear, I wasn't saying your idea was bad. I was just making it clear that whatever it is should reflect the actual ABI of the module. So if there are compile options (does JIT vs not change it?) then that should be reflected in it too. Ideally it'd contain enough information to know if something compiled for a particular PyPy is ABI compatible with another one. As far as how it's exported, the suffixes is a good starting place, and if the ABI tag can also be added to sysconfig.get_config_var("SOABI") that would be good too. For example, CPython (in the 3.x branch) has:: >>> import sysconfig >>> sysconfig.get_config_var("SOABI") 'cpython-34m' This is a CPython 3.4 compiled with --with-pymalloc. The CPython bits are here: https://www.python.org/dev/peps/pep-3149/ Generally though, I'm +1 on starting to define an actual ABI (and not breaking it without updating the tag to reflect that). As far as wheels goes, the important thing is that we'll make the SOABI part of the wheel filename and use it to select which wheel we'll download and install. You don't need to worry about that much other than the fact that defining one makes it better and being conservative in changing it makes it better. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
2015-06-13 9:31 GMT+02:00 Donald Stufft <donald@stufft.io>:
On June 13, 2015 at 3:23:21 AM, Armin Rigo (arigo@tunes.org) wrote:
Hi Donald,
On 13 June 2015 at 09:13, Donald Stufft wrote:
For the uses of pip, it would be preferable if there was an exported stable ABI that we could leverage to tag wheels with. For the people *making* Wheels, the less often that stable ABI changed the better, since a single wheel would be useful for more versions of PyPy then.
Sorry, I meant that nothing changed in the ABI, in addition to the API. You can just rename an existing "foo.pypy-25.so" to "foo.pypy-26.so" and it should work.
Note that you have to assume that I don't know the problems related to making wheels, and give some more details in your answer. I'm ready to hear about any suggestion if fixing "imp.get_suffixes()" is either not a good idea, or ok but the same information needs to be exported some other way too. The point is that using sys.version_info[:2] is too vague, given that it is always (2, 7), and using sys.pypy_version_info[:2] is (or will become) too precise.
A bientôt,
Armin.
Sorry, to be clear, I wasn't saying your idea was bad. I was just making it clear that whatever it is should reflect the actual ABI of the module. So if there are compile options (does JIT vs not change it?) then that should be reflected in it too. Ideally it'd contain enough information to know if something compiled for a particular PyPy is ABI compatible with another one.
But cffi extension modules are not real modules, right? When compiled for PyPy, the init function has a different name, and fills a C struct that cannot be considered as a Python module. They can be imported only because there is special support in the import framework. In this case, why not have a completely different extension or SOABI? Like _sqlite.cffi1.0.so
As far as how it's exported, the suffixes is a good starting place, and if the ABI tag can also be added to sysconfig.get_config_var("SOABI") that would be good too.
For example, CPython (in the 3.x branch) has::
>>> import sysconfig >>> sysconfig.get_config_var("SOABI") 'cpython-34m'
This is a CPython 3.4 compiled with --with-pymalloc. The CPython bits are here: https://www.python.org/dev/peps/pep-3149/
Generally though, I'm +1 on starting to define an actual ABI (and not breaking it without updating the tag to reflect that). As far as wheels goes, the important thing is that we'll make the SOABI part of the wheel filename and use it to select which wheel we'll download and install. You don't need to worry about that much other than the fact that defining one makes it better and being conservative in changing it makes it better.
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Amaury Forgeot d'Arc
Hi Amaury, On 13 June 2015 at 10:20, Amaury Forgeot d'Arc <amauryfa@gmail.com> wrote:
But cffi extension modules are not real modules, right? When compiled for PyPy, the init function has a different name, and fills a C struct that cannot be considered as a Python module.
They can be imported only because there is special support in the import framework.
In this case, why not have a completely different extension or SOABI? Like _sqlite.cffi1.0.so
They are "real" modules in the sense that the .c file is exactly the same as for CPython, and they are compiled with the same unmodifed distutils. They are not "real" only in the sense that there are some "#ifdef PYPY_VERSION" which end up not making use of any of the Python.h functions and exporting a differently-named init function. That's why, so far, I didn't bother trying to convince distutils that it should give different extensions for CPython C extension modules and for CFFI modules. We could try to call them _sqlite.cffi1.0.so, or even have .so's compiled for CPython 2.7 be directly importable under PyPy2 (by always exporting both variants of the init function). This requires hacks to distutils which I'm not comfortable with. At least the current solution has one standard SOABI, namely ".pypy-26.so", rather than trying to export two of them and confuse the other tools that depend on distutils (setuptools, pip...). A bientôt, Armin.
participants (3)
-
Amaury Forgeot d'Arc -
Armin Rigo -
Donald Stufft