Best practices for optional C extensions
I'd like to add a C extension to speed up a small bit of code in a package (Tornado), but make it optional both for compatibility with non-cpython implementations and for ease of installation on systems without a C compiler available. Ideally any user who runs "pip install tornado" on a system capable of compiling extensions would get the extensions; if this capability cannot be detected automatically I'd prefer the opt-out case to be the one that requires non-default arguments. Are there any packages that provide a good example to follow for this? PEP 426 uses "c-accelerators" as an example of an "extra", but it's unclear how this would work (based on the equivalent setuptools feature). There doesn't appear to be a way to know what extras are requested at build time. If the extra required a package like cython then you could build the extension whenever that package is present, but what about hand-written extensions? Extras are also opt-in instead of opt-out, so I'd have to recommend that most people use "pip install tornado[fast]" instead of "pip install tornado" (with "tornado[slow]" available as an option for limited environments). Thanks, -Ben
On 13 July 2013 14:14, Ben Darnell <ben@bendarnell.com> wrote:
I'd like to add a C extension to speed up a small bit of code in a package (Tornado), but make it optional both for compatibility with non-cpython implementations and for ease of installation on systems without a C compiler available. Ideally any user who runs "pip install tornado" on a system capable of compiling extensions would get the extensions; if this capability cannot be detected automatically I'd prefer the opt-out case to be the one that requires non-default arguments. Are there any packages that provide a good example to follow for this?
I believe that coverage has an optional C extension like this. Paul
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/13/2013 09:14 AM, Ben Darnell wrote:
I'd like to add a C extension to speed up a small bit of code in a package (Tornado), but make it optional both for compatibility with non-cpython implementations and for ease of installation on systems without a C compiler available. Ideally any user who runs "pip install tornado" on a system capable of compiling extensions would get the extensions; if this capability cannot be detected automatically I'd prefer the opt-out case to be the one that requires non-default arguments. Are there any packages that provide a good example to follow for this?
PEP 426 uses "c-accelerators" as an example of an "extra", but it's unclear how this would work (based on the equivalent setuptools feature). There doesn't appear to be a way to know what extras are requested at build time. If the extra required a package like cython then you could build the extension whenever that package is present, but what about hand-written extensions? Extras are also opt-in instead of opt-out, so I'd have to recommend that most people use "pip install tornado[fast]" instead of "pip install tornado" (with "tornado[slow]" available as an option for limited environments).
zope.interface subclasses the 'build_ext' command so: - ---------------------------- %< ----------------------------- from distutils.command.build_ext import build_ext from distutils.errors import CCompilerError from distutils.errors import DistutilsExecError from distutils.errors import DistutilsPlatformError class optional_build_ext(build_ext): """Allow the building of C extensions to fail. """ def run(self): try: build_ext.run(self) except DistutilsPlatformError as e: self._unavailable(e) def build_extension(self, ext): try: build_ext.build_extension(self, ext) except (CCompilerError, DistutilsExecError) as e: self._unavailable(e) def _unavailable(self, e): print('*' * 80) print("""WARNING: An optional code optimization (C extension) could not be compiled. Optimizations for this package will not be available!""") print() print(e) print('*' * 80) - ---------------------------- %< ----------------------------- Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlHitjoACgkQ+gerLs4ltQ7SJQCgrhgN58g9ztFPEkFAOM49Wu4p RpQAoLnboKietjTx3eXho1kyRvH3r2uN =qWRK -----END PGP SIGNATURE-----
participants (3)
-
Ben Darnell -
Paul Moore -
Tres Seaver