Python packaging + conditional use of C library shipped with the package
Hi, I would like to make a package that is able to use a system's given C-library if found, or compile its own version shipped with the package. This is bringing up two questions: - Is there a distutils/distribute facility to help test for the presence (and version) of a C library, or do I have to roll my own system ? - When having the source for a C library shipping with a package, is there a way to get distutils/distribute compile it, and get it seen by Python at runtime (so I can just use ctypes, or cffi, and even C extensions in other Python package see the headers and compiled libraries) ? Best, Laurent PS: I have noticed that pyzmq seems to be doing a lot of that, but they also appear to have a relatively big set of custom code do so (setup.py is already a little longer than most install scripts, to which a complete package 'buildutils' should be added)... this makes me anticipate that the answer to my two questions will be "no" ( :/ ) but I'd like a confirmation from the experts.
On Mon, 08 Jul 2013 13:01:25 +0200, Laurent Gautier wrote:
Hi,
I would like to make a package that is able to use a system's given C-library if found, or compile its own version shipped with the package.
This is bringing up two questions:
- Is there a distutils/distribute facility to help test for the presence (and version) of a C library, or do I have to roll my own system ?
No. Here is a rather clean script that encapsulates pkg-config: http://git.enlightenment.org/bindings/python/python-efl.git/tree/setup.py
- When having the source for a C library shipping with a package, is there a way to get distutils/distribute compile it, and get it seen by Python at runtime (so I can just use ctypes, or cffi, and even C extensions in other Python package see the headers and compiled libraries) ?
I don't think you should do any kind of bundling. That will easily decuple the complexity of your setup scripts, for a subpar result. Point to the C project and let people install it as they will.
On Mon, Jul 08, 2013 at 01:01:25PM +0200, Laurent Gautier wrote:
Hi,
I would like to make a package that is able to use a system's given C-library if found, or compile its own version shipped with the package.
We do something similar in pycryptopp, but instead of automatically testing for the locally-available C library, we just ask the human to manually pass "--disable-embedded-cryptopp" if they want it to attempt to link to a library external to its own bundled one: https://tahoe-lafs.org/trac/pycryptopp/browser/git/setup.py?annotate=blame&rev=f789ed951b49b33e7cc49d16fdc8b398f7ec7223
- Is there a distutils/distribute facility to help test for the presence (and version) of a C library, or do I have to roll my own system ?
If you succeed at this, I'd like to know how you did it! Maybe we could do something similar for pycryptopp.
- When having the source for a C library shipping with a package, is there a way to get distutils/distribute compile it, and get it seen by Python at runtime (so I can just use ctypes, or cffi, and even C extensions in other Python package see the headers and compiled libraries) ?
I don't understand the question. This sounds like the normal thing that distutils has always done for modules made up of compiled C code. By the way, if I were starting pycryptopp today I would use cffi. (And I would name it "crpyto".) Regards, Zooko
On 07/08/2013 05:33 PM, zooko wrote:
Hi,
I would like to make a package that is able to use a system's given C-library if found, or compile its own version shipped with the package. We do something similar in pycryptopp, but instead of automatically testing for
On Mon, Jul 08, 2013 at 01:01:25PM +0200, Laurent Gautier wrote: the locally-available C library, we just ask the human to manually pass "--disable-embedded-cryptopp" if they want it to attempt to link to a library external to its own bundled one:
Thanks for this, I'll look at it.
- Is there a distutils/distribute facility to help test for the presence (and version) of a C library, or do I have to roll my own system ? If you succeed at this, I'd like to know how you did it! Maybe we could do something similar for pycryptopp.
pyzmq seems to be doing this sort of thing (from a quick look, it seems like they are implementing something similar to what autoconf can do), but that's a lot of work if each project must reimplement its own.
- When having the source for a C library shipping with a package, is there a way to get distutils/distribute compile it, and get it seen by Python at runtime (so I can just use ctypes, or cffi, and even C extensions in other Python package see the headers and compiled libraries) ? I don't understand the question. This sounds like the normal thing that distutils has always done for modules made up of compiled C code.
May be I was not clear enough. What I mean is the C library is just a C library, not a C-extension to Python. For example: mypackage/ clib/ myClibrary.c myClibrary.h src/ mypackage.py (calling myClibrary.so) myotherpackage/ clib/ myotherClibrary.c (requiring myClibrary.h, and obviously link to myClibrary.so) I'll look at the code example you are giving and see it this is automagically taken care of by the package installation system. L.
By the way, if I were starting pycryptopp today I would use cffi. (And I would name it "crpyto".)
Regards,
Zooko
participants (3)
-
Gabriel de Perthuis -
Laurent Gautier -
zooko