[ python-Bugs-1464444 ] ctypes should be able to link with installed libffi

Tue Apr 11 13:13:17 CEST 2006

Bugs item #1464444, was opened at 2006-04-04 21:42
Message generated for change (Settings changed) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464444&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Matthias Klose (doko)
Assigned to: Thomas Heller (theller)
Summary: ctypes should be able to link with installed libffi

Initial Comment:
ctypes always uses the included libffi implementation
(which should be updated to current GCC HEAD). Pleae
add an option to build and link against a libffi, which
is installed on the system.

attached is a hack to just always build using the
installed libffi. I'm unsure how to find the correct
ffi implementation (maybe check for LIBFFI_H defined in
the ffi header?), and where to implement this check
(configure.ac --with-system-ffi=/usr?).

----------------------------------------------------------------------

>Comment By: Martin v. LÃ¶wis (loewis)
Date: 2006-04-11 13:13

Message:
Logged In: YES 
user_id=21627

Thanks for the patch. Committed as r45278.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2006-04-10 22:45

Message:
Logged In: YES 
user_id=60903

documentation added.

----------------------------------------------------------------------

Comment By: Martin v. LÃ¶wis (loewis)
Date: 2006-04-10 14:00

Message:
Logged In: YES 
user_id=21627

The code is fine, but it lacks documentation. There should
be some way to learn about --with-system-ffi, preferably by
either reading ./configure --help output, or reading README.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2006-04-10 12:44

Message:
Logged In: YES 
user_id=60903

Setup.local has the disadvantage that you have to edit the
file. I updated the patch to only have an effect, if
configure is passed --with-system-ffi.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2006-04-07 19:49

Message:
Logged In: YES 
user_id=11105

I'm astonished how flexible the build-system is :-), once
you leave Windows.

setup.py needs a change, though, so that the _ctypes/libffi
configure script is not run when not needed.

One possible advange of Modules/Setup.local is that one can
define additional compilation flags for ctypes - for
example, if ctypes would support it, to enable/disable
certain features (varargs function calls, for example).

----------------------------------------------------------------------

Comment By: Martin v. LÃ¶wis (loewis)
Date: 2006-04-06 23:46

Message:
Logged In: YES 
user_id=21627

If this configuration needs an operator decision, I think
this decision is best made in Modules/Setup.local. Anybody
who wants to use the local libffi installation can do so, by
writing a Setup.local that lists the source files, and gives
-lffi as a library option. Setup.dist could provide a
commented version of such a configuration.

setup.py *should* then skip over building _ctypes, as it
should find out that _ctypes was already built through Makefile.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2006-04-06 22:02

Message:
Logged In: YES 
user_id=11105

I do not really know.  Maybe using
distutils.sysconfig.get_config_var() ?

A pragmatic approach would be to parse pyconfig.h itself in
the setup script.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2006-04-06 21:45

Message:
Logged In: YES 
user_id=60903

maybe a configure switch --with-system-ffi could do it?
How to pass this to the setup.py file?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2006-04-06 21:17

Message:
Logged In: YES 
user_id=11105

I tried your patch on an AMD64 ubuntu 5.10, after I
installed the 4.0.1-4ubuntu9 libffi package.  It crashes in
ctypes/test/test_python_api.py, test_PyOS_snprintf.  IIRC,
libffi did not support varargs functions, but Andreas Degert
added support for them (for x86_64), and it works fine.

Ok, new features in libffi might not be the norm, but I see
no way how I can check whether a system-installed libffi
supports this feature or not.

Do *you* see a solution for that?

For the actual changes to setup.py, if the patch really is a
good idea:  Since I don't even know what a convience library
is I'll trust you fully that your patch does the right thing.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2006-04-06 15:12

Message:
Logged In: YES 
user_id=60903

> do any of the buildbot slaves have a system libffi
> installed?

the Debian and Ubuntu bots have libffi from gcc-4.1 installed.

I think in the past, there was a complaint about a
conflicting ffi.h, or maybe was this ffitarget.h? not sure
if the check for LIBFFI_H really helps with this. yes, if
gcc and ffi.h are installed from the same source / into the
same path, then the check for include dirs and libraries are
obsolete. the check for the different library names comes
from the observation that libgcj is linked against libffi as
a convenience library (for performance reasons?). These are
checked first so that not another external dependency is
introduced, if these libraries are available (not installed
by a standard gcc installation).

updating the code would at least make the library buildable
on hppa (or maybe this is just the failed (it's really a
32bit runtime, but hppa64-linux is detected).

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2006-04-06 14:34

Message:
Logged In: YES 
user_id=11105

I tend to accept this patch.  Martin, any objections?

Matthias: do any of the buildbot slaves have a system libffi
installed?

About the code:
And, shouldn't it be enough to let the compiler find ffi.h?
 I assume that the presence of this file should indicate
that libffi.so is installed - why do we have to scan this
file for '#define LIBFFI_H', and why do we have to search
for the ffi shared library?

Would updating the included libffi code to GCC HEAD bring
any advantages?  The currently included code has been
patched in a few places, to avoid some of the compiler
warnings, and to allow compilation on OS X 10.3 (which does
not know the 'live_support' segment attribute, see Python
bug 1460514).

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2006-04-05 00:43

Message:
Logged In: YES 
user_id=60903

It's not an external dependency, if it's not found. The size
is not the argument. For distributors it's an additional
maintainance burden to update copies of libraries. Security
advisories for zlib (and pcre?) show this.

patch updated.

----------------------------------------------------------------------

Comment By: Martin v. LÃ¶wis (loewis)
Date: 2006-04-05 00:14

Message:
Logged In: YES 
user_id=21627

Why is it desirable to link with the system ffi? That just
adds an external dependency, for a negligable space saving
(on x86, libffi.so.4.0.1 is 7840 bytes).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1464444&group_id=5470