[Python-Dev] libffi embedded in CPython

[Zachary Ware]

I started this message about 3 months ago; at this point I'm just
getting it posted so it stops rotting in my Drafts folder.

On Mon, Dec 22, 2014 at 3:49 PM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
> On Thu, Dec 18, 2014, at 14:13, Maciej Fijalkowski wrote:
>> ... http://bugs.python.org/issue23085 ...
>> is there any reason any more for libffi being included in CPython?
>
>
> Paul Moore wrote:
>> Probably the easiest way of moving this forward would be for someone
>> to identify the CPython-specific patches in the current version ...
>
> Christian Heimes wrote:
>> That's easy. All patches are tracked in the diff file
>> https://hg.python.org/cpython/file/3de678cd184d/Modules/_ctypes/libffi.diff
>
> That (200+ lines) doesn't seem to have all the C changes, such as the
> win64 sizeof changes from issue 11835.

I took a little bit of time to look at this a while ago.  Something
that quickly became clear that I'm not sure everybody is on the same
page about (I, for one, had no idea previously) is that libffi for
Windows and libffi for everything else are entirely separate beasts:
the Windows build pulls all of its libffi source from
Modules/_ctypes/libffi_msvc and doesn't even touch
Modules/_ctypes/libffi (which is libffi for nearly everything else).
Most of the files in libffi_msvc haven't been touched in about 4
years, except for a couple fixes by Steve Dower recently.
README.ctypes in that folder states that the files were taken from the
libffi CVS tree in 2004 and there have been some custom modifications
since then, so there is really very little resemblance between
libffi_msvc and current libffi.

There's also 'libffi_arm_wince' and 'libffi_osx'.  I would be fairly
amazed if Python itself built on any version of Windows CE; personally
I would be for ripping out any supposed support that we have for it
(though if somebody is actually using it and it works, it's not
hurting anything and should stay).  That's a completely different
issue, though.

I'm all for ditching our 'libffi_msvc' in favor of adding libffi as
another 'external' for the Windows build.  I have managed to get
_ctypes to build on Windows using vanilla libffi sources, prepared
using their configure script from within Git Bash and built with our
usual Windows build system (properly patched).  Unfortunately, making
things usable will take some work on ctypes itself, which I'm not
qualified to do. I'm happy to pass on my procedure and patches for
getting to the point of successful compilation to anyone who feels up
to fixing the things that are broken.

As for the lightly-patched version of libffi that we keep around for
Linux et. al., looking at the diff file I don't see that there's a
whole lot of reason to keep it around.

-- 
Zach