Hi libffi has bugs sometimes (like this http://bugs.python.org/issue17580). Now this is a thing that upstream fixes really quickly, but tracking down issues on bugs.python.org is annoying (they never get commited as quickly as the upstream). is there a good reason why cpython has it's own copy of libffi? I understand historical reasons, but PyPy gets along relying on the system library, so maybe we can kill the inclusion. Cheers, fijal
In article <CAPZV6o_HRmkU_=1MZPJDzJGZOBTwBZneV-rjtqBP8P-6tu+AHA@mail.gmail.com>, Benjamin Peterson <benjamin@python.org> wrote:
The cpython copy also has custom OS X patches. I've never looked at them so I don't have a feel for how much work would be involved in migrating to current upstream. If it's just a matter of supporting universal builds, it could be done with some Makefile hacking to do a lipo dance. Ronald, any additional thoughts? http://bugs.python.org/issue15194 Currently, for the OS X installer builds, we build a number of third-party libs that are either missing from OS X (like lzma) or are too out-of-date on the oldest systems we support. It would be useful to generalize the third-party lib support and move it out of the installer build process so that all builds could take advantage of the libs if needed. libffi could be added to those. Of course, that wouldn't help for Windows builds. -- Ned Deily, nad@acm.org
On 18 apr. 2013, at 18:09, Ned Deily <nad@acm.org> wrote:
It is probably just a matter of supporting universal builds, but I haven't checked the state of upstream libffi in the last couple of years. The libffi_osx tree is a fork from upstream that started around the time of OSX 10.4, and possibly earlier. As Thomas mentioned the upstream maintainer weren't very responsive in the past, at the time I hacked on libffi (IIRC for Darwin/i386 support) it wasn't even clear how the maintainers could be reached. Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang. Ronald
On Thu, Apr 18, 2013 at 11:17 PM, Ronald Oussoren <ronaldoussoren@mac.com> wrote:
Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang.
If you mean http://bugs.python.org/issue17136, I think that has been fixed in libffi upstream? Cheers, Dirkjan
On 19 Apr, 2013, at 10:13, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
Appearently, but not in /usr/lib/libffi.dylib (at least nog when I checked it last time, which was at least one minor release of 10.8 ago). Which means that the binary installers will still have to use a bundled copy of libffi, but as Ned wrote that's not really a problem because the script that builds those installers already includes some other libraries that are not available on OSX or are too old. That said, I haven't tested yet if upstream libffi actually works although others seem to have. Ronald
On Thu, Apr 18, 2013 at 2:47 PM, Benjamin Peterson <benjamin@python.org> wrote:
Upstream seems to be incredibly responsive. Why not merge them there? Is it just "we can't patch distutils" crap? If we *really* need to provide libraries for systems that don't have them, how about simply having a separate download for those systems instead of using it on systems that *do* have them and in fact *do* have them in a better version. Note that the recent patches (one for 32bit, one for ARM) are easily security holes. I'm not 100% sure if segfault caused by stack misalignment is an exploitable thing, but you can definitely cause a segfault on any ctypes code, even if perfectly safe. Cheers, fijal
The only windows patch that is most certainly not in upstream is the ability to check and correct the stack pointer after a windows function call depending on the calling convention (stdcall or cdecl). Since this is windows 32-bit only (on windows 64-bit these calling conventions are aliases for the same thing), it would probably be good to remove the distinction between them for function calls.
Upstream seems to be incredibly responsive. Why not merge them there?
At the time when ctypes was implemented, this was different. They didn't even do libffi releases, IIRC. Thomas
In article <CAPZV6o_HRmkU_=1MZPJDzJGZOBTwBZneV-rjtqBP8P-6tu+AHA@mail.gmail.com>, Benjamin Peterson <benjamin@python.org> wrote:
The cpython copy also has custom OS X patches. I've never looked at them so I don't have a feel for how much work would be involved in migrating to current upstream. If it's just a matter of supporting universal builds, it could be done with some Makefile hacking to do a lipo dance. Ronald, any additional thoughts? http://bugs.python.org/issue15194 Currently, for the OS X installer builds, we build a number of third-party libs that are either missing from OS X (like lzma) or are too out-of-date on the oldest systems we support. It would be useful to generalize the third-party lib support and move it out of the installer build process so that all builds could take advantage of the libs if needed. libffi could be added to those. Of course, that wouldn't help for Windows builds. -- Ned Deily, nad@acm.org
On 18 apr. 2013, at 18:09, Ned Deily <nad@acm.org> wrote:
It is probably just a matter of supporting universal builds, but I haven't checked the state of upstream libffi in the last couple of years. The libffi_osx tree is a fork from upstream that started around the time of OSX 10.4, and possibly earlier. As Thomas mentioned the upstream maintainer weren't very responsive in the past, at the time I hacked on libffi (IIRC for Darwin/i386 support) it wasn't even clear how the maintainers could be reached. Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang. Ronald
On Thu, Apr 18, 2013 at 11:17 PM, Ronald Oussoren <ronaldoussoren@mac.com> wrote:
Stripping libffi from python's source tree would be fine by me, but would require testing with upstream libffi. AFAIK system libffi on osx wouldn't be goog enough, it doesn't work properly with clang.
If you mean http://bugs.python.org/issue17136, I think that has been fixed in libffi upstream? Cheers, Dirkjan
On 19 Apr, 2013, at 10:13, Dirkjan Ochtman <dirkjan@ochtman.nl> wrote:
Appearently, but not in /usr/lib/libffi.dylib (at least nog when I checked it last time, which was at least one minor release of 10.8 ago). Which means that the binary installers will still have to use a bundled copy of libffi, but as Ned wrote that's not really a problem because the script that builds those installers already includes some other libraries that are not available on OSX or are too old. That said, I haven't tested yet if upstream libffi actually works although others seem to have. Ronald
On Thu, Apr 18, 2013 at 2:47 PM, Benjamin Peterson <benjamin@python.org> wrote:
Upstream seems to be incredibly responsive. Why not merge them there? Is it just "we can't patch distutils" crap? If we *really* need to provide libraries for systems that don't have them, how about simply having a separate download for those systems instead of using it on systems that *do* have them and in fact *do* have them in a better version. Note that the recent patches (one for 32bit, one for ARM) are easily security holes. I'm not 100% sure if segfault caused by stack misalignment is an exploitable thing, but you can definitely cause a segfault on any ctypes code, even if perfectly safe. Cheers, fijal
The only windows patch that is most certainly not in upstream is the ability to check and correct the stack pointer after a windows function call depending on the calling convention (stdcall or cdecl). Since this is windows 32-bit only (on windows 64-bit these calling conventions are aliases for the same thing), it would probably be good to remove the distinction between them for function calls.
Upstream seems to be incredibly responsive. Why not merge them there?
At the time when ctypes was implemented, this was different. They didn't even do libffi releases, IIRC. Thomas
participants (7)
-
Benjamin Peterson -
Dirkjan Ochtman -
Maciej Fijalkowski -
Ned Deily -
Ronald Oussoren -
Stefan Behnel -
Thomas Heller