[Python-3000] Ctypes as cross-interpreter C calling interface

[Jean-Paul Calderone]

On Mon, 14 Aug 2006 11:08:56 -0700, Guido van Rossum <guido at python.org> wrote:
>On 8/14/06, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>>On Mon, 14 Aug 2006 09:09:49 -0700, Guido van Rossum <guido at python.org> 
>>wrote:
>I also consider .NET's CLR a success, based on the testimony of Jim
>Hugunin (who must be Microsoft's most reluctant employee :).
>
>And I see the JVM as a successful case too -- Jython can link to
>anything written in Java or compiled to JVM bytecode, and so can other
>languages that use JVM introspection the same way as Jython (I hear
>there's  Ruby analogue).

These successes are necessarily limited in scope.  Jython can use any
Java library, and that's great, as far as it goes.  Clearly, though,
it isn't a complete solution.  Relying on Parrot to have a rich library
of wrapper modules seems ill advised.  If it /already/ had a rich library,
then maybe it would seem more reasonable.

>
>The major difference between all these examples and ctypes is that
>ctypes has no way of introspecting the wrapped library; you have to
>repeat everything you know about the API in your calls to ctypes (and
>as was just shown in another thread about 64-bit issues, that's not
>always easy).

The codegenerator package which is closely related to ctypes is capable of
this as well.  PyPy has a complete ctypes-based OpenSSL wrapper which is
automatically generated.

>That's not exactly the point I am making. I find Parrot's approach,
>assuming the project won't fail due to internal friction, much more
>long-term viable than ctypes. The big difference being (I hope)
>introspective generation of APIs rather than having to repeat the
>linkage information in each client language.

Given the existence of codegenerator, do you still find Parrot's approach
more viable?  It seems to me that it easily levels the playing field, and
makes ctypes still more attractive than Parrot, since it side-steps the
not insignificant internal political issues with the Parrot team.

>This seems a mostly theoretical viewpoint to me. Can you point me to
>an example of a Python-like language that is successful in reusing a
>Lisp runtime? (And I don't consider Lisp or Scheme Python-like in this
>context. ;-)

PyPy has a Common Lisp backend.  It's not the primary target, but it's not
inconceivable that it could someday provide an ffi from a Common Lisp
runtime to Python programs.

There has also been work done on an IL backend for PyPy.  This could be
used to make any CLR library available to Python programs.

Of course, with those two examples in hand, we see a fundamental drawback
to the Parrot-style solution (of which these are both essentially examples).
What if I want to use the CL FFI at the same time as a library exposed via
.NET?  I'm out of luck.  Had the libraries I wanted both been wrapped with
ctypes, I could have used them both from either runtime.

In general, what are alternate runtimes like PyPy to do if Parrot becomes
the de facto standard for extension modules?  Link against Parrot?  Suffer
without those modules until someone does a custom binding for that runtime?

Jean-Paul