[Python-ideas] stdlib with its own release cycle ?

Michael Foord fuzzyman at gmail.com
Mon Oct 26 18:20:19 CET 2009

2009/10/26 Guido van Rossum <guido at python.org>

> On Mon, Oct 26, 2009 at 2:47 AM, Michael Foord <fuzzyman at gmail.com> wrote:
> > Ironclad itself has binary compatibility with Python C extensions, they
> > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work with
> > these extensions and on the JVM would use its FFI.
> Ok, but that assumes the C extensions are already compiled for CPython
> on that platform (presumably .NET).

It uses the standard compiled binaries for CPython on Windows. Binary
compatibility - so no need to compile a special version, just use the same
binaries you use for CPython.

> This sounds limiting, and hacky
> (need lots of faith in the DLL loader).

It needs an *understanding* of the DLL loader sure. You seem to be relying a
lot on faith these days (well - two emails today anyway).

> I also assume that calls back
> into the CPython runtime are somehow intercepted and somewhat limited
> to what is reasonable to support in the .NET world -- unless you
> basically link all of CPython in and then you might as well just make
> RPCs to CPython...

The CPython API is implemented so that calls to the API go through our code.
This works in both directions (C extensions calling back into Python and
Python calling into C extensions). Where the Python source code has C API
functions that just call other functions we can reuse a lot of the CPython
implementation, yes. This C code is running natively - it is *not* running
on .NET, but where we need the bridge we use the FFI.

The API functions that, for example, create a new type create an IronPython
type instead of a CPython type though. This is the whole raison d'etre of

It isn't limiting, what it gives you is the ability to use existing code
that depends on C extensions without having to write your own wrapper layer
or use RPC. So far there has been very little that has proved impossible
with this approach.

> > My understanding is that Android now allows native code,
> Theoretically, yes. I don't think it's a good use of the platform
> though, you get huge binaries, long startup times, and quite limited
> functionality. Check out "Android Scripting Environment".

Right. *Most* Java code is not running on Android though (in the same way as
most .NET code isn't running on .NET CE), so it could be useful if people
wanted it.

> > so if Dalvik has
> > the same FFI APIs and you can compile the Python extensions for it *and*
> > Jython runs on Dalvik (not currently the case I believe?)
> Not yet, and not easy (the bytecode generator would have to be
> retargeted and probably a bunch of unsupported Java features would
> have to be worked around).
> Phones are slow and small...

A lot of modern phones have 500mhz ARM processors and 128Mbyte RAM. Python
was running on desktop devices a lot slower and 'smaller' not very long ago.


> > The CoreCLR in Silverlight does run IronPython but because of the
> sandboxing
> > you can't call into native code, so you can't use Ironclad in the
> browser.
> > Despite the places where Ironclad doesn't work, *most* .NET code runs on
> > desktops or servers using the standard .NET framework - and this is where
> > Ironclad is used. Despite the variety of different JVMs this is an
> analogous
> > situation and I don't see why a port wouldn't be just as useful.
> Why not just use CPython? At Google we mix Python and Java using RPCs
> intead of mixing runtimes. It works quite well.
The use case for Resolver Systems (who created the Ironclad project) was
that having written a programmable spreadsheet (desktop application) written
in IronPython we had a lot of customers who wanted to use Numpy with it.

We could have told them to go away and write their own RPC layer. Instead we
decided to solve the problem. :-)

With the Ironclad compatibility layer C extensions can be used without
having to write an RPC layer and the data marshalling that involves. It
works well... Other architectures (multi-process and RPC) may be suitable to
some problems but it is nice to have the choice.

There are a couple of interesting aspects of Ironclad that might have wider
applicability in the future. IronPython doesn't have a GIL and doesn't use
reference counting for garbage collection, but Ironclad has to work with C
extensions that use both.

If (for example) Unladen Swallow were to (eventually) be successful in
removing the GIL and moved away from reference counting then it would be
possible to retain binary API (ABI) compatibility with extensions written
for 'standard' CPython.

The GIL is simply faked. IronPython code is not restricted by the GIL but
only one code path into Ironclad can acquire the GIL at a time.

For reference counting we have a hybrid system keeping 'bridge' objects
alive whilst the C extension has a reference to them but they may or may not
be in use from the IronPython side.

All the best,


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20091026/20931ca4/attachment.html>

More information about the Python-ideas mailing list