[Python-Dev] Packaging JIT-less versions of Python

[Collin Winter]

Hey David,

On Mon, Mar 1, 2010 at 7:29 PM, David Malcolm <dmalcolm at redhat.com> wrote:
> On Mon, 2010-03-01 at 15:35 -0800, Collin Winter wrote:
[snip]
>> - How would you prefer to build the JIT-less package (current options:
>> via a ./configure flag; or by deleting _llvmjit.so from the
>> JIT-enabled package)?
>> - Would the two packages be able to exist side-by-side, or would they
>> be mutually exclusive?
>
> I have a particular interest in ABI compatibility: if turning JIT on and
> off is going to change the ABI of extension modules, that would be a
> major pain, as I hope that we will have dozens of C extension modules
> available via RPM for our Python 3 stack by the time of the "great
> unladen merger".

Do you have a good way of testing ABI compatibility, or is it just
"build modules, see if they work"? Some general way of testing ABI
compatibility would be really useful for PEP 384, too.

> So I'm keen for the ability to toggle the JIT code in the face of bugs
> and have it not affect ABI.  "-Xjit" will do this at runtime (once
> that's renamed), but I think it would be useful to be able to toggle the
> JIT on/off default during the build, so that I can fix a broken
> architecture for non-technical users, but have individual testers opt
> back in with -Xjit whilst tracking down a major bug.

That's something we can definitely do: you'd just change the default
value for the -Xjit flag from "whenhot" to "never". Those individual
testers would pass -Xjit=whenhot to opt back in. We could make that a
./configure flag if it would be useful to you and the other distros.

> In either case, I don't want to have to recompile 30 extension modules
> to try with/without JIT; that would introduce too much change during
> bug-hunts, and be no fun at all.

That would suck indeed; I want to avoid that. I think that kind of
thing falls under PEP 384, which we will have to obey once it is
accepted.

> (In the blue-sky nirvana future, I'd love to be able to ship
> ahead-of-time compiled versions of the stdlib, pre-optimized based on
> realworld workloads.  Back in my reality, though, I have bugs to fix
> before I can work on _that_ patch :( )

Reid Kleckner may be looking at that for his Master's project. It's
definitely doable.

>> My strong preference would be to have the JIT included by default so
>> that it receives as much testing as possible.
>
> Sounds reasonable.  Hope the above made sense and is useful.

Thanks for your perspective,
Collin Winter