[Cython] CEP1000: Native dispatch through callables
Dag Sverre Seljebotn
d.s.seljebotn at astro.uio.no
Sat Apr 14 21:04:06 CEST 2012
On 04/14/2012 08:10 PM, Robert Bradshaw wrote:
> On Sat, Apr 14, 2012 at 1:56 AM, Stefan Behnel<stefan_ml at behnel.de> wrote:
>> Dag Sverre Seljebotn, 14.04.2012 10:41:
>>> Greg Ewing wrote:
>>>> Dag Sverre Seljebotn wrote:
>>>>> 1) It doesn't work well with multiple interpreter states. Ok, nothing
>>>>> works with that at the moment, but it is on the roadmap for Python
>>>> Is it really? I got the impression that it's not considered feasible,
>>>> since it would require massive changes to the entire implementation
>>>> and totally break the existing C API. Has someone thought of a way
>>>> around those problems?
>>> I was just referring to the offhand comments in PEP3121, but I guess that PEP had multiple reasons, and perhaps this particular arguøent had no significance...
>> IIRC, the last status was that even after this PEP, Py3 still has serious
>> issues with keeping extension modules in separate interpreters. And this
>> probably isn't worth doing anything about because it won't work without a
>> major effort in all sorts of places. And I never heard that any extension
>> module even tried to support this.
>> I don't think we should invest too much thought into this direction.
A shame; short of getting rid of the GIL, multiple interpreter states
would be my favourite shared-memory parallel computation approach, as
they could share NumPy buffers (and other C-level structures), without
worrying about allocating in process-shared memory (and most data
structures, like std::map, won't even work (or, work portably and
reliably) in process-shared memory anyway).
Multiple seperate interpreter states would be a very nice way of getting
the benefits of multi-threading without the disadvantages.
> I had never even heard of this PEP before this thread, but this
> certainly seems reasonable to me. Aside from this, there is some value
> with the inlined signature in that a pure C library can easily support
> the ABI as well.
Yes -- I think both "sides" of this discussion prefer their approach out
of aesthetics more than performance :-)
I'll post a revamped CEP in a minute to at least try to sum them up.
> Has anyone done any experiments/timings to see if having constants vs.
> globals even matters?
It'd be interesting to see; won't have time myself until Monday earliest.
More information about the cython-devel