(resending to cython-devel list since my first attempt bounced) On Tue, Dec 11, 2018, 11:56 Nathaniel Smith <njs@pobox.com wrote:

FYI – you should be aware that subinterpreters are poorly tested (AFAIK ceph is the third project to try using them, ever), not well supported in general, and there has even been some discussion of removing them from CPython. For example, numpy has never supported being used with subinterpreters, and currently has no plans to fix this. I suspect other extension modules are in similar positions, but since the bugs that subinterpreters trigger are often really hard to detect or debug, no one really knows.

On Tue, Dec 11, 2018, 11:11 Ricardo Dias <rdias@suse.com wrote:

...

Hi Cython developers,

In the recent Cython 0.29 version was introduced a commit [1] that hinders the usage of python subinterpreters.

I discovered this the hard way when suddenly a component I was working on started to crash. The component in question is the ceph-mgr daemon from the Ceph project [2].

Python subinterpreters are the basic building block for the plugin/module architecture of ceph-mgr. Each "manager module" runs in its own python subinterpreter. Furthermore, all python bindings for the client libraries of Ceph, such as librados, librbd, libcephfs, and librgw, are implemented as Cython modules, and in the particular case of librados, all ceph-mgr plugin modules import the rados Cython module upon initialization.

In practice, with Cython 0.29 we can only load one module, because the following modules will refuse to load.

After discovering this issue, we "temporarily" prevent the issue by restricting the version of Cython as a dependency [3]. But we don't want to keep this restriction indefinitely and would prefer a fix from the Cython side.

Do you think it's feasible to implement a flag to disable the safe guard introduced in [1]? That way we could re-enable subinterpreters at our own risk.

[1]

https://github.com/cython/cython/commit/7e27c7cd51a2f048cd6d3c246740cd977f8d... [2] https://github.com/ceph/ceph [3] https://github.com/ceph/ceph/pull/25328

-- Ricardo Dias Senior Software Engineer - Storage Team SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel