[Numpy-discussion] Long-standing issue with using numpy in embedded CPython

[Yang Zhang]

On Fri, Dec 9, 2011 at 12:31 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On Thu, Dec 8, 2011 at 23:15, Yang Zhang <yanghatespam at gmail.com> wrote:
>> On Tue, Oct 4, 2011 at 12:05 PM, Yang Zhang <yanghatespam at gmail.com> wrote:
>>> On Tue, Oct 4, 2011 at 1:28 AM, Robin <robince at gmail.com> wrote:
>>>> On Mon, Oct 3, 2011 at 9:42 PM, Yang Zhang <yanghatespam at gmail.com> wrote:
>>>>> It turns out that there's a long-standing problem in numpy that
>>>>> prevents it from being used in embedded CPython environments:
>>>>
>>>> Just wanted to make the point for reference that in general Numpy does
>>>> work fine in (non-threaded) embedded CPython situations, see for
>>>> example pymex [1] which embeds Python + Numpy in a Matlab mex file and
>>>> works really well.
>>>>
>>>> This seems to a be a problem specific to Jepp.
>>>>
>>>> Just wanted to mention it in case it puts someone off trying something
>>>> unnecessarily in the future.
>>>
>>> My (second-hand) understanding is that this is a problem with having
>>> multiple CPython interpreters, which both Jepp and numpy utilize,
>>> incompatibly - is that right?  I.e., if either one were restricted to
>>> using a single CPython interpreter, we wouldn't see this problem?
>>>
>>> I'm curious how to disable threads in numpy (not an ideal solution).
>>> Googling seems to point me to setting NPY_ALLOW_THREADS to
>>> 0....somewhere.
>>
>> Anyone?
>
> numpy does not use multiple interpreters. The threading options have
> nothing to do with multiple interpreters, and will not let you use
> multiple CPython interpreters in your application. The problem is that
> Python does not have good isolation between multiple interpreters for
> extension modules. Many extension modules happen to work in this
> environment, but numpy is not one of them. We have some global state
> that we need to keep, and this gets interfered with in a multiple
> interpreter environment.

Thanks for the clarification.  Alas.  So is there no simple workaround
to making numpy work in environments such as Jepp?

>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
>   -- Umberto Eco
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



-- 
Yang Zhang
http://yz.mit.edu/