On Tue, Jul 17, 2018 at 4:34 AM, Victor Stinner <vstinner@redhat.com> wrote:

IMHO you need a different approach to implement optimizations. For
example, use your objects which don't rely on the GIL to be
consistent. Sadly, I have no experience with that and so cannot
provide any example.

I DO NOT understand the technical details, but IIUC, numpy releases the GIL while doing pure numpy operations.

(and I have experimented with it, and it does seem to be the case -- that is, you can get enhanced performance with computationally heavy numpy code an multiple threads)

So maybe some lessons learned there.

The key differences may be that numpy arrays are full-fledged python objects, but they are not (generally) containers of python objects -- that is, the actual values are i memory accessed via a C pointer. Whereas a PyList holds pointers to Python objects, so even if are confident in the stability of the list itself, the objects in the list may not be stable.

 For example, os.stat() writes into a memory block allocated
by Python. But it's a raw memory block, it's much simpler than a
Python object like a tuple.

exactly. -- so is a numpy .data element

-CHB




--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov