[New-bugs-announce] [issue46451] Possibly bad interaction with tracing and cython?

[Sebastian Berg]

New submission from Sebastian Berg <sebastian at sipsolutions.net>:

Starting here, but there could be Cython interaction or something else in theory.  But, when running the following:

* Python 3.10.1 (not 3.9.9, debug version or not)
* Setting a tracing function  (not setting a trace-function will fix the issue)
* Running Cython (maybe also C code) calling back into Python (the best I can tell)

It can happen that module globals in the called funtions scope seem to be modified.  Oddly enough to a value used in the locals of that function?!

The pandas issue:

    https://github.com/pandas-dev/pandas/issues/41935

has a reproducer (sorry that it takes NumPy and pandas for now).  I will paste it at the end here also.

I can find that the value is modified by the time the `trace` function is called.  No other "trace" triggers are processed before in this example (Cython calls other functions in NumPy, but all of those are C implemented, so I guess that is why).
The function in question (unlike all?) should further be called with `__Pyx_PyFunction_FastCall`, so that is probably an important data point: Fastcall protocol seems involved.

(Reproducible using NumPy 1.21.5 and Pandas 1.3.5, but except maybe pandas due to the Cython version, I don't expect version dependency.)

The output of the script is very brief:

    Something happened here, `np.core.numeric.dtype IS np.dtype`
    <frame at 0x7ff8d84416c0, file '/home/sebastian/forks/numpy/build/testenv/lib/python3.10/site-packages/numpy/core/numeric.py', line 289, code full> call None


The full reproducer script is:


import sys
import numpy as np
import pandas as pd

from numpy.core import numeric

stop = False
def trace(frame, event, arg):
    global stop
    if stop:
        return None

    if np.core.numeric.dtype is not np.dtype:
        print("Something happened here, `np.core.numeric.dtype IS np.dtype`")
        print(frame, event, arg)
        stop = True
    else:
        print(frame, event, arg)

    return trace

sys.settrace(trace)

pd._libs.lib.maybe_convert_objects(np.array([None], dtype=object))


For completeness, the Cython code calling the NumPy function in question, is (likley, there is more, this is Cython, I just cut it out a bit :)):

  #if CYTHON_FAST_PYCALL
  if (PyFunction_Check(__pyx_t_5)) {
    PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_t_6, Py_False};
    __pyx_t_15 = __Pyx_PyFunction_FastCall(__pyx_t_5, __pyx_temp+1-__pyx_t_8, 2+__pyx_t_8); if (unlikely(!__pyx_t_15)) __PYX_ERR(0, 2441, __pyx_L1_error)
    __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0;
    __Pyx_GOTREF(__pyx_t_15);
    __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
  } else
  #endif

----------
components: Interpreter Core
messages: 411082
nosy: seberg
priority: normal
severity: normal
status: open
title: Possibly bad interaction with tracing and cython?
type: crash
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46451>
_______________________________________