[New-bugs-announce] [issue46451] Possibly bad interaction with tracing and cython?

Sebastian Berg report at bugs.python.org
Thu Jan 20 23:54:22 EST 2022

New submission from Sebastian Berg <sebastian at sipsolutions.net>:

Starting here, but there could be Cython interaction or something else in theory.  But, when running the following:

* Python 3.10.1 (not 3.9.9, debug version or not)
* Setting a tracing function  (not setting a trace-function will fix the issue)
* Running Cython (maybe also C code) calling back into Python (the best I can tell)

It can happen that module globals in the called funtions scope seem to be modified.  Oddly enough to a value used in the locals of that function?!

The pandas issue:


has a reproducer (sorry that it takes NumPy and pandas for now).  I will paste it at the end here also.

I can find that the value is modified by the time the `trace` function is called.  No other "trace" triggers are processed before in this example (Cython calls other functions in NumPy, but all of those are C implemented, so I guess that is why).
The function in question (unlike all?) should further be called with `__Pyx_PyFunction_FastCall`, so that is probably an important data point: Fastcall protocol seems involved.

(Reproducible using NumPy 1.21.5 and Pandas 1.3.5, but except maybe pandas due to the Cython version, I don't expect version dependency.)

The output of the script is very brief:

    Something happened here, `np.core.numeric.dtype IS np.dtype`
    <frame at 0x7ff8d84416c0, file '/home/sebastian/forks/numpy/build/testenv/lib/python3.10/site-packages/numpy/core/numeric.py', line 289, code full> call None

The full reproducer script is:

import sys
import numpy as np
import pandas as pd

from numpy.core import numeric

stop = False
def trace(frame, event, arg):
    global stop
    if stop:
        return None

    if np.core.numeric.dtype is not np.dtype:
        print("Something happened here, `np.core.numeric.dtype IS np.dtype`")
        print(frame, event, arg)
        stop = True
        print(frame, event, arg)

    return trace


pd._libs.lib.maybe_convert_objects(np.array([None], dtype=object))

For completeness, the Cython code calling the NumPy function in question, is (likley, there is more, this is Cython, I just cut it out a bit :)):

  if (PyFunction_Check(__pyx_t_5)) {
    PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_t_6, Py_False};
    __pyx_t_15 = __Pyx_PyFunction_FastCall(__pyx_t_5, __pyx_temp+1-__pyx_t_8, 2+__pyx_t_8); if (unlikely(!__pyx_t_15)) __PYX_ERR(0, 2441, __pyx_L1_error)
    __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0;
    __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0;
  } else

components: Interpreter Core
messages: 411082
nosy: seberg
priority: normal
severity: normal
status: open
title: Possibly bad interaction with tracing and cython?
type: crash
versions: Python 3.10

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list