On 29Mar2019 0334, Christian Heimes wrote:
On 28/03/2019 23.35, Steve Dower wrote:
In order to observe actions taken by the runtime (on behalf of the caller), an API is required to raise messages from within certain operations. These operations are typically deep within the Python runtime or standard library, such as dynamic code compilation, module imports, DNS resolution, or use of certain modules such as ``ctypes``.
The following new C APIs allow embedders and CPython implementors to send and receive audit hook messages::
# Add an auditing hook typedef int (*hook_func)(const char *event, PyObject *args, void *userData); int PySys_AddAuditHook(hook_func hook, void *userData);
# Raise an event with all auditing hooks int PySys_Audit(const char *event, PyObject *args);
# Internal API used during Py_Finalize() - not publicly accessible void _Py_ClearAuditHooks(void);
The new Python APIs for receiving and raising audit hooks are::
# Add an auditing hook sys.addaudithook(hook: Callable[[str, tuple]])
# Raise an event with all auditing hooks sys.audit(str, *args)
Hooks are added by calling ``PySys_AddAuditHook()`` from C at any time, including before ``Py_Initialize()``, or by calling ``sys.addaudithook()`` from Python code. Hooks cannot be removed or replaced.
I wonder if the hooks could be replaced by a more efficient mechanism. These days, Linux, macOS, and most recently Windows  support dtrace probes. DTrace is a very powerful and efficient mechanism to trace user-space processes from Kernel space. At least we should consider to add DTrace probes to the auditing framework.
Calling into those frameworks will still require as much work as these hooks do, and also make it very difficult to do things like auditing unit tests or using pure-Python code when you're not concerned about initialization (e.g. in a long-running web server).
So I'm inclined to say that if you want those probes, you can enable them by adding a hook that calls into them? A similar argument is made for using ETW on Windows (which will work on versions of Windows that have been released, unlike DTrace) (and yes, this is a real argument I've already had over this proposal ;) ), so I really think leaving it open-ended and Python-specific is the best approach.
(Reading further down the link you provided, it seems DTrace in Windows will only be enabled for essentially-administrators. So that rules it out as a substitute for this proposal in my opinion.)