[Python-Dev] PEP 578: Python Runtime Audit Hooks

[Steve Dower]

Thanks Christian for responding - I endorse and support all your 
comments. (I'd hoped that by explicitly saying "this is not a sandbox" 
it would avoid people thinking it was a sandbox, but apparently I would 
have been better just to avoid the keyword completely...)

On 29Mar2019 0324, Christian Heimes wrote:
> On 29/03/2019 01.02, Victor Stinner wrote:
>> Here I expect a small overhead. But the global overhead will be
>> proportional to the number of hooks, no? Maybe it's not significant
>> with the proposed list of events, but it will be more significant with
>> 100 or 1000 events?
>>
>> I'm not saying that it's a blocker issue, I'm just thinking aloud to
>> make sure that I understood correctly :-)
> 
> The performance impact can be remedied and reduced with a simple check.
> If there is no audit hook installed, it's just a matter of a pointer
> deref + JNZ.

Yep, the performance case we care about is when there are no hooks 
attached, since that's the only time a user cannot do anything to 
improve performance themselves. See the "Performance Impact" section in 
the PEP.

In my implementation the cost is about as low as I can make it - see 
https://github.com/python/cpython/pull/12613/files#diff-f38879f4833a6b6847e556b9a07bf4edR115 
(looking at it again I can probably avoid the exception preservation and 
a few conditionals at the end)

Basically, PySys_Audit takes a format string and arguments, rather than 
making callers eagerly construct the tuple that gets passed to the hook, 
and only actually allocates when there is a hook to call. There aren't 
even any Py_INCREF's if there are no hooks. And as Christian says, it's 
a deref+JNZ.

Now, if someone has implemented a hook and that hook has performance 
issues, yeah things will slow down. In general, the places where we are 
interested in hooks is where calls are being made into the operating 
system, so most of them will also involve a few syscalls and the cost of 
the hook should be minimal in comparison. But there isn't another way to 
provide the functionality - offloading it to the OS just means the OS is 
going to suffer the performance penalty, so it really is just moving the 
blame elsewhere. I dislike playing that game.

Cheers,
Steve