pytest segfault, not with -v

[MRAB]

On 2021-11-19 23:44, Marco Sulla wrote:
> On Fri, 19 Nov 2021 at 20:38, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> On 2021-11-19 17:48, Marco Sulla wrote:
>> > I have a battery of tests done with pytest. My tests break with a
>> > segfault if I run them normally. If I run them using pytest -v, the
>> > segfault does not happen.
>> >
>> > What could cause this quantical phenomenon?
>> >
>> Are you testing an extension that you're compiling? That kind of problem
>> can occur if there's an uninitialised variable or incorrect reference
>> counting (Py_INCREF/Py_DECREF).
> 
> Ok, I know. But why can't it be reproduced if I do pytest -v? This way
> I don't know which test fails.
> Furthermore I noticed that if I remove the __pycache__ dir of tests,
> pytest does not crash, until I re-ran it with the __pycache__ dir
> present.
> This way is very hard for me to understand what caused the segfault.
> I'm starting to think pytest is not good for testing C extensions.
> 
If there are too few Py_INCREF or too many Py_DECREF, it'll free the 
object too soon, and whether or when that will cause a segfault will 
depend on whatever other code is running. That's the nature of the 
beast: it's unpredictable!

You could try running each of the tests in a loop to see which one 
causes a segfault. (Trying several in a loop will let you narrow it down 
more quickly.)

pytest et al. are good for testing behaviour, but not for narrowing down 
segfaults.