[pypy-dev] GC problems with PyObject

[Carl Friedrich Bolz]

Hi Christian!

Christian Tismer wrote:
> I spent the last two days hunting refcounting problems
> with pypy generated extension modules.
>
> After infinitely long staring at the final flow graphs,
> I found out that our transition to the gctransform
> is not as complete as I assumed. Impossible too find
> by flow graphs, because we are still chiming extra increfs
> in from the backend, see funcgen.py .
>
> I tried to solve this by disabling all the extra increfs,
> but this doesn't work, yet. Something else must be adjusted.
> 
> The problem is an inconsistency about how getattr is handled.
> The code in funcgen.py always adds an incref when we access
> a field or array item of a PyObject.

I am not exactly surprised. When we wrote the gctransformer we were 
mostly annoyed by PyObjects and did the minimal thing to make all tests 
that use them pass.

> In gctransformation.py, there are deallocators created,
> which produce code like
> 
>   v_xxx = obj.xxx
>   pop_alive(v_xxx)
> 
> but this doesn't work for PyOnject, since funcget.py adds
> an extra ref, and the object stays as alive as it was before.
> 
> I hoped to solve this, assuming that our gc code is generic
> enough to get this right for PyObject as well. Maybe I'm missing
> something, have to give up. I'm probably anyway interfering
> with other people's work.

It would be quite helpful if you checked in a small test that shows the 
faulty behaviour. It's a bit hard to look for the problem if you have no 
way to reproduce it.

Anyway, as I said, the gctransformer is a complete mess right now and 
needs a rewrite -- which we will tackle as soon as we found out why our 
exception transformation crashes.

Cheers,

Carl Friedrich