[pypy-dev] Reference counting

[Marko Tasic]

Maciej,

> What do you mean by explicit reference count? One that you manipulate> from the source of the Python interpreter? Then no. But if you mean> "can I have arbitrary fields on objects depending on the GC strategy"> then yes. The object layout is completely orthogonal to how the Python> interpreter is implemented.

I meant that I need to manually increment and decrement
refcount from Python interpreter, and also have arbitrary
fields on objects depending on the GC strategy.

> People from unladen swallow tried to use the Cosmos GC or was it something else?

I think it is not that one. This is something different. I haven't had
any motivation
to fix CPython GC because I don't use it anymore for production code unless it
requires py2exe/py2app and PySide/Qt. PyPy is simply more challenging, ATM :)

My approach sacrifices memory but does not spend time on traversing objects.
Idea is not to have pauses caused by tracing GCs. Anyways, your approach
probably beats me, but anyway I want to experiment and research.

> Again does that answer your questions?

Yes, you did ;)

> PS. it sometimes easier to discuss such stuff on IRC than via mail.

I'll find some time soon to discuss this with you guys.

Cheers,Marko Tasic



On Sat, Jan 7, 2012 at 3:03 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Sat, Jan 7, 2012 at 3:54 PM, Marko Tasic <mtasic85 at gmail.com> wrote:
>> Maciej,
>>
>> Thank you for an express response.
>>
>>> Hi Marko, we're very glad to hear that!
>>
>> Off topic, but I have to mention that I've been an ambassador of PyPy
>> for last few years.
>> In beginning, it was hard, but now results that you've achieved are
>> obvious and promising,
>> and everyone starts to trust your implementation. However, I have to
>> say that there
>> are many notoriously xenophobic developers. Funny thing is that they
>> trust how their
>> source code is written but not how its executed.
>
> Thanks :)
>
>>
>>> * refcounting is implemented as a transformation of flow graphs. See
>>> pypy/rpython/memory/gctransform/refcounting.py for details. This
>>> approach works very well in the sense we don't have to maintain
>>> refcounts by hand in the interpreter (we'll never do that). This is
>>> all well and good, however it also means there is a significant
>>> redundancy in references. You would need to either implement a smarter
>>> policy or a refcount-removing optimization (typically run after
>>> inlining) in order for that to be viable.
>>
>> Is there a way to always have explicit reference counting field for all objects?
>> I know this is bad, but I want to go even further. I would like to have even
>> more fields in object structure besides refcount field. Reason for this is a
>> way to test correctness of alternative RC implementations that handle cycles.
>> I guess that I should turn off inlining?
>
> What do you mean by explicit reference count? One that you manipulate
> from the source of the Python interpreter? Then no. But if you mean
> "can I have arbitrary fields on objects depending on the GC strategy"
> then yes. The object layout is completely orthogonal to how the Python
> interpreter is implemented.
>
>>
>>> * we generally never cared about avoiding cycles in the python
>>> interpreter itself. I don't think pypy's refcounting implementation
>>> comes with cycle detector included, but don't quote me on that
>>
>> I've developed an alternative way of handling cycles related to reference
>> counting without using tracing GC, or traversing items in container objects
>> such are list, dict, set, etc. Still checking algorithm and testing correctness,
>> but everything has been fine until now. It is not my intention to make anything
>> spectacular but just to research this field.
>>
>> If anyone has interest in my reference counting mechanism can check:
>> http://code.google.com/p/cosmos-lang/wiki/CosmosRefCounter
>> http://code.google.com/p/cosmos-lang/source/browse/doc/corc.py
>
> People from unladen swallow tried to use the Cosmos GC or was it something else?
>
> Again does that answer your questions?
>
> PS. it sometimes easier to discuss such stuff on IRC than via mail.
>
> Cheers,
> fijal