On Fri, Dec 11, 2020 at 6:08 AM Jean-Paul Calderone < exarkun@twistedmatrix.com> wrote:
On Fri, Dec 11, 2020 at 8:19 AM Jasper Phillips <jasperisjaded@gmail.com> wrote:
I'm using perspective broker to transfer objects in a networked game, which I'm having trouble unjellying -- the remote versions wind up with dangling twisted.persisted.crefutil._Dereference instances, so don't match the original objects.
I'm seeing this for objects that have circular references to each other. I've refactored things to mostly avoid circular references and sidestep this, but have one remaining case where I find circular references mean clearer code that I'm reluctant to refactor.
Is there some trick I'm missing to avoid _Dereferences?
No, it's supposed to Just Work™ so you've found a bug in some part of the implementation. If you can produce a minimal reproducing example then it may be worth a bug report. But after that, I'd suggest investigating HTTP or AMP as a replacement for PB.
Here's a test case demonstrating the bug: import sys from twisted.persisted.crefutil import _Dereference from twisted.spread import pb, jelly class RemoteCopyable( pb.Copyable, pb.RemoteCopy ): pass jelly.globalSecurity.allowInstancesOf( RemoteCopyable ) pb.setCopierForClassTree( sys.modules[__name__], pb.Copyable ) if __name__ == '__main__': # circular object ref cell = RemoteCopyable() cell.link = RemoteCopyable() cell.link.cell = cell # Mimic sending across network broker = pb.Broker() serializedCell = broker.serialize( cell ) remoteCell = broker.unserialize( serializedCell ) print( _Dereference is type(remoteCell.link.cell) ) This bug stems from twisted.spread.flavors.RemoteCopy.setCopyableState(), which works fine for Python 2! But the UTF fix for Python 3 broke circular references. It looks like this: def setCopyableState(self, state): if _PY3: state = {x.decode('utf8') if isinstance(x, bytes) else x:y for x,y in state.items()} self.__dict__ = state Here is a simple tweak that fixes the problem: def setCopyableState(self, state): if _PY3: for x, y in list(state.items()): if isinstance(x, bytes): del state[x] state[x.decode('utf8')] = y self.__dict__ = state Basically the reference unwinding that takes place in twisted.spread.jelly._Unjellier._unjelly_reference()'s call to ref.resolveDependants() relies upon setCopyableStates()'s passed in state being used directly, such that all matching references' __dict__ point to the same object. Hope this helps clarify. Is there some more formal location than this list that you'd like a bug report filed?