[pypy-dev] _marshal EOF error?

Jonathan Slenders jonathan at slenders.be
Wed Aug 8 23:27:01 CEST 2012


Hi everyone,

I have almost a sandlib.py library ready for the Twisted Matrix framework.
(The sandlib which is shipped with PyPy is not usable within Twisted.)

However, the deserialisation of marshalled messages fails when the sandbox
Python process does very big write system calls. Actually, the problem is
that the write calls arrive in multiple chunks in the outReceived of the
ProcessProtocol (Twisted code). Marshal.loads thinks the first chunk is
valid on its own while it isn't yet.

Long story short, there is a difference between the Python built-in and the
pure python _marshall.py from PyPy which should be used for the sandbox
hypervisor. It should throw EOF when a string-object is truncated, while it
doesn't.


>>> import _marshal # lib_pypy/_marshal.py
>>>
_marshal.loads('(\x02\x00\x00\x00i\x03\x00\x00\x00sB\xf9\x00\x00\n<stations
version="1.1" ')
(3, '\n<stations version="1.1" ')
>>>
>>> import marshal # Python built-in
>>>
marshal.loads('(\x02\x00\x00\x00i\x03\x00\x00\x00sB\xf9\x00\x00\n<stations
version="1.1" ')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
EOFError: EOF read where object expected



The patch, just add this one line to lib_pypy/_marshal.py (line 433)


def _read(self, n):
    pos = self.bufpos
    newpos = pos + n
   * if newpos > len(self.bufstr): raise EOFError*
    ret = self.bufstr[pos : newpos]
    self.bufpos = newpos
    return ret



This is the first bug that I found. Should I create a bug report or patch
file for this?


Thanks,
Jonathan Slenders
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120808/b35d7388/attachment.html>


More information about the pypy-dev mailing list