ctypes, memory mapped files and context manager
Peter Otten
__peter__ at web.de
Wed Dec 28 15:58:38 EST 2016
Hans-Peter Jansen wrote:
> On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
>> Hans-Peter Jansen wrote:
>> > Dear Peter,
>> >
>> > thanks for taking valuable time to look into my issue.
>>
>> You're welcome!
>>
>> > It might be related to my distinct silliness, but the problem persists
>> > with your code as well.
>>
>> Unfortunately I posted the broken toy example rather than the fixed one.
>> Here's the latter. Basically you have to keep a reference in the context
>> manager (whether you implement it as a class or a generator doesn't
>> matter) without giving another reference away to client code:
>>
>> $ cat mmap_after.py
>> import ctypes
>> import mmap
>> import weakref
>>
>> from contextlib import contextmanager
>>
>> class T(ctypes.Structure):
>> _fields = [("foo", ctypes.c_uint32)]
>>
>>
>> @contextmanager
>> def map_struct(m, n):
>> m.resize(n * mmap.PAGESIZE)
>> keep_me = T.from_buffer(m)
>> yield weakref.proxy(keep_me)
>
> Hooray, that did the trick. Great solution, thank you very much!
>
> If you don't mind, I will mention you and your solution at the various
> places, I placed this issue over the last weeks.
>
> You made my day, Peter!
>
> It leaves the question on why is Python2 acting as one would expect
> related to context managers, and Python3 needs this weakref juggling.
> Maybe something, that's worth to be placed in python-dev. What do you
> think?
Well, given
$ cat mmap_resize.py
import ctypes
import mmap
class T(ctypes.Structure):
_fields_ = [("foo", ctypes.c_uint32)]
SIZE = 2 * mmap.PAGESIZE
f = open("tmp.dat", "w+b")
f.write(b"\0" * SIZE)
f.seek(0)
m = mmap.mmap(f.fileno(), SIZE)
a = T.from_buffer(m, mmap.PAGESIZE)
m.resize(mmap.PAGESIZE)
a.foo = 42
$ python3 mmap_resize.py
Traceback (most recent call last):
File "mmap_resize.py", line 15, in <module>
m.resize(mmap.PAGESIZE)
BufferError: mmap can't resize with extant buffers exported.
$ python2.7 mmap_resize.py
Segmentation fault
do you really prefer the Python 2 behaviour?
More information about the Python-list
mailing list