[New-bugs-announce] [issue45125] Improve tests and docs of how `pickle` works with `SharedMemory` obejcts
report at bugs.python.org
Tue Sep 7 06:17:52 EDT 2021
New submission from Nikita Sobolev <mail at sobolevn.me>:
As we've discussed in https://bugs.python.org/msg401146 we need to improve how `SharedMemory` is documented and tested. Right now docs do not cover `pickle` at all (https://github.com/python/cpython/blob/main/Doc/library/multiprocessing.shared_memory.rst). And this is an important aspect in multiprocessing.
`SharedMemory` and `pickle` in `test_shared_memory_basics` (https://github.com/python/cpython/blob/a5c6bcf24479934fe9c5b859dd1cf72685a0003a/Lib/test/_test_multiprocessing.py#L3789-L3794):
sms.buf[0:6] = b'pickle'
pickled_sms = pickle.dumps(sms)
sms2 = pickle.loads(pickled_sms)
self.assertEqual(bytes(sms.buf[0:6]), bytes(sms2.buf[0:6]), b'pickle')
`ShareableList` has better coverage in this regard (https://github.com/python/cpython/blob/a5c6bcf24479934fe9c5b859dd1cf72685a0003a/Lib/test/_test_multiprocessing.py#L4121-L4144):
sl = shared_memory.ShareableList(range(10))
serialized_sl = pickle.dumps(sl)
deserialized_sl = pickle.loads(serialized_sl)
self.assertFalse(sl is deserialized_sl)
deserialized_sl = "changed"
# Verify data is not being put into the pickled representation.
name = 'a' * len(sl.shm.name)
larger_sl = shared_memory.ShareableList(range(400))
serialized_larger_sl = pickle.dumps(larger_sl)
self.assertTrue(len(serialized_sl) == len(serialized_larger_sl))
So, my plan is:
1. Improve testing of `SharedMemory` after pickling/unpickling. I will create a separate test with all the `pickle`-related stuff there
2. Improve docs: user must understand what will have when `SharedMemory` / `SharableList` is pickled and unpickled. For example, the fact that it will still be shared can be a surprise to many.
I am going to send a PR with both thing somewhere this week.
I will glad to head any feedback before / after :)
assignee: docs at python
components: Documentation, Tests
nosy: docs at python, sobolevn
title: Improve tests and docs of how `pickle` works with `SharedMemory` obejcts
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce