As suggested by Guido I am posting a potential solution to the following problem for review.
A quick fix for this problem is to prevent resource tracker from handling destruction/unlinking of shared memory. A PR https://github.com/python/cpython/pull/15989 for the same was opened by Davin (@applio), and at this moment it is the only implemented solution to fix some of the critical issues in shared memory.
But, as soon as resource tracker is removed from the picture cleanup of shared memory is dependent on the user itself and this also makes Linux’s shared memory behaviour inconsistent with windows.
Windows uses a reference counting mechanism to count the number of processes using a shared memory segment. When all of them are done using it, Windows simply unlinks and frees the memory allocated to the shared memory segment. On the other hand, in Linux, even after all the processes are done using the shared memory segment, the segment will continue to exist unless it is cleaned by the respective processes.
Having said that, the current implementation of Linux’s shared memory is also not consistent with windows. So, I think it makes sense to go ahead with the removal of resource tracker for now until we reach a better solution.
On that note, I have mentioned some of the possible solutions here https://bugs.python.org/msg351445. Out of these solutions I think 5 is the best. This issue and solution 5 is currently being discussed here https://bugs.python.org/issue38119.