[Python-checkins] cpython: Issue #21116: Avoid blowing memory when allocating a multiprocessing shared
antoine.pitrou
python-checkins at python.org
Mon Apr 13 20:54:24 CEST 2015
https://hg.python.org/cpython/rev/0f944e424d67
changeset: 95595:0f944e424d67
user: Antoine Pitrou <solipsis at pitrou.net>
date: Mon Apr 13 20:53:43 2015 +0200
summary:
Issue #21116: Avoid blowing memory when allocating a multiprocessing shared
array that's larger than 50% of the available RAM.
Patch by Médéric Boquien.
files:
Lib/multiprocessing/heap.py | 9 ++++++++-
Misc/ACKS | 1 +
Misc/NEWS | 3 +++
3 files changed, 12 insertions(+), 1 deletions(-)
diff --git a/Lib/multiprocessing/heap.py b/Lib/multiprocessing/heap.py
--- a/Lib/multiprocessing/heap.py
+++ b/Lib/multiprocessing/heap.py
@@ -71,7 +71,14 @@
os.unlink(name)
util.Finalize(self, os.close, (self.fd,))
with open(self.fd, 'wb', closefd=False) as f:
- f.write(b'\0'*size)
+ bs = 1024 * 1024
+ if size >= bs:
+ zeros = b'\0' * bs
+ for _ in range(size // bs):
+ f.write(zeros)
+ del zeros
+ f.write(b'\0' * (size % bs))
+ assert f.tell() == size
self.buffer = mmap.mmap(self.fd, self.size)
def reduce_arena(a):
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -154,6 +154,7 @@
Gawain Bolton
Forest Bond
Gregory Bond
+Médéric Boquien
Matias Bordese
Jonas Borgström
Jurjen Bos
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -29,6 +29,9 @@
Library
-------
+- Issue #21116: Avoid blowing memory when allocating a multiprocessing shared
+ array that's larger than 50% of the available RAM. Patch by Médéric Boquien.
+
- Issue #22982: Improve BOM handling when seeking to multiple positions of
a writable text file.
--
Repository URL: https://hg.python.org/cpython
More information about the Python-checkins
mailing list