[Python-checkins] cpython: whatsnew: multiprocessing start methods and context (#8713 and #18999)

[r.david.murray]

http://hg.python.org/cpython/rev/b941a320601a
changeset:   89569:b941a320601a
user:        R David Murray <rdmurray at bitdance.com>
date:        Mon Mar 10 14:27:21 2014 -0400
summary:
  whatsnew: multiprocessing start methods and context (#8713 and #18999)

Also tweaked the docs a bit to use our standard style for
versionadded/changed.  (I'm guessing there are other places
in the multiprocessing docs where similar tweaks should be made.)

files:
  Doc/library/multiprocessing.rst |  35 ++++++++++++--------
  Doc/whatsnew/3.4.rst            |  24 +++++++++++---
  Misc/NEWS                       |   7 ++++
  3 files changed, 47 insertions(+), 19 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -136,9 +136,11 @@
     Available on Unix platforms which support passing file descriptors
     over Unix pipes.
 
-Before Python 3.4 *fork* was the only option available on Unix.  Also,
-prior to Python 3.4, child processes would inherit all the parents
-inheritable handles on Windows.
+.. versionchanged:: 3.4
+   *span* added on all unix platforms, and *forkserver* added for
+     some unix platforms.
+   Child processes no longer inherit all of the parents inheritable
+     handles on Windows.
 
 On Unix using the *spawn* or *forkserver* start methods will also
 start a *semaphore tracker* process which tracks the unlinked named
@@ -1853,25 +1855,30 @@
    callbacks and has a parallel map implementation.
 
    *processes* is the number of worker processes to use.  If *processes* is
-   ``None`` then the number returned by :func:`os.cpu_count` is used.  If
-   *initializer* is not ``None`` then each worker process will call
+   ``None`` then the number returned by :func:`os.cpu_count` is used.
+
+   If *initializer* is not ``None`` then each worker process will call
    ``initializer(*initargs)`` when it starts.
 
+   *maxtasksperchild* is the number of tasks a worker process can complete
+   before it will exit and be replaced with a fresh worker process, to enable
+   unused resources to be freed. The default *maxtasksperchild* is None, which
+   means worker processes will live as long as the pool.
+
+   *context* can be used to specify the context used for starting
+   the worker processes.  Usually a pool is created using the
+   function :func:`multiprocessing.Pool` or the :meth:`Pool` method
+   of a context object.  In both cases *context* is set
+   appropriately.
+
    Note that the methods of the pool object should only be called by
    the process which created the pool.
 
    .. versionadded:: 3.2
-      *maxtasksperchild* is the number of tasks a worker process can complete
-      before it will exit and be replaced with a fresh worker process, to enable
-      unused resources to be freed. The default *maxtasksperchild* is None, which
-      means worker processes will live as long as the pool.
+      *maxtasksperchild*
 
    .. versionadded:: 3.4
-      *context* can be used to specify the context used for starting
-      the worker processes.  Usually a pool is created using the
-      function :func:`multiprocessing.Pool` or the :meth:`Pool` method
-      of a context object.  In both cases *context* is set
-      appropriately.
+      *context*
 
    .. note::
 
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -1063,11 +1063,25 @@
 (``spawn`` and ``forkserver``) have been added for starting processes using
 :mod:`multiprocessing`.  These make the mixing of processes with threads more
 robust, and the ``spawn`` method matches the semantics that multiprocessing has
-always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`).
-
-Also, except when using the old *fork* start method, child processes
-will no longer inherit unneeded handles/file descriptors from their parents
-(part of :issue:`8713`).
+always used on Windows.  New function
+:func:`~multiprocessing.get_all_start_methods` reports all start methods
+available on the platform, :func:`~multiprocessing.get_start_method` reports
+the current start method, and :func:`~multiprocessing.set_start_method` sets
+the start method.  (Contributed by Richard Oudkerk in :issue:`8713`).
+
+:mod:`multiprocessing` also now has the concept of a ``context``, which
+determines how child processes are created.  New function
+:func:`~multiprocessing.get_context` returns a context that uses a specified
+start method.  It has the same API as the :mod:`multiprocessing` module itself,
+so you can use it to create :class:`~multiprocessing.pool.Pool`\ s and other
+objects that will operate within that context.  This allows a framework and an
+application or different parts of the same application to use multiprocessing
+without interfering with each other.  (Contributed by Richard Oudkerk in
+:issue:`18999`.)
+
+Except when using the old *fork* start method, child processes no longer
+inherit unneeded handles/file descriptors from their parents (part of
+:issue:`8713`).
 
 :mod:`multiprocessing` now relies on :mod:`runpy` (which implements the
 ``-m`` switch) to initialise ``__main__`` appropriately in child processes
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1636,6 +1636,9 @@
 
 - Issue #18281: Unused stat constants removed from `tarfile`.
 
+- Issue #18999: Multiprocessing now supports 'contexts' with the same API
+  as the module, but bound to specified start methods.
+
 - Issue #18468: The re.split, re.findall, and re.sub functions and the group()
   and groups() methods of match object now always return a string or a bytes
   object.
@@ -2051,6 +2054,10 @@
 - Issue #18532: Change the builtin hash algorithms' names to lower case names
   as promised by hashlib's documentation.
 
+- Issue #8713: add new spwan and forkserver start methods, and new functions
+  get_all_start_methods, get_start_method, and set_start_method, to
+  multiprocessing.
+
 - Issue #18405: Improve the entropy of crypt.mksalt().
 
 - Issue #12015: The tempfile module now uses a suffix of 8 random characters

-- 
Repository URL: http://hg.python.org/cpython