[issue16104] Compileall script: add option to use multiple cores

[Jim Jewett]

Jim Jewett added the comment:

Trying to put bounds on the disagreements.  Does anyone disagree with any of the following:

(1)  compileall currently runs single-threaded in a single process.


(2)  This enhancement intends to allow parallelization by process.


(3)  Users MAY need to express whether they (require/forbid/are expressly apathetic concerning) paralellization.

(3A)  There is some doubt that this even needs to be user-controlled.

(3B)  If it is user-controlled, the patch proposes adding a "processes" parameter to do this.

(3C)  There have been suggestions of other names (notably "workers"), but *if* it is user-controlled, the idea of a new parameter is not controversial.


(4)  Users MAY need to control the degree of parallelization.

(4A)  If so, setting the value of the new parameter to a positive integer > 1 is an acceptable solution.

(4B)  There is not yet consensus on how to represent "Use multi-processing, with the default degree for this system.", "Do NOT use multiprocessing.", or "I don't care."

(4C)  Suggested values have included 1, 0, -1, any negative number, None, and specific strings.  The precise mapping between some of these and the three cases of 4B is not agreed.


(5)  If multiprocessing is explicitly requested, what should happen when it is not available?

(5A)  Fall back to the current way, without multi-processing.

(5B)  Fall back to the current way, without multi-processing, but issue a Warning.

(5C)  Raise an Exception.  (ValueError, ImportError, NotImplemented?)


(6)  Portions of the documentation unrelated to this should be fixed.  But ideally, that would be done separately, and it will NOT be a pre-requisite to this patch.


---------------------------------------------------

Another potential value set

None (the default) ==> let the system parallelize as best it can -- as it does in multiprocessing.  If the system picks "not in parallel at all", that is also OK, and no warning is raised.

0 ==> Do not parallelize.

positive integers ==> Use that many processes.

negative ==> ValueError

Would these uses of 0 and negative be too surprising for someone?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16104>
_______________________________________