Multi-threading in Python vs Java
Cameron Simpson
cs at zip.com.au
Fri Oct 11 17:35:26 EDT 2013
On 11Oct2013 15:53, Terry Reedy <tjreedy at udel.edu> wrote:
> On 10/11/2013 4:41 AM, Peter Cacioppi wrote:
> >I should add that the computational heavy lifting is done in a third party library. So a worker thread looks roughly like this (there is a subtle race condition I'm glossing over).
> >
> >while len(jobs) :
> > job = jobs.pop()
> > model = Model(job) # Model is py interface for a lib written in C
> > newJobs = model.solve() # This will take a long time
> > for each newJob in newJobs :
> > jobs.add(newJob)
> >
> >Here jobs is a thread safe object that is shared across each worker thread. It holds a priority queue of jobs that can be solved in parallel.
> >
> >Model is a py class that provides the API to a 3rd party library written in C.I know model.solve() will be the bottleneck operation for all but trivial problems.
[...]
> Given that model.solve takes a 'long time' (seconds, at least), the
> extra time to start a process over the time to start a thread will
> be inconsequential. I would therefore look at the multiprocessing
> module.
And, for contrast, I would not. Threads are my friends and Python
threads seem eminently suited to the above scenario.
Cheers,
--
Cameron Simpson <cs at zip.com.au>
[Alain] had been looking at his dashboard, and had not seen me, so I
ran into him. - Jean Alesi on his qualifying prang at Imola '93
More information about the Python-list
mailing list