Catching exceptions with multi-processing
Fabien
fabien.maussion at gmail.com
Sat Jun 20 05:03:51 EDT 2015
On 06/20/2015 05:14 AM, Cameron Simpson wrote:
> I would keep your core logic Pythonic, raise exceptions. But I would
> wrap each task in something to catch any Exception subclass and report
> back to the queue. Untested example:
>
> def subwrapper(q, callable, *args, **kwargs):
> try:
> q.put( ('COMPLETED', callable(*args, **kwargs)) )
> except Exception as e:
> q.put( ('FAILED', e, callable, args, kwargs) )
>
> then dispatch tasks like this:
>
> pool.map(subwrapper, q, task1, dirs, chunksize=1)
>
> and have a thread (or main program) collect things from the queue for
> logging and other handling. Obviously you might return something more
> sophisticated that my simple tuple above, but I'm sure you get the idea.
>
> Cheers,
> Cameron Simpson
Perfect! Much more elegant and easier to implement on top of my existing
workflow based on raising exceptions.
thanks to all responses,
Fabien
More information about the Python-list
mailing list