[New-bugs-announce] [issue40195] multiprocessing.Queue.put can fail silently due to pickle errors

[Sander Land]

New submission from Sander Land <sander.land at cognite.com>:

The multiprocessing Queue uses a thread to pickle and send the object after a call to put. When pickling fails (e.g. due to recursion depth) the exception is not returned to the caller to .put but instead dumped on the screen, leaving any multiprocessing pools in a very unhappy state.

Suggested fix: pickle the object in the same thread as the caller and send the pickled object to the thread, ensuring the caller to .put can catch the exception.

Sad workaround: (un)pickle anything sent via this queue yourself.

----------
components: Library (Lib)
files: minimal_repr.py
messages: 365816
nosy: Sander Land
priority: normal
severity: normal
status: open
title: multiprocessing.Queue.put can fail silently due to pickle errors
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file49036/minimal_repr.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40195>
_______________________________________