This means that if a ThreadExecutor is called from both daemon and non-daemon threads (which is a very bad idea though), the kind of spawned
Remy NOEL wrote: threads might be unpredictable. IIUC, I don't think there's a way that we could reasonably support stability for provide any guarantees for spawning or interacting with the Executors from a daemon thread. It might even be worth considering to explicitly warn or prevent that in some capacity. Antoine Pitrou wrote:
Daemon threads are already a bad idea, actually. It is really worth the effort to ensure your threads exit cleanly rather than expect the interpreter to kill them at shutdown.
Agreed. Furthermore, if there's a thread in a program that can potentially
hang indefinitely, converting them into daemon threads (allowing it to be
killed on interpreter shutdown) just obscures the problem in most cases,
IMO. There should be some consideration placed into preventing a thread
from hanging indefinitely in the first place, such as with timeouts or some
form of outside signaling (such as threading.Event).
That's not to say that daemon threads aren't sometimes useful. Essentially,
if you're okay with the thread being abruptly killed without allowing it's
resources to be cleaned up gracefully, making it a daemon thread shouldn't
be a significant issue. But, the core problem here is that
ThreadPoolExecutor is an example of a resource that should be cleaned up
gracefully, so spawning it in a daemon thread or trying to interact with it
through one can interfere with that process.
On Tue, Mar 10, 2020 at 3:40 PM Antoine Pitrou
On Tue, 10 Mar 2020 15:16:06 +0100 Remy NOEL
wrote: On Mon, Mar 9, 2020 at 11:18 PM Kyle Stanley
wrote: We're currently looking into adjusting the current Executor implementation (for both TPE and PPE) to not use daemon threads at all for subinterpreter compatibility, see https://bugs.python.org/issue39812. Instead of using an atexit handler, there's some consideration for using a similar threading-specific "exit handler". Instead of being upon program exit, registered functions would be called just before all of the non-daemon threads are joined in `threading._shutdown()`.
As a result, I suspect that we are unlikely to implement something like the above. But, as a convenient side effect, it should somewhat address the current potentially surprising behavior.
There might still be some fun involved given that current implementation *might* lazy-spawn threads upon submit. This means that if a ThreadExecutor is called from both daemon and non-daemon threads (which is a very bad idea though), the kind of spawned threads might be unpredictable.
Daemon threads are already a bad idea, actually. It is really worth the effort to ensure your threads exit cleanly rather than expect the interpreter to kill them at shutdown.
Regards
Antoine.
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CUY3K4... Code of Conduct: http://python.org/psf/codeofconduct/