threading and multiprocessing deadlock
Johannes Bauer
dfnsonfsduifb at gmx.de
Mon Dec 6 10:05:17 EST 2021
Am 06.12.21 um 13:56 schrieb Martin Di Paola:
> Hi!, in short your code should work.
>
> I think that the join-joined problem is just an interpretation problem.
>
> In pseudo code the background_thread function does:
>
> def background_thread()
> # bla
> print("join?")
> # bla
> print("joined")
>
> When running this function in parallel using threads, you will probably
> get a few "join?" first before receiving any "joined?". That is because
> the functions are running in parallel.
>
> The order "join?" then "joined" is preserved within a thread but not
> preserved globally.
Yes, completely understood and really not the issue. That these pairs
are not in sequence is fine.
> Now, I see another issue in the output (and perhaps you was asking about
> this one):
>
> join?
> join?
> myfnc
> myfnc
> join?
> join?
> joined.
> joined.
>
> So you have 4 "join?" that correspond to the 4 background_thread
> function calls in threads but only 2 "myfnc" and 2 "joined".
Exactly that is the issue. Then it hangs. Deadlocked.
> Could be possible that the output is truncated by accident?
No. This is it. The exact output varies, but when it hangs, it always
also does not execute the function (note the lack of "myfnc"). For example:
join?
join?
myfnc
join?
myfnc
join?
myfnc
joined.
joined.
joined.
(only three threads get started there)
join?
myfnc
join?
join?
join?
joined.
(this time only a single one made it)
join?
join?
join?
myfnc
join?
myfnc
joined.
myfnc
joined.
joined.
(three get started)
> I ran the same program and I got a reasonable output (4 "join?", "myfnc"
> and "joined"):
>
> join?
> join?
> myfnc
> join?
> myfnc
> join?
> joined.
> myfnc
> joined.
> joined.
> myfnc
> joined.
This happens to me occasionally, but most of the time one of the
processes deadlocks. Did you consistently get four of each? What
OS/Python version were you using?
> Another issue that I see is that you are not joining the threads that
> you spawned (background_thread functions).
True, I kindof assumed those would be detached threads.
> I hope that this can guide you to fix or at least narrow the issue.
Depending on what OS/Python version you're using, that points in that
direction and kindof reinforces my belief that the code is correct.
Very curious.
Thanks & all the best,
Joe
More information about the Python-list
mailing list