[issue15097] Improving wording on the thread-safeness of import
New submission from Merlijn van Deen <valhallasw@gmail.com>: http://docs.python.org/library/threading.html#importing-in-threaded-code Currently, the documentation states "Firstly, other than in the main module, an import should not have the side effect of spawning a new thread and then waiting for that thread in any way. Failing to abide by this restriction can lead to a deadlock if the spawned thread directly or indirectly attempts to import a module." which, I think, fails to make the main point: a call to import acquires the import lock. A call to import from a second thread will thus block. As such, I would suggest rephrasing it to something like: "Firstly, an import acquires the import lock for that thread. Therefore, the import should not have the side effect of waiting for a different thread in any way, as this can lead to a deadlock if that thread directly or indirectly attempts to import a module." There are two additional points that might be interesting to note: (1) Any module can be imported. If the import causes a deadlock, that is a bad thing. Every module *will* be imported by tools such as nosetests. (1b) so: never, ever, have code that causes locks in a different thread in module level code witout 'if __name__=="__main__" ' blocks? (2) The lock is also acquired if a module has already been imported. For instance, in import sys # (1) def f(): import sys # (2) the import lock is acquired in (1) /and/ (2). Adding example code and/or a flow diagram might be a bit too much, but it does clarify how easy it is to make this mistake. See the attached for an example (both a simple example script, as well as a flow diagram explaining what happens). ---------- assignee: docs@python components: Documentation files: deadlock.py messages: 163068 nosy: docs@python, valhallasw priority: normal severity: normal status: open title: Improving wording on the thread-safeness of import type: enhancement Added file: http://bugs.python.org/file26037/deadlock.py _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15097> _______________________________________
Changes by Merlijn van Deen <valhallasw@gmail.com>: Removed file: http://bugs.python.org/file26037/deadlock.py _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15097> _______________________________________
Changes by Merlijn van Deen <valhallasw@gmail.com>: Added file: http://bugs.python.org/file26038/deadlock.py _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15097> _______________________________________
Martin v. Löwis <martin@v.loewis.de> added the comment:
which, I think, fails to make the main point:
I disagree. It currently makes it main point, but stops doing so under your rephrasing. The main point of that section is "While the import machinery is thread-safe, there are two key restrictions on threaded imports due to inherent limitations in the way that thread-safety is provided:" The existence of an import lock is deliberately omitted from the text, and the reader is supposed to abide by the restriction as written regardless of the motivation behind it.
Adding example code and/or a flow diagram might be a bit too much, but it does clarify how easy it is to make this mistake. See the attached for an example (both a simple example script, as well as a flow diagram explaining what happens).
The entire notion of an import lock is obsolete. Python 3.3 does not have that anymore. ---------- nosy: +loewis _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15097> _______________________________________
Merlijn van Deen <valhallasw@gmail.com> added the comment: First off, thank you for your response.
The existence of an import lock is deliberately omitted from the text, and the reader is supposed to abide by the restriction as written regardless of the motivation behind it.
The entire notion of an import lock is obsolete. Python 3.3 does not have that anymore.
" This warning is still valid but for a different reason " or " this warning is no longer valid in 3.3 "? Assuming the first (which is what I guess based on the fact the deadlock still occurs in 3.3), I think the text can still be improved; the current wording suggests to me a) it's OK to wait for a thread as long as you did not create it and b) it's OK to import something that waits for a thread as long as you do it from the main module - while both cases can still lead to a deadlock. so, leaving the implementation details out, this is my suggestion: "Firstly, an import should not have the side effect of waiting for a thread in any way. This can lead to a deadlock if that thread directly or indirectly attempts to import a module." ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue15097> _______________________________________
Irit Katriel <iritkatriel@yahoo.com> added the comment: The threading doc no longer mentions import at all. Any objections to closing this issue as out of date? Or is there anything else to look into here? ---------- nosy: +iritkatriel resolution: -> out of date status: open -> pending _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue15097> _______________________________________
Change by Irit Katriel <iritkatriel@yahoo.com>: ---------- stage: -> resolved status: pending -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue15097> _______________________________________
participants (3)
-
Irit Katriel
-
Martin v. Löwis
-
Merlijn van Deen