thread/queue bug

[Steve Holden]

Antoon Pardon wrote:

> Op 2004-12-13, Tim Peters schreef <tim.peters at gmail.com>:
> 
>>[Antoon Pardon]
>>
>>>I don't see why starting a thread as a side effect of importing is
>>>bad thread practice. Sure python doesn't cater for it, but IMO
>>>that seems to be python failing.
>>
>>Obviously, it's bad practice in Python because it can lead to
>>deadlocks in Python.
> 
> 
> By that argument any use of locks is bad practice because it
> can lead to deadlock.
> 
Not at all. You mentioned locks, not Tim. The reason it can lead to 
deadlocks is because import, being an executable statement, must 
terminate correctly in order to return control to the module that 
executed the import statement.
> 
>>It's nearly tautological.  Import is an
>>executable statement in Python, not, e.g., as in many other languages,
>>a declaration directed at the system linker.  With that power comes
>>opportunities to shoot yourself, although they're generally easy to
>>avoid.  Come up with a practical design that doesn't have this
>>limitation, and then perhaps your characterization of the current
>>design as "a failing" would be both credible and constructive.
> 
> 
> If a car model has cranky brakes, I think I can call that a failing
> even without having the ability to come up with a pratical design
> that doesn's has those limitations.
> 
But in fact your situation is more closely analogous to a customer who's 
bought a car that can be stopped by pressing on the brake pedal now 
complaining that sideways pressure on the brake pedal doesn;t affect the 
car's motion.

> I judge a language by what it can and cannot do, not by my ability
> to correct the things I perceive as failings. For all I know python
> may have taken some design decisions that might have seen perfectly
> logical but now prohibit a a practical design that doesn't have this
> limitation. I don't see why something like that would make this
> any less a failing then when a practical design was easy in the
> current implemenation.
> 
All that Tim was suggesting is that it's MORE SENSIBLE to start a thread 
as the result of a specific call to programmed functionality rather than 
as the side effect of an import. The reason for this is due to the 
semantics of the import statement. If you perceive that as a failing 
then you'd be better rewarded by an attempt to modify your perceptions.

I've always found "don't argue with Tim about Python" to be a useful 
rule of thumb. He's wrong much less often than I am. I suspect he's also 
wrong much less often than you ;-)
> 
>>Apart from that, ya, I do think it would *uisually* be poor practice
>>to start a thread as a side effect of importing anyway.  It's too
>>mysterious, and IME has caused trouble even when it didn't lead to
>>deadlocks.  The fundamental purpose of import in Python is to add
>>useful names to the importer's namespace, and users of a module
>>generally think of it as doing no more than that.
> 
> 
> Importing a module in general also does some kind of initialisation.
> If starting a thread is a logical thing to do in this initialisation
> fase I see nothing wrong with it.
> 
But the reason, I suspect, that it's being suggested this is bad design 
is because it would be tricky (in any language) to ensure that the 
satisfactory conclusion of the import didn't depend on successful thread 
startup (and possibly termination).

Plus, of course, in Python the side-effect of the import (creating the 
thread) can happen precisely once, because the module body is executed 
precisely once no matter how many times it's imported. That's not really 
the issue, however.

> 
>>Note that the OP's example had a module that, upon the first attempt
>>to import it, ran an infinite loop (even if it hadn't deadlocked), and
>>it's clearly severe abuse of import's purpose.to write a module M such
>>that "import M" *never* returns.  Indeed, that's the other half of how
>>deadlock occurs:  not only that the imported module spawn a thread as
>>a side effect of importing, but also that the imported module refuse
>>to allow the import to complete.
> 
> 
> Well I'll agree here. An import that has as a side effect that the
> import doesn't return is bad practice.
> 
But that's precisely the risk you run when starting up threads!
> 
>>The current design actually supports spawning all the threads you like
>>as a side effect of importing, provided you ensure also that the
>>import ompletes.
> 
> 
> Well in that case I don't have any problems with it. The perceived
> failing was because of only knowing part of it based on what I had
> read in this thread.
> 
> 
>>The easiest way to avoid trouble remains not to
>>spawn threads as a side effect of importing to begin with, although a
>>programmer determined to demonstrate their bad taste <wink> can easily
>>enough make it work.
> 
> 
> Well then probably I have a bad taste. I have though of designs in which
> it seemed very natural to have a thread started as part of the
> initialisation in a module. Other limitations of python didn't make
> it workable but in priciple I saw nothing wrong with doing it.
> 
Bearing in mind that module initialisation is always one-off, relying on 
import to trigger such complex behavior is probably a design that will 
mislead users into false expectations.

regards
  Steve
-- 
Steve Holden               http://www.holdenweb.com/
Python Web Programming  http://pydish.holdenweb.com/
Holden Web LLC      +1 703 861 4237  +1 800 494 3119