Threaded alternatives to smtplib?

[Alex Jurkiewicz]

Piet van Oostrum wrote:
>> AJ> if __name__ == '__main__':
>> AJ>    THREADS = []
>> AJ>    for i in range(CONCURRENCY):
>> AJ>        THREADS.append(threading.Thread(target=threadProcessRecipient))
>> AJ>    for thread in THREADS:
>> AJ>        thread.run()
>>     
>
> You should use thread.start(), not thread.run(). When you use run(), it
> will be sequential execution, as you experience. With start() you get
> concurrency
>   

Thanks! Changing this method call fixes the original problem we saw. In 
the process I've run into another though, seemingly a race condition / 
deadlock.

Our email script DKIM signs all messages with pydkim. Because this is so 
CPU intensive we run the signing in an external process, using 
dkimsign.py (which basically accepts the message on stdin, calls 
dkim.sign() on it and passes the result back on stdout). We call the 
DKIM signing while preparing each message in each thread like this:
cmd = subprocess.Popen(
    ['/usr/bin/nice', PYTHONBIN, 'dkimsign.py',
    DKIMSELECTOR,
    DKIMDOMAIN,
    DKIMPRIVATEKEYFILE],
    stdin = subprocess.PIPE,
    stdout = subprocess.PIPE)
message = cmd.communicate(message)[0]

The majority of the time (80%) our mail merge works fine -- messages are 
sent out in a multi-threaded manner. The rest of the time, some of the 
threads deadlock when calling dkimsign.py. Specifically, they stop 
during the communicate() call above. dkimsign.py never receives any 
input (hangs on message=sys.stdin.read()), and sits there waiting 
forever, which stops the thread from doing anything.

Is this a bug with subprocess? Is there some way to set a timeout on the 
communicate() call so I can detect these locks?

Cheers,
AJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090505/f0bfa484/attachment-0001.html>