Multi-Threading and KeyboardInterrupt

[Matt]

I'm going to use the multipocessing library from here forward so I can  
take advantage of multiple cores and clusters. Either one should work  
for my use, since in my non-demonstration code each thread spends most  
of it's time waiting for a separate non-Python subprocess (created  
with subprocess.Popen) to finish anyway. (I guess Python would see  
this as IO-blocking) Therefore, if you can fix my toy example with  
threading, that's fine.

DB.py, followed by a KeyboardInterrupt yields the output in a.out. I  
want roughly the output in desired.out.

What do I need to do to modify this code to get my desired output and  
corresponding functionality? It would be a shame if this wasn't  
possible in any pure-Python way.

~Matt

On Jun 15, 2009, at 11:53 AM, Mike Kazantsev wrote:

> On Mon, 15 Jun 2009 05:37:14 -0700 (PDT)
> OdarR <Olivier.Darge at gmail.com> wrote:
>
>> On 13 juin, 07:25, Mike Kazantsev <mk.frag... at gmail.com> wrote:
>>> There was quite interesting explaination of what happens when you  
>>> send
>>> ^C with threads, posted on concurrency-sig list recently:
>>>
>>>  http://blip.tv/file/2232410
>>>  http://www.dabeaz.com/python/GIL.pdf
>>>
>>> Can be quite shocking, but my experience w/ threads only confirms  
>>> that.
>>
>> Hi there,
>> please read this package page (in 2.6), this is very interesting.
>> http://docs.python.org/library/multiprocessing.html
>>
>> I tested it : it works. Multi-core cpu's are happy :-)
>
> I'd certainly prefer using processes because they indeed work
> flawlessly in that respect, but threads are way simplier and much more
> integrated into the language, so I can avoid re-imlementing tons of
> shared stuff, IPC and locking by using threads which bassically run in
> the same context.
> That might mean 90% of code for trivial but parallel task.
>
> Alas, they don't work flawlessly in all cases, but there is still
> million and one use for them.
>
> -- 
> Mike Kazantsev // fraggod.net
> -- 
> http://mail.python.org/mailman/listinfo/python-list
I'm going to use the multipocessing library from here forward so I can  
take advantage of multiple cores and clusters. Either one should work  
for my use, since in my non-demonstration code each thread spends most  
of it's time waiting for a separate non-Python subprocess (created  
with subprocess.Popen) to finish anyway. (I guess Python would see  
this as IO-blocking) DB.py, followed by a KeyboardInterrupt yields the  
output in a.out. I want roughly the output in desired.out.

What do I need to do to modify this code to get my desired output and  
corresponding functionality? It would be a shame if this wasn't  
possible in any pure-Python way.

~Matt

On Jun 15, 2009, at 11:53 AM, Mike Kazantsev wrote:

> On Mon, 15 Jun 2009 05:37:14 -0700 (PDT)
> OdarR <Olivier.Darge at gmail.com> wrote:
>
>> On 13 juin, 07:25, Mike Kazantsev <mk.frag... at gmail.com> wrote:
>>> There was quite interesting explaination of what happens when you  
>>> send
>>> ^C with threads, posted on concurrency-sig list recently:
>>>
>>> http://blip.tv/file/2232410
>>> http://www.dabeaz.com/python/GIL.pdf
>>>
>>> Can be quite shocking, but my experience w/ threads only confirms  
>>> that.
>>
>> Hi there,
>> please read this package page (in 2.6), this is very interesting.
>> http://docs.python.org/library/multiprocessing.html
>>
>> I tested it : it works. Multi-core cpu's are happy :-)
>
> I'd certainly prefer using processes because they indeed work
> flawlessly in that respect, but threads are way simplier and much more
> integrated into the language, so I can avoid re-imlementing tons of
> shared stuff, IPC and locking by using threads which bassically run in
> the same context.
> That might mean 90% of code for trivial but parallel task.
>
> Alas, they don't work flawlessly in all cases, but there is still
> million and one use for them.
>
> -- 
> Mike Kazantsev // fraggod.net
> -- 
> http://mail.python.org/mailman/listinfo/python-list
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a.out
Type: application/octet-stream
Size: 2780 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20090615/319539ab/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: desired.out
Type: application/octet-stream
Size: 142 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20090615/319539ab/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Sub.py
Type: text/x-python-script
Size: 165 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20090615/319539ab/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DB.py
Type: text/x-python-script
Size: 604 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20090615/319539ab/attachment-0001.bin>