FYI: I shot an email to stdlib-sig about the fact I am proposing the
inclusion of the pyProcessing module into the stdlib. Comments and
thoughts regarding that would be welcome. I've got a rough outline of
the PEP, but I need to spend more time with the code examples.
-jesse
On Wed, Mar 19, 2008 at 9:52 PM, Alex Martelli
Hmmm, sorry if I'm missing something obvious, but, if the occasional background computations are sufficiently heavy -- why not fork, do said computations in the child thread, and return the results via any of the various available IPC approaches? I've recently (at Pycon, mostly) been playing devil's advocate (i.e., being PRO-threads, for once) on the subject of utilizing multiple cores effectively -- but the classic approach (using multiple _processes_ instead) actually works quite well in many cases, and this application server would appear to be one. (the pyProcessing package appears to offer an easy way to migrate threaded code to multiple-processes approaches, although I've only played around with it, not [yet] used it for production code).
Alex
On Wed, Mar 19, 2008 at 10:49 AM, Adam Olsen
wrote: On Wed, Mar 19, 2008 at 11:25 AM, Stefan Ring
wrote: Adam Olsen
writes: So you want responsiveness when idle but throughput when busy?
Exactly ;)
Are those calculations primarily python code, or does a C library do the grunt work? If it's a C library you shouldn't be affected by safethread's increased overhead.
It's Python code all the way. Frankly, it's a huge mess, but it would be very very hard to come up with a scalable solution that would allow to optimize certain hotspots and redo them in C or C++. There isn't even anything left to optimize in particular because all those low hanging fruit have already been taken care of. So it's just ~30kloc Python code over which the total time spent is quite uniformly distributed :(.
I see. Well, at this point I think the most you can do is file a bug so the problem doesn't get forgotten. If nothing else, if my safethread stuff goes in it'll very likely include a --with-gil option, so I may put together a FIFO scheduler.
-- Adam Olsen, aka Rhamphoryncus
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com