Hello there. I'd like to draw your attention to two feature requests / patches that I've subbmitted: http://bugs.python.org/issue9609 http://bugs.python.org/issue9622 These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python. The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered. The second patch allows to set the trace/profile function in python globally, so that all threads are affected. This is essential if you want to take a profililng snapshot of a running application. We now use this extensively, to monitor the live behaviour of our EVE game servers. A HTTP backend server is used to control the profiler (profile.enable(), profile disable() at runtime) and explore its output. I haven't seen any feadback on these submissions and would appreciate some. Cheers, Kristján
2010/8/17 Kristján Valur Jónsson <kristjan@ccpgames.com>:
Hello there.
I‘d like to draw your attention to two feature requests / patches that I‘ve subbmitted:
http://bugs.python.org/issue9609 http://bugs.python.org/issue9622
These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python.
Both look like good ideas to me (multi-threaded profiling and debugging is fairly painful and it would be good to be able to do something to improve that situation).
The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered.
I'm curious as to the memory impact this has on the profiler (it obviously can't be too bad if you're able to run it against your live servers).
The second patch allows to set the trace/profile function in python globally, so that all threads are affected. This is essential if you want to take a profililng snapshot of a running application.
One minor quibble here is that I would suggest using "global=False" in your docstring signatures. Both patches seem to be missing updates to the relevant documentation. I expect this would be difficult to unit test properly, but at least some basic tests to check that the new global configuration of settrace and setprofile don't blow, and that a profiler can be shared between two threads would be good. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-----Original Message----- From: Nick Coghlan [mailto:ncoghlan@gmail.com] Sent: 17. ágúst 2010 10:04
Both look like good ideas to me (multi-threaded profiling and debugging is fairly painful and it would be good to be able to do something to improve that situation).
Indeed. I expect that profiling web server frameworks would be a good candidate, since a "profile page" can be easily set up.
The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered.
I'm curious as to the memory impact this has on the profiler (it obviously can't be too bad if you're able to run it against your live servers). The change is minimal. We have to have an extra rotatingtree to match tls to stack anchor points. There are also a few extra members in the profiling "Context" entries, but these are ephemeral.
One minor quibble here is that I would suggest using "global=False" in your docstring signatures.
Okay. The docs also need to be in line with the docstrings (e.g. the docs say "setprofile(profilefunc)" while the docstring says "setprofile(function)"
Both patches seem to be missing updates to the relevant documentation.
Yes, this is intentional. I didn't want to waste effort on writing documentation before having exposed this to you. Sometimes my good ideas turn out to be not so good and end up being rejected.
I expect this would be difficult to unit test properly, but at least some basic tests to check that the new global configuration of settrace and setprofile don't blow, and that a profiler can be shared between two threads would be good. This is my intention too, but again, I wanted to give this some airing first. What't I probably end up doing is setting up a few threads, have them do some token work, and then do analysis on the Profile.stats member to make sure that all of them were accounted for and all and only callgraph relations show up.
Kristján
On Tue, 17 Aug 2010 09:22:15 +0000 Kristján Valur Jónsson <kristjan@ccpgames.com> wrote:
Hello there. I'd like to draw your attention to two feature requests / patches that I've subbmitted: http://bugs.python.org/issue9609 http://bugs.python.org/issue9622
These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python. The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered.
Does that mean you're proposing code for inclusion in CPython that can only be tested with Stackless? Can't Stackless use its own patches instead?
The second patch allows to set the trace/profile function in python globally, so that all threads are affected. This is essential if you want to take a profililng snapshot of a running application.
I've often heard that cProfile didn't give correct profiling information with multiple threads. Is it true? Thanks Antoine.
Ok, I've looked at the patch and it's actually stackless-agnostic. Regards Antoine. On Tue, 17 Aug 2010 12:31:30 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 17 Aug 2010 09:22:15 +0000 Kristján Valur Jónsson <kristjan@ccpgames.com> wrote:
Hello there. I'd like to draw your attention to two feature requests / patches that I've subbmitted: http://bugs.python.org/issue9609 http://bugs.python.org/issue9622
These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python. The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered.
Does that mean you're proposing code for inclusion in CPython that can only be tested with Stackless? Can't Stackless use its own patches instead?
The second patch allows to set the trace/profile function in python globally, so that all threads are affected. This is essential if you want to take a profililng snapshot of a running application.
I've often heard that cProfile didn't give correct profiling information with multiple threads. Is it true?
Thanks
Antoine.
-----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com@python.org [mailto:python-dev-bounces+kristjan=ccpgames.com@python.org] On Behalf Of Antoine Pitrou
Does that mean you're proposing code for inclusion in CPython that can only be tested with Stackless? Can't Stackless use its own patches instead?
While this work originated with Stackless, I have gone through the trouble of porting it to CPython, because it is relevant for CPython too. Multiple "stacks" translates to multiple "threads" in C Python. (Confusingly though, the _lsprof.c patch actually has a #ifdef STACKLESS that I forgot to remove. Sorry about that)
I've often heard that cProfile didn't give correct profiling information with multiple threads. Is it true?
Yes, and that's the purpose of this submission: To fix exactly that. K
2010/8/17 Kristján Valur Jónsson <kristjan@ccpgames.com>:
Yes, this is intentional. I didn't want to waste effort on writing documentation before having exposed this to you. Sometimes my good ideas turn out to be not so good and end up being rejected.
Cool, I thought it would be something like that. In this case, at least from where I'm standing, your good idea looks like a good one. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
2010/8/17 Kristján Valur Jónsson <kristjan@ccpgames.com>
These patches are the result of work that we have done in profiling Stackless Python server applications at runtime, but they apply just as well to C Python.
The first patch makes _lsprof, the engine behind cProfile, multi-stack aware. This allows the same cProfiler.Profile() instance to be active on multiple python threads and still meaningful information is gathered.
+1
The second patch allows to set the trace/profile function in python * globally*, so that all threads are affected. This is essential if you want to take a profililng snapshot of a running application.
+1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
participants (4)
-
Antoine Pitrou -
Daniel Stutzbach -
Kristján Valur Jónsson -
Nick Coghlan