Request review of cProfile/profile series issue
Hi all, I'm now looking for cProfile/profile lib's issue, and have solve a series of dependent problem, here is the list: #9285 - Add a profile decorator to profile and cProfile #30113 - Allow helper functions to wrap sys.setprofile #18971 - Use argparse in the profile/cProfile modules #30118 - Add unittest for cProfile/profile command line interface It can divide into two categories, first is the context manager problem, and the second is optparse to argparse problem. 1. context manager problem: Relative issue: #9285, #30113 Relative PR: #287, #1212, #1253 This is an issue since 2010, and stop at profile can't simply add a __enter__ and __exit__ to make it a context manager. The main problem is, sys.setprofile() will hit the return and get bad return in profile dispatch_return function. The solution is to insert a simulate call in the helper function, to provide the context between helper frame and where the profile is defined. 2. optparse to argparse problem: Relative issue: #18971, #30118 Relative PR: #1232 Relative patch: profile_argparse.patch Serhiy have provide the patch of replace optparse to argparse in the profile and cProfile, but block by Ezio request of unittest for command line interface. The unittest is provide at #1232, and need to be reivew. If the unittest is add and argparse patch is apply, we can then solve more problem, e.g.: #23420, #29238, #21862 This is what I've investigated for cProfile / profile library now, to be move on, it will need others to review the work. Thanks! Best Regards, Louie.
On 23 April 2017 at 13:21, Louie Lu <me@louie.lu> wrote:
Hi all,
I'm now looking for cProfile/profile lib's issue, and have solve a series of dependent problem, here is the list:
#9285 - Add a profile decorator to profile and cProfile #30113 - Allow helper functions to wrap sys.setprofile #18971 - Use argparse in the profile/cProfile modules #30118 - Add unittest for cProfile/profile command line interface
It can divide into two categories, first is the context manager problem, and the second is optparse to argparse problem.
1. context manager problem:
Relative issue: #9285, #30113 Relative PR: #287, #1212, #1253
This is an issue since 2010, and stop at profile can't simply add a __enter__ and __exit__ to make it a context manager. The main problem is, sys.setprofile() will hit the return and get bad return in profile dispatch_return function. The solution is to insert a simulate call in the helper function, to provide the context between helper frame and where the profile is defined.
I'd like to second Louie's call for additional review here. I've been reviewing them so far, and they look sensible to me, but I don't personally know the profile code base at all, so there's a strong chance I'll miss subtle details. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Apr 23, 2017 at 11:12 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 23 April 2017 at 13:21, Louie Lu <me@louie.lu> wrote:
Hi all,
I'm now looking for cProfile/profile lib's issue, and have solve a series of dependent problem, here is the list:
#9285 - Add a profile decorator to profile and cProfile
%prun -s Src: https://github.com/ipython/ipython/blob/master/IPython/core/magics/execution... Docs: https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-prun http://pynash.org/2013/03/06/timing-and-profiling/ %time, %timeit %prun, %lprun %mprun, %memit
#30113 - Allow helper functions to wrap sys.setprofile #18971 - Use argparse in the profile/cProfile modules #30118 - Add unittest for cProfile/profile command line interface
It can divide into two categories, first is the context manager problem, and the second is optparse to argparse problem.
1. context manager problem:
Relative issue: #9285, #30113 Relative PR: #287, #1212, #1253
This is an issue since 2010, and stop at profile can't simply add a __enter__ and __exit__ to make it a context manager. The main problem is, sys.setprofile() will hit the return and get bad return in profile dispatch_return function. The solution is to insert a simulate call in the helper function, to provide the context between helper frame and where the profile is defined.
I'd like to second Louie's call for additional review here. I've been reviewing them so far, and they look sensible to me, but I don't personally know the profile code base at all,
so there's a strong
chance I'll miss subtle details.
https://westurner.github.io/wiki/awesome-python-testing#benchmarks - plop, https://github.com/python/cpython/blob/master/Lib/profile.py https://github.com/python/cpython/blob/master/Lib/cProfile.py
For who may be reviewing cProfile / profile context manager problem, I prepare a post for explaining why the problem become, and why the patch will fix this problem: post: https://blog.louie.lu/2017/04/23/python-libs-profile-cant-context-manager/ Thanks, Louie. 2017-04-23 11:21 GMT+08:00 Louie Lu <me@louie.lu>:
Hi all,
I'm now looking for cProfile/profile lib's issue, and have solve a series of dependent problem, here is the list:
#9285 - Add a profile decorator to profile and cProfile #30113 - Allow helper functions to wrap sys.setprofile #18971 - Use argparse in the profile/cProfile modules #30118 - Add unittest for cProfile/profile command line interface
It can divide into two categories, first is the context manager problem, and the second is optparse to argparse problem.
1. context manager problem:
Relative issue: #9285, #30113 Relative PR: #287, #1212, #1253
This is an issue since 2010, and stop at profile can't simply add a __enter__ and __exit__ to make it a context manager. The main problem is, sys.setprofile() will hit the return and get bad return in profile dispatch_return function. The solution is to insert a simulate call in the helper function, to provide the context between helper frame and where the profile is defined.
2. optparse to argparse problem:
Relative issue: #18971, #30118 Relative PR: #1232 Relative patch: profile_argparse.patch
Serhiy have provide the patch of replace optparse to argparse in the profile and cProfile, but block by Ezio request of unittest for command line interface. The unittest is provide at #1232, and need to be reivew. If the unittest is add and argparse patch is apply, we can then solve more problem, e.g.: #23420, #29238, #21862
This is what I've investigated for cProfile / profile library now, to be move on, it will need others to review the work.
Thanks!
Best Regards, Louie.
participants (3)
-
Louie Lu -
Nick Coghlan -
Wes Turner