[pypy-dev] how to extend VTUNE support from pypy for application hot spot analysis

[Maciej Fijalkowski]

Hi Peter

The first request is fulfilled by vmprof

The second one can be worked on using the same mechanisms as vmprof -
there is C API that given the assembler address will give you the
python stack. It's defined in
rpython/jit/backend/llsupport/src/codemap.c I believe

On Thu, Aug 4, 2016 at 3:05 AM, Wang, Peter Xihong
<peter.xihong.wang at intel.com> wrote:
> HI Armin and Maciej,
>
> Let us know once you have something we could actually try.  One requirement is on the sampling/profiling overhead, ideally <1%, but >5% could be troublesome.  We'd like to allow people to do performance analysis on production systems.
>
> Meanwhile, could I make this as two separate requests:
> 1.  Application hot spot analysis.  Today I could run cProfile with CPython and get application code profiles running OpenStack Swift, but can't do the same thing with PyPy
> 2.  JITed code (assembly) mapping back to the application Python code.  VTUNE integration with HHVM and node.js are completed and working today, and I'd hope to see same capability with PyPy.
>
> Thanks,
>
> Peter
>
>
>
> -----Original Message-----
> From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of Armin Rigo
> Sent: Tuesday, August 02, 2016 1:18 AM
> To: Maciej Fijalkowski <fijall at gmail.com>
> Cc: Wang, Peter Xihong <peter.xihong.wang at intel.com>; pypy-dev at python.org
> Subject: Re: [pypy-dev] how to extend VTUNE support from pypy for application hot spot analysis
>
> Hi,
>
> On 2 August 2016 at 10:09, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> As far as I know (my team members tried this), vmprof does not allow us to attach to a running process? We will evaluate https://github.com/vmprof/vmprof-python if you think it's doable.
>>
>> You would need some form of process cooperation (I think) but it does
>> not seem impossible. What I would do is I would run a separate thread
>> that accepts something (e.g. a pipe write) and then starts vmprof.
>> vmprof once started is global to all threads
>
> Also, please note that I mentioned vmprof as a way to get started.
> You would need some *like*  what vmprof does; for a clean solution you don't want to enable an additional profiler on your vtune code.
>
>
> A bientôt,
>
> Armin.