[pypy-dev] how to extend VTUNE support from pypy for application hot spot analysis

[Wang, Peter Xihong]

Hi Armin and Maciej,

We are looking into "rpython/jit/backend/llsupport/src/codemap.c" for the mapping.  Really appreciate this tip.
Regarding to VTUNE license/open source, fully understood the concern, and I did escalate the issue.

Thanks,

Peter

 
-----Original Message-----
From: Maciej Fijalkowski [mailto:fijall at gmail.com] 
Sent: Wednesday, August 03, 2016 11:06 PM
To: Wang, Peter Xihong <peter.xihong.wang at intel.com>
Cc: Armin Rigo <arigo at tunes.org>; pypy-dev at python.org
Subject: Re: [pypy-dev] how to extend VTUNE support from pypy for application hot spot analysis

Hi Peter

The first request is fulfilled by vmprof

The second one can be worked on using the same mechanisms as vmprof - there is C API that given the assembler address will give you the python stack. It's defined in rpython/jit/backend/llsupport/src/codemap.c I believe

On Thu, Aug 4, 2016 at 3:05 AM, Wang, Peter Xihong <peter.xihong.wang at intel.com> wrote:
> HI Armin and Maciej,
>
> Let us know once you have something we could actually try.  One requirement is on the sampling/profiling overhead, ideally <1%, but >5% could be troublesome.  We'd like to allow people to do performance analysis on production systems.
>
> Meanwhile, could I make this as two separate requests:
> 1.  Application hot spot analysis.  Today I could run cProfile with 
> CPython and get application code profiles running OpenStack Swift, but can't do the same thing with PyPy 2.  JITed code (assembly) mapping back to the application Python code.  VTUNE integration with HHVM and node.js are completed and working today, and I'd hope to see same capability with PyPy.
>
> Thanks,
>
> Peter
>
>
>
> -----Original Message-----
> From: armin.rigo at gmail.com [mailto:armin.rigo at gmail.com] On Behalf Of 
> Armin Rigo
> Sent: Tuesday, August 02, 2016 1:18 AM
> To: Maciej Fijalkowski <fijall at gmail.com>
> Cc: Wang, Peter Xihong <peter.xihong.wang at intel.com>; 
> pypy-dev at python.org
> Subject: Re: [pypy-dev] how to extend VTUNE support from pypy for 
> application hot spot analysis
>
> Hi,
>
> On 2 August 2016 at 10:09, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> As far as I know (my team members tried this), vmprof does not allow us to attach to a running process? We will evaluate https://github.com/vmprof/vmprof-python if you think it's doable.
>>
>> You would need some form of process cooperation (I think) but it does 
>> not seem impossible. What I would do is I would run a separate thread 
>> that accepts something (e.g. a pipe write) and then starts vmprof.
>> vmprof once started is global to all threads
>
> Also, please note that I mentioned vmprof as a way to get started.
> You would need some *like*  what vmprof does; for a clean solution you don't want to enable an additional profiler on your vtune code.
>
>
> A bientôt,
>
> Armin.