[Numpy-discussion] Profiling (was GSoC : Performance parity between numpy arrays and Python scalars)

Arink Verma arinkverma at gmail.com
Mon Jun 17 17:06:22 EDT 2013


I am building numpy from source, python setup.py build --fcompiler=gnu95
then installation, python setup.py install --user, on ubuntu 13.04

for analysis results
pprof --svg /usr/bin/python py.prof



On Mon, Jun 17, 2013 at 10:04 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, Jun 17, 2013 at 4:29 PM, Arink Verma <arinkverma at gmail.com> wrote:
> > Hi Nathaniel
> >
> >> It's a probabilistic sampling profiler, so if it doesn't have enough
> >> samples then it can miss things. 227 samples is way way too low. You
> need to
> >> run the profiled code for longer (a few seconds at least), and if
> that's not
> >> enough then maybe increase the sampling rate too (though be careful
> because
> >> setting this too high can also add noise).
> >
> >
> > I ran code '1000000000' times, which record 229115 samples. Callgraph[1]
> > generated is converging to PyArray_DESCR, and rest are unconnected.
> >
> > Does it mean anything?
> > [1]
> https://docs.google.com/file/d/0B3Pqyp8kuQw0MzRoTTNVbmlaNFE/edit?usp=sharing
>
> I think it means that pprof is failing to walk to the stack to compute
> callers. That's consistent with PyArray_DESCR being the only "call"
> that it can find, because PyArray_DESCR isn't a real function, it
> always gets inlined, so detecting the caller doesn't require walking
> the stack.
>
> Is your numpy compiled with -fomit-frame-pointer or something like
> that? Any other weird build options used while building it? Is your
> binary stripped? If you're using a distro version, do you have the
> debug symbols installed? Did you compile this numpy yourself? (If not,
> the simplest thing to do would be to just build it yourself using the
> default settings and see if that helps.) What OS are you on?
>
> When I run your profiling code (but being lazier and only running
> 12000 samples), then do 'google-pprof --pdf /path/to/python
> /path/to/my/profile.prof', then I get the graph attached below.
>
> -n
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

Arink Verma
www.arinkverma.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130618/9d994ba8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.py
Type: application/octet-stream
Size: 254 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130618/9d994ba8/attachment.obj>


More information about the NumPy-Discussion mailing list