[pydotorg-www] editing https://wiki.python.org/moin/PythonSpeed/PerformanceTips

[Emery Berger]

Hi,

I'd like to edit this to update some stuff (there are broken links, and
outdated packages), as well as to include a reference to Scalene, a Python
profiler I developed that offers a number of useful advantages over other
profilers.

Thanks,
-- emery (Scalene developer, among many other things)

>From https://github.com/emeryberger/scalene:

Scalene is a high-performance CPU *and* memory profiler for Python that
does a few things that other Python profilers do not and cannot do. It runs
orders of magnitude faster than other profilers while delivering far more
detailed information.

   1. Scalene is *fast*. It uses sampling instead of instrumentation or
   relying on Python's tracing facilities. Its overhead is typically no more
   than 10-20% (and often less).
   2. Scalene is *precise*. Unlike most other Python profilers, Scalene
   performs CPU profiling *at the line level*, pointing to the specific
   lines of code that are responsible for the execution time in your program.
   This level of detail can be much more useful than the function-level
   profiles returned by most profilers.
   3. Scalene separates out time spent running in Python from time spent in
   native code (including libraries). Most Python programmers aren't going to
   optimize the performance of native code (which is usually either in the
   Python implementation or external libraries), so this helps developers
   focus their optimization efforts on the code they can actually improve.
   4. Scalene *profiles memory usage*. In addition to tracking CPU usage,
   Scalene also points to the specific lines of code responsible for memory
   growth. It accomplishes this via an included specialized memory allocator.

Below is a table comparing the performance of various profilers to Scalene:

ProfilerTimeSlowdown
*original program* 6.71s 1.0x
cProfile 11.04s 1.65x
Profile 202.26s 30.14x
pyinstrument 9.83s 1.46x
line_profiler 78.0s 11.62x
pprofile *(deterministic)* 403.67s 60.16x
pprofile *(statistical)* 7.47s 1.11x
yappi *(CPU)* 127.53s 19.01x
yappi *(wallclock)* 21.45s 3.2x
py-spy 7.25s 1.08x
memory_profiler *> 2 hours* >1000x
scalene *(CPU only)* 6.98s 1.04x
scalene *(CPU + memory)* 7.68s 1.14x

the features of other profilers vs. Scalene:
ProfilerLine-level?CPU?Wall clock vs. CPU time?Python vs.
native?Memory?Unmodified
code?Threads?
cProfile ✔ wall clock ✔
Profile ✔ CPU time ✔
pyinstrument ✔ wall clock ✔
line_profiler ✔ ✔ wall clock
pprofile *(deterministic)* ✔ ✔ wall clock ✔ ✔
pprofile *(statistical)* ✔ ✔ wall clock ✔ ✔
yappi *(CPU)* ✔ CPU time ✔ ✔
yappi *(wallclock)* ✔ wall clock ✔ ✔
py-spy ✔ ✔ both ✔ ✔
memory_profiler ✔ ✔
scalene *(CPU only)* ✔ ✔ both ✔ ✔ ✔
scalene *(CPU + memory)* ✔ ✔ both ✔ ✔ ✔ ✔
--
Professor Emery Berger
College of Information and Computer Sciences
University of Massachusetts Amherst
www.emeryberger.org, @emeryberger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pydotorg-www/attachments/20200307/42384ecd/attachment-0001.html>