Pypy with Cython

[Oscar Benjamin]

On Thu, 3 Feb 2022 at 23:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> On 4/02/22 5:07 am, Albert-Jan Roskam wrote:
> >     On Feb 3, 2022 17:01, Dan Stromberg <drsalists at gmail.com> wrote:
> >
> >     What profiler do you recommend
>
> If it runs for that long, just measuring execution time should
> be enough. Python comes with a "timeit" module to help with
> that, or you can use whatever your OS provides (e.g. the
> "time" shell command in unices).

Yes, exactly. It's important to distinguish between timing or
benchmarking as compared to profiling.

When you use a profiler it does not usually give an accurate
representation of the time taken by the same code when the profiler is
not running because of the overhead added by the profiler itself. The
purpose of the profiler is instead to give lots of information that
can help you to *think* about how to make the code faster (e.g. what
proportion of time is spent in different functions or how many times
is a function called etc). This information is useful for considering
at a *high level* what parts of the code could be improved to make it
faster by e.g. calling a particular function less or making that
function faster.

The timeit module can be used to time micro-operations i.e. things
that take nanoseconds or maybe milliseconds. It repeats an operation
in a loop which is often needed to get reliable timings for things
that are otherwise too fast to reliably time from a single run. It can
give information that helps to identify good approaches to try at a
*low level* e.g. when optimising a single line of code.

If you want to *evaluate* whether or not a change actually makes your
*whole* program faster you should just run your code normally but time
how long it takes (which is what the unix "time" command does). You
can also use time.time() from Python for this. Profilers and timeit
help to identify ways to speed up your code but should not be used as
the way to assess the final impact of the changes you make to a long
running program.

--
Oscar