[Tutor] performance loss -- profiling

[spir]

Le Mon, 18 May 2009 23:16:13 +0100,
"Alan Gauld" <alan.gauld at btinternet.com> s'exprima ainsi:

> "spir" <denis.spir at free.fr> wrote
> 
> > Also, it's the first time I really have to cope with machine-time; 
> > so I'm totally new to technics like using a profiler. 
> > Any hints on the topics heartfully welcome :-)
> 
> Profilers are a bit like debuggers. Very useful when needed 
> but usually a point of last resort.
> 
> First, what OS are you on?
> Linux/Unix has a host of tools that can help pinpoint the causes, 
> such as whether there is excess disk I/O or memory use or 
> if its another process stealing CPU resource.
> 
> Windows can do these things too but the tools and techniques 
> are totally different - and require a bit more experience to 
> interpret IMHO.
> 
> But my first bet would be to check the CPU and memory usage 
> to see if either is high. Also if it uses the network, either to go to 
> a server(databases? files?) then check network traffic.
> 
> If you can spot the nature of the hot spot you can often guess 
> where in the code the issues are likely to be.
> 
> Other common causes to consider are:
> - Locking of files? ie concurrent access issues?
> - Queuing for shared resources? Have you started running multiple 
>   instances?
> - data base query time - have you increased the volume of 
>   data being searched? Did you index the key columns?
> - Directory path traversal - have you moved the location of 
> any key files? Either the executables/scriopts or the data?
> 
> Just some ideas before you resort to profiling.

Thank you for all these tips, Alan.
I've found the naughty bug.
[It brings another question: see separate thread.]

For the story, the app in question is a parsing and processing tool (see http://spir.wikidot.com/pijnu). I've put muuuch attention to usability, esp. feedback to developper. To help users fix grammars, which is always a difficult job, parse errors are very complete and even include a visual pointer to the failure location in source text.
The main trick is that "wrapper" patterns also report errors from sub-patterns. For instance, a sequence will tell the user which pattern in the sequence has failed and add this pattern's own error on output; a choice will report why _each_ pattern in the choice has failed. To do this, wrapper patterns have to record sub-pattern errors: _my_ error was that they recorded the error message (which is complicated to create) instead of a reference to the error object. Error objects correctly don't write their message until they are requested -- but wrapper patterns asked them to do it.
As a consequence, the app spent ~98% of its time writing error messages that never got printed ;-) My biggest test's running time had raised from ~1.5s to more than 1mn!

Denis
------
la vita e estrany