[Baypiggies] Output generation - (string operations/triple quote/template)

[Dennis Reinhardt]

At 03:50 PM 2/13/2007, wesley chun wrote:
>i cannot justify the CPU time and performance
>lost in doing 
>malloc()+realloc()+realloc()+realloc()+realloc()+realloc()+realloc()+realloc()+realloc()..

We can quantify this, albeit abstractly.  The actual allocation and re-use 
of memory is not known to me.  The simplest case to consider is once 
allocated, memory is never re-claimed, also a worst case.  If we allocate a 
memory object N units in length, the memory impact is N.  If we allocate 
this in units of 1 by copy/append (a.k.a tail recursion), the impact is as 
bad as

         1+2+3+4 ....

or more generally N*(N+1)/2

For moderate N (say =10), the impact is 5.5X worse than allocating a single 
block of N.  For larger N, the N*(N+1)/2 gets bad fast.  If N is less than 
10, the impact is better than 5.5X.

A copy/append such as

         A += ...    [done 30 times]

can be re-expressed as

         X += ...    [done 10 times]
         Y += ...    [done 10 times]
         Z += ...    [done 10 times]
        A  + X + Y + Z [done once]

In DialogDevil, scanning and processing click actions against the running 
processes *is* a CPU drain and I have had to put a lot of optimization work 
into that.  In SpamAI, I had to pay lots of attention to efficiency of 
regular expressions I wrote.

HTML generation simply is not even a blip on the cpu radar screen for 
either program, both of which required optimization work.

Regards, Dennis





  ---------------------------------
| Dennis    | DennisR at dair.com    |
| Reinhardt | http://www.dair.com |
  ---------------------------------