[XML-SIG] Reading characters

Albert Chin xml-sig@python.org
Tue, 21 May 2002 19:09:26 -0500


On Tue, May 21, 2002 at 07:49:34PM -0400, Thomas B. Passin wrote:
> [Albert Chin]
> 
> > > What do you mean by "read through"? Can you share a bit of code?
> >
> >     def characters (self, ch, start, length):
> >       if self.in_data and self.extract:
> >         self.payload = self.payload + ch[start:start+length]
> >
> 
> If there are many "ch" chunks to add to the payload, that might be the
> slowest part of the chain, especially if the payload gets large.  There are
> two standard ways to speed such string concatenations up.  You either use
> CStringIO and get the data from it when you are done adding, or you append
> each chunk to a list, then do list.join() at the end.  Either one can give
> dramatic speedups if that's your problem.

				WOW! WOW! WOW!

Without cStringIO:
  def characters (self, ch, start, length):
    if self.in_data and self.extract:
      self.payload += ch

  $ time ./pkg-inst [blah] [blah]
  93.25s user 0.48s system 94% cpu 1:39.50 total

With cStringIO:
  def characters (self, ch, start, length):
    if self.in_data and self.extract:
      self.payload.write (ch)

  $ time ./pkg-inst [blah] [blah]
  5.36s user 0.41s system 50% cpu 11.467 total

This is just unbelievable! Thanks a lot!

-- 
albert chin (china@thewrittenword.com)