[XML-SIG] 4XSLT Performance Problems with Large Files

[Uche Ogbuji]

> Here is the memory used by the various processors during processing.
> 
> processor       decrease in free memory, MB
> msxsl                    17
> saxon                    21
> 4xslt.py                 32                   (Py 1.5.2)
> 4xslt.py                 45                   (Py 2.1.1)
> 4xst.bat              > 194
> 
> The 167 seconds using my script is not acceptable for my particular
> application, but the behavior when the transformation is launched by
> 4xslt.bat is impossible.  Why should the very same transformation take ten
> times the memory that msxsl or Saxon use?  You can't have an application run
> down your memory like this.  And I don't even know how much virtual memory
> was used on top of the 194 MB.  These results have been reasonably
> repeatable tonight.
> 
> I hope something can be done to improve the performance and memory usage for
> large files.  How about it, Uche and Mike?  Any thoughts about what is
> happening here?

I'm late to this as well, but I bet in your 4xslt.py files you used cDomlette 
rather than pDomlette.  pDomlette uses ridiculous amounts of memory.  
cDomlette is much better behaved.  cDomlette will be the default in 4Suite 
0.12.0, and pDomlette will probably be merged with minidom or eliminated.


> I'll be happy to send my files for testing if anyone likes.  The source file
> is pretty horrid ( I don't have any control over that, I'm afraid).  It has
> very long paths, and the element names are extremely long, the result of
> machine translation of some CORBA IDL.

This would be great for testing.  Please do send along.

Thanks.


-- 
Uche Ogbuji                               Principal Consultant
uche.ogbuji@fourthought.com               +1 303 583 9900 x 101
Fourthought, Inc.                         http://Fourthought.com 
4735 East Walnut St, Boulder, CO 80301-2537, USA
XML strategy, XML tools (http://4Suite.org), knowledge management