Kasimier Buchcik wrote:
Hi,
On Tue, 2005-09-13 at 11:03 +0200, Martijn Faassen wrote:
David Levy wrote:
Thanks ! I did test it successfully too yet. However iterpase is not supported :( So i can not parse large file (>100Mo) ...?! I am trying to mix celementtree and lxml right now !!
lxml right now is not optimal for parsing such large files. Support for iterparse would be nice though likely a lot of work, and full xpath support in that context might be extremely hard to accomplish.
libxml2 does seem to have something called 'streaming xpath' that might be useful in this area though (but this is not yet exposed to lxml either).
I dunno if this can help, but Libxml2 has the pattern.c module, which is, amongst others, used by the xmlschemas.c module for the XPaths of identity-constraints. It is streamable, but supports only a minimal subset of XPath: only element and attribute tests without predicates, and only the child and descendant-or-self axis. An example would be: ".//foo/boo/@bar" - and that's it already.
Is this the 'streamable XPath' I heard mentions of or is this something else? Note that I think David is interested in lxml precisely because it offers more complete XPath support than ElementTree; the subset you describe here is quite similar to ElementTree's. So, using pattern.c wouldn't be much of a big win for his particular use case. Then again, I think streaming anything close to full xpath is a very difficult problem; I recall reading a research paper about it for instance, so David might want something that doesn't really exist yet... Regards, Martijn