How to request data from a lazily-created tree structure ?

[méchoui]

Problem:

- You have tree structure (XML-like) that you don't want to create
100% in memory, because it just takes too long (for instance, you need
a http request to request the information from a slow distant site).
- But you want to be able to request data from it, such has "give me
all nodes that are under a "//foo/bar" tree, and have a child with an
"baz" attribute of value "zzz".

Question :

Do you have any other idea to request data from a lazily-created tree
structure ?

And does it make sense to create a DOM-like structure and to use a
generic XPath engine to request the tree ? (and does this generic
XPath engine exist ?)

The idea is to have the tree structure created on the fly (we are in
python), only when the XPath engine requests the data. Hopefully the
XPath engine will not request all the data from the tree (if the
request is smart enough and does not contain **, for instance).

Thanks