lxml - The Python XML Toolkit August 2010

lxml@python.org

17 participants
21 discussions

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

[lxml-dev] confusing xpath performance characteristics
by jholg＠gmx.de 24 Aug '12

24 Aug '12

Hi, I ran into some performance characteristics of lxml/libxml2 xpath that I find rather confusing: I try to find the @type attribute of a certain element in an XML Schema (which contains lots of complexType definitions with lots of elements in them; unfortunately I can't post the schema): >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.095885038375854492, 0.096823930740356445, 0.096174955368041992] So I think I'm being smart and give a little more path information - reckoning that this should *improve* performance: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.1770780086517334, 0.1775970458984375, 0.17748594284057617] Hm. Performance degrades slightly. I'm adding even more of the path to where my desired elements live in the schema: >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType//xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [103.79744100570679, 103.83671712875366, 103.61817717552185] What??? >>> timeit.Timer(stmt="""xpath(schema)""", setup="""from lxml import etree, objectify; schema=etree.parse('/ae/data/pydev/hjoukl/NDM/SVN_CO/TRUNK/ndm/reference/xsd/NDM.xsd').getroot(); xpath = etree.XPath('/xs:schema/xs:complexType/*/xs:element[@name="equity"]/@type', namespaces={'xs': 'http://www.w3.org/2001/XMLSchema'})""").repeat(number=10) [0.044407129287719727, 0.044126987457275391, 0.044229030609130859] >>> Ok, this version's better than my naive approach, which seems logical to me. But why would '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' perform drastically slower than '/xs:schema/xs:complexType//xs:element[@name="equity"]/@type' ? libxml2 problem? Running the same xpaths in Oxygen I don't notice performance differences (can't profile this). Holger -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

2 5

[lxml-dev] SyntaxErrors with Python 3
by Arfrever Frehtes Taifersar Arahesis 23 Nov '10

23 Nov '10

LXML r76211 generally supports Python 3, but there are still some SyntaxErrors. (The problem with ez_setup.py can be ignored.) $ python3.1 -m compileall -q . *** Error compiling ./benchmark/bench_etree.py ... File "./benchmark/bench_etree.py", line 10 UTEXT = u"some klingon: \F8D2" ^ SyntaxError: invalid syntax *** Error compiling ./benchmark/benchbase.py ... File "./benchmark/benchbase.py", line 10 _UTEXT = u"some klingon: \F8D2" * TREE_FACTOR ^ SyntaxError: invalid syntax *** Error compiling ./doc/mklatex.py ... File "./doc/mklatex.py", line 247 print "Creating %s" % outname ^ SyntaxError: invalid syntax *** Error compiling ./doc/rest2html.py ... File "./doc/rest2html.py", line 41 except ValueError, e: ^ SyntaxError: invalid syntax *** Error compiling ./doc/rest2latex.py ... File "./doc/rest2latex.py", line 44 except ValueError, e: ^ SyntaxError: invalid syntax *** Error compiling ./doc/s5/rst2s5.py ... File "./doc/s5/rst2s5.py", line 82 parsed = highlight(u'\n'.join(self.content), lexer, formatter) ^ SyntaxError: invalid syntax *** Error compiling ./ez_setup.py ... File "./ez_setup.py", line 106 except pkg_resources.VersionConflict, e: ^ SyntaxError: invalid syntax *** Error compiling ./src/local_doctest.py ... File "./src/local_doctest.py", line 372 raise TypeError, 'Expected a module: %r' % module ^ SyntaxError: invalid syntax *** Error compiling ./src/lxml/html/_diffcommand.py ... File "./src/lxml/html/_diffcommand.py", line 85 print "Not yet implemented" ^ SyntaxError: invalid syntax *** Error compiling ./src/lxml/html/tests/transform_feedparser_data.py ... File "./src/lxml/html/tests/transform_feedparser_data.py", line 91 print 'Bad data in %s:' % filename ^ SyntaxError: invalid syntax *** Error compiling ./src/lxml/tests/test_errors.py ... File "./src/lxml/tests/test_errors.py", line 1 # -*- coding: utf-8 -*- ^ SyntaxError: invalid character in identifier *** Error compiling ./tools/xpathgrep.py ... File "./tools/xpathgrep.py", line 5 except ImportError, e: ^ SyntaxError: invalid syntax -- Arfrever Frehtes Taifersar Arahesis

2 4

[lxml-dev] Access to ElementTree for XML schema
by Dave Kuhlman 15 Sep '10

15 Sep '10

I'm looking for a way to get access to an etree._ElementTree that represents an XML schema document in which the xsd:include and xsd:import elements have been recursively expanded. When I create an instance of etree.XMLSchema, libxml2 expands the underlying C tree for the schema. Am I right about that? If so, is there a way for me to get an etree._ElementTree that wraps that underlying C tree? Or, perhaps to have a way to create an etree._ElementTree from the XMLSchema object? If that document is not available, I suppose that I am asking for a new feature that enables us to retrieve the processed and expanded etree Document from an etree.XMLSchema object. Or, is there already some other way to get an XML schema document tree in which the include and import elements have been (recursively) expanded? The reason I'm asking for this -- I process XML schema documents, and I think we should encourage other Python hackers to do so, too. This (new) feature would enable lxml to support that. I'm trying to implement this capability myself in Python using lxml, but my implementation still has bugs and I'm sure that libxml2 does it better than I can. Thanks for help. - Dave -- Dave Kuhlman http://www.rexx.com/~dkuhlman

2 2

[lxml-dev] Code hangs when calling etree.XMLSchema
by Bryan Hughes 04 Sep '10

04 Sep '10

When I pass in a valid XSD file into this line of code, the application hangs: logging.debug("start") xmlschema_ = etree.XMLSchema(xsd_file) logging.debug("finish") I've wrapped that piece of code with debug logging to confirm that it hangs. In other words, my logs will print "start", but do not print "finish". Unfortunately, no exceptions are thrown -- it simply does nothing once that line of code executes. Any thoughts on what may be happening?

2 1

[lxml-dev] Question about etree vs html
by Dimitrios Pritsos 02 Sep '10

02 Sep '10

Hello, I am Dimitrios Pritsos and I am working on a WebCrawler. In order to analyse the pages that I am getting while crawling I am using lxml. However I cannot tell the difference of lxml.html and lxml.etree when coming to the XHTML parsing. In particular I am confused of what to use from the variety of options lxml is providing. Moreover, the documentation is a bit misleadings. Let me be more specific. Firstly I ve seen that lxml.html has been developed on Python and in fact is a shortcut for extracting several common information from an HTML page instead of building your own paths and xpaths, similarly to XML() and HTML() shortcuts. In addition all of these sortcuts are using the HTML() (ie the HTMLParser()). Unfortunately this took me few days to realize it and I found the answer here: http://zdar.trinet.as/doc/python-lxml-2.0.11/doc/html/api/lxml-module.html. Because no documentation is clarifying this. Not even the one of John W. Shipman, which is the best for newbies like me. However, in the documentation (found in http://codespeak.net/lxml/lxmldoc-2.2.7.pdf) there is a statement that says that "Note that XHTML is best parsed as XML, parsing it with the HTML parser can lead to unexpected results". Considering that, using lxml.etree is the best choice for the www right because of the great variety of web pages are in XHTML and not HTML markup. On the other hand lxml.html has all the good staff. So, what exactly is going on here which library should I use, or how I could combine them for not loosing any information from the pages? After several test, for several days, I found that different "parsing" function gives different results and different tostring() call (from html or etree) again gives different results even for the same ElementTree. So, why is that? No Documentation found for this eather. In general the lxml it seems to me really great, however, because of the limited documentation some times you cannot tell what is what and all just seems a different path to do the same thing, but this is not the case as I can tell from my several tests. So, in practice it is totally different. For example try this: >>>xhtmlsrc = '<!DOCTYPE html PUBLIC "-//W3C/DTD XHTML 1.0 Transitional//EN" "http://www.w3c.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><div><br />Testing</div>' >>> lxml.etree.tostring(lxml.html.soupparser.fromstring(xhtmlsrc)) '<html>DOCTYPE html PUBLIC "-//W3C/DTD XHTML 1.0 Transitional//EN" "http://www.w3c.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"<div><br/>Testing</div></html>' >>> lxml.etree.tostring(lxml.html.fromstring(xhtmlsrc)) '<html><body><div><br/>Testing</div></body></html>' >>> lxml.etree.tostring(lxml.html.parse(StringIO(xhtmlsrc))) '<!DOCTYPE html PUBLIC "-//W3C/DTD XHTML 1.0 Transitional//EN" "http://www.w3c.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html><body><div><br/>Testing</div></body></html>' /Why the above give different result when based on the documentation the suppose to give the same result/? >>> xhtmltree = lxml.html.parse(StringIO(xhtmlsrc)) >>> xhtmltree.test_content() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'lxml.etree._ElementTree' object has no attribute 'test_content' >>> >>> xhtmltree = lxml.html.document_fromstring(xhtmlsrc) >>> xhtmltree.text_content() 'Testing' >>> /Again why there is this deferent result when it documentation it is not reported?/ So could you please advise me what should I do? And one more question: When I am using XMLParser() which DTD is used for building the ElementTree? I the case of HTMLParser() I can tell it is HTML 4.0 because this is what I get when I am doing this: >>> xhtmlsrc2 = '<div><br />Testing</div>' >>> xhtmltree = lxml.html.parse(StringIO(xhtmlsrc2)) >>> lxml.html.tostring(xhtmltree) '<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n<html><body><div><br>Testing</div></body></html>' >>> PLEASE I NEED SOME HELP HERE! Best Regards, Dimitrios

3 3

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit August 2010