lxml - The Python XML Toolkit February 2019

lxml@python.org

4 participants
10 discussions

[lxml-dev] Reparenting a node
by Lawrence Oluyede 30 Jan '23

30 Jan '23

I have a doc A and a doc B, I'd like to put a node extracted from A in the document B but I always get a ValueError: ValueError: Element is not a child of this node. I didn't find any "setparent" in the API. How can I do this? -- Lawrence, oluyede.org - neropercaso.it "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair

3 2

[lxml-dev] Checking whether a node is a comment/element
by Geoffrey Sneddon 11 Jan '23

11 Jan '23

Hi, What's the best way to check whether a given node is a comment or an element? For the former, I'm currently using isinstance(node, etree._Comment), which is rather obviously sub-optimal. -- Geoffrey Sneddon <http://gsnedders.com/>

4 4

[lxml-dev] lxml 2.0.5 released
by Stefan Behnel 11 Jan '23

11 Jan '23

Hi all, lxml 2.0.5 is on PyPI. This is a bug-fix-only release of the stable 2.0 series. Have fun, Stefan 2.0.5 (2008-05-01) Bugs fixed * Resolving to a filename in custom resolvers didn't work. * lxml did not honour libxslt's second error state "STOPPED", which let some XSLT errors pass silently. * Memory leak in Schematron with libxml2 >= 2.6.31.

3 4

[lxml-dev] lxml has its page on launchpad
by Stefan Behnel 21 Sep '22

21 Sep '22

Hi all, I added the lxml project to launchpad, the Ubuntu Bug-Tracker. It also has a FAQ engine and a couple of other goodies. https://launchpad.net/lxml It's easy to sign up for launchpad, BTW, no 90%-footnotes-contract. Have fun, Stefan

3 2

[lxml-dev] Building LXML Trunk
by Sidnei da Silva 31 Aug '22

31 Aug '22

Hi, I've tried to build lxml from trunk today, on Win32. Got the following error: src\lxml\etree.c(880) : error C2059: syntax error : ')' src\lxml\etree.c(881) : error C2059: syntax error : ')' src\lxml\etree.c(882) : error C2059: syntax error : ')' src\lxml\etree.c(883) : error C2059: syntax error : ')' Any clue? Smells like a Pyrex issue? -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214

4 4

Fwd: validation error message consistency across Python versions
by Tom Kralidis 15 Mar '19

15 Mar '19

Hi: I'm finding different behaviour when trying validate XML documents against lxml 4.3.0 in Python 2.7 and 3.x environments. Example validation error using 4.3.0 and Python 2.7.15, libxml2 2.9.3+dfsg1-1ubuntu0.6 (xenial) Error: Element '{http://www.opengis.net/foo}Filter': This element is not expected. Expected is one of ( {http://www.opengis.net/ogc}Filter, {http://www.opengis.net/cat/csw/2.0.2}CqlText ). (line 0) Example validation error using 4.3.0 and Python 3.4.8, libxml2 2.9.3+dfsg1-1ubuntu0.6 (xenial) Error: Element '{http://www.opengis.net/foo}Filter': This element is not expected. Expected is one of ( {http://www.opengis.net/ogc}Filter, {http://www.opengis.net/cat/csw/2.0.2}CqlText ). (<string>, line 0) As a result this causes our (strict) functional testing to fail. Any idea what can be done to keep error messages consistent? Thanks ..Tom

3 3

Can't process from string
by Ovnicraft 15 Mar '19

15 Mar '19

Hi I am trying to process this file https://github-production-repository-file-5c1aeb.s3.amazonaws.com/142435147… But I have problems with tags, it doesn't in correct form, 3th party system generate <> with lt/gt; in tags. How can i resolve this to load with fromstring() ? -- Sent from Gmail Mobile

2 1

Crash in lxml 4.3.1
by Paul Adenot 01 Mar '19

01 Mar '19

Hi, I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing. This can be reproduced deterministically (on travis and also on my local linux machine), by doing: git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec Let me know if I can help debugging this, I've posted some background info on https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675 Thanks, Paul.

2 4

LXML getchildren() method does not return a child div tag with style="display:none"
by 王凡 28 Feb '19

28 Feb '19

I was using beautifulsoup to parse the page at: https://irs.thsrc.com.tw/IMINT/ In the rendering, there is a confirmation box popping out in front of all other tags asking for a confirmation. The tag is inside the sole form tag with xpath: /html/body/div[1]/form/div[2]. Both this tag and the previous tag at /html/body/div[1]/form/div[1] are with style attribute "display:none". The weird thing is that when I used getchildren() to get the child tags of the form tag, the second DIV tag in the form tag is not in the returned list. In fact, LXML will skip the second tag. I tried beautifulsoup. But the problem is the same. E.children attribute does not contain the second div tag in the form tag. I copied part of the html code and the whole method that recursively scan all tags in the following. Some unicode data that cannot pass the stackoverflow filter is changed to ascii. I will appreciate it very much if anyone can tell me how to get the tag which is the pop-out tag waiting for the confirmation. Thanks <html> <head> <title> taiwan hsrc </title> </head> <body topmargin="0" rightmargin="0" bottommargin="0" bgcolor="#FFFFFF" leftmargin="0">  <form action="/IMINT/;jsessionid=4A74C40B8D68474DF0B6F49E953DD825?wicket:interface=:0:BookingS1Form::IFormSubmitListener" id="BookingS1Form" method="post"> <div style="display:none"> <input type="hidden" name="BookingS1Form:hf:0" id="BookingS1Form:hf:0" /> </div> <div style="display:none; padding:3px 10px 5px;text-align:center;" id="dialogCookieInfo" title="Taiwan high-speed rail" wicket:message="title=bookingdialog_3"> <div class="JCon"> <div class="TCon"> <div class="overDiffText"> <div style="text-align: left;"> <span>for better service <a target="_blank" class="c" style="color:#FF9900;" href="https://www.thsrc.com.tw/tw/Article/ArticleContent/d1fa3bcb-a016-47e2-88c6-…"> privacy </a> 。 </span> </div> </div> <div class="action"> <table border="0" cellpadding="0" cellspacing="0" align="center"> <tr> <td> <input hidefocus="true" name="confirm" id="btn-confirm" type="button" class="button_main" value="我同意"/> </td> </tr> </table> </div> </div> </div> </div> <div id="content" class="content">  <marquee id="marqueeShow" behavior="scroll" scrollamount="1" direction="left" width="755"> </marquee>  <div class="tit"> <span>一般訂票</span> </div> </form> |</div> </body> </html> My code with LXML for scanning the html is the following. def actionableLXML(cls, e): global count print ("rec[", count, "], xpath: ", xmlTree.getpath(e)) countLabelActionableInside += 1 flagActionableInside = False if e.tag in cls._clickable_tags \ or e.tag == 'input' or e.tag == 'select': flagActionableInside = True else: flagActionableInside = False for c in e.getchildren(): flagActionableInside |= cls.actionableLXML(c) if e.attrib and 'style' in e.attrib \ and 'display:' in e.attrib['style'] \ and 'none' in e.attrib['style']: if not flagActionableInside: e.getparent().remove(e) return flagActionableInside

2 1

lxml.html.diff.htmldiff does not handle HTML comments well.
by Jens Quade 12 Feb '19

12 Feb '19

Hi, basically, HTML comments are currently handled like elements in html.diff's flatten_el, which leads to interesting results: >>> from lxml.html.diff import htmldiff >>> >>> a = '<p>test</p>' >>> b = '<p>test</p>' >>> >>> print(htmldiff(a,b)) <p>test <ins><<cyfunction comment at>>Hey <cyfunction comment at>></cyfunction></cyfunction></ins> </p> I added a check to not include comments in the generated token list, which effectively strips all comments from the diff: diff --git a/src/lxml/html/diff.py b/src/lxml/html/diff.py index 5d143bd2..9d4a4f72 100644 --- a/src/lxml/html/diff.py +++ b/src/lxml/html/diff.py @@ -4,7 +4,7 @@ from __future__ import absolute_import import difflib from lxml import etree -from lxml.html import fragment_fromstring +from lxml.html import fragment_fromstring, HtmlComment import re __all__ = ['html_annotate', 'htmldiff'] @@ -688,6 +688,14 @@ def flatten_el(el, include_hrefs, skip_tag=False): If skip_tag is true, then the outermost container tag is not returned (just its contents).""" + + if isinstance(el, HtmlComment): + if el.tail: + end_words = split_words(el.tail) + for word in end_words: + yield html_escape(word) + return + if not skip_tag: if el.tag == 'img': yield ('img', el.get('src'), start_tag(el)) jens

2 2

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

lxml - The Python XML Toolkit February 2019