Mailman 3 September 2017 - lxml - The Python XML Toolkit

Test failure(s) in lxml-4.0.0
by Ken Moffat 20 Sep '17

20 Sep '17

Hi, testing 4.0.0 I'm getting the following error in with both python2 and python3 on x86_64 linux (LFS : gcc-7.2, python-2.7.13, python-3.6.2, with libxml2-2.9.4, libxslt-1.1.29 AND on another machine with libxml2-2.9.5, libxslt-1.1.30 with a fix so that xslt-config --cflags works again), both with linux-4.13.1. I will admit that I take a jaundiced view of testsuites, but now that I've read your page on building from source I see you want to know about failures ;) ERROR: test_xmlschema_parse_fixed_attributes (lxml.tests.test_xmlschema.ETreeXMLSchemaTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.6/unittest/case.py", line 59, in testPartExecutor yield File "/usr/lib/python3.6/unittest/case.py", line 605, in run testMethod() File "/scratch/working/lxml-4.0.0/src/lxml/tests/test_xmlschema.py", line 217, in test_xmlschema_parse_fixed_attributes parser=parser) File "/scratch/working/lxml-4.0.0/src/lxml/tests/common_imports.py", line 164, in parse return etree.parse(f, parser=parser) File "src/lxml/etree.pyx", line 3441, in lxml.etree.parse doc = _parseDocument(source, parser, base_url) File "src/lxml/parser.pxi", line 1851, in lxml.etree._parseDocument return _parseMemoryDocument(source.getvalue(), url, parser) File "src/lxml/parser.pxi", line 1871, in lxml.etree._parseMemoryDocument c_doc = _parseDoc(text, url, parser) File "src/lxml/parser.pxi", line 1752, in lxml.etree._parseDoc return (<_BaseParser>parser)._parseUnicodeDoc(text, c_filename) File "src/lxml/parser.pxi", line 1066, in lxml.etree._BaseParser._parseUnicodeDoc return context._handleParseResultDoc(self, result, None) File "src/lxml/parser.pxi", line 598, in lxml.etree._ParserContext._handleParseResultDoc return _handleParseResult(self, self._c_ctxt, result, File "src/lxml/parser.pxi", line 709, in lxml.etree._handleParseResult _raiseParseError(c_ctxt, filename, context._error_log) File "src/lxml/parser.pxi", line 638, in lxml.etree._raiseParseError raise error_log._buildParseException( File "<string>", line 0 lxml.etree.XMLSyntaxError: Element 'b', attribute 'hardy': The value '(null)' does not match the fixed value constraint 'hey'. On one build on the 2.9.4/1.1.29 system I also got two other failures, which I cannot repeat (I ran it for real and logged it, testing before and after only ever showed the above 1 error in each). ERROR: test_etree_parse_io_error (lxml.tests.test_io.ETreeIOTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.6/unittest/case.py", line 59, in testPartExecutor yield File "/usr/lib/python3.6/unittest/case.py", line 605, in run testMethod() File "/scratch/working/lxml-4.0.0/src/lxml/tests/test_io.py", line 276, in test_etree_parse_io_error dn = tempfile.mkdtemp(prefix=dirnameRU) File "/usr/lib/python3.6/tempfile.py", line 368, in mkdtemp _os.mkdir(file, 0o700) UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-18: ordinal not in range(128) and ERROR: test_etree_parse_io_error (lxml.tests.test_io.ElementTreeIOTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.6/unittest/case.py", line 59, in testPartExecutor yield File "/usr/lib/python3.6/unittest/case.py", line 605, in run testMethod() File "/scratch/working/lxml-4.0.0/src/lxml/tests/test_io.py", line 276, in test_etree_parse_io_error dn = tempfile.mkdtemp(prefix=dirnameRU) File "/usr/lib/python3.6/tempfile.py", line 368, in mkdtemp _os.mkdir(file, 0o700) UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-18: ordinal not in range(128) and similar in 2.7 - unless I'm missing something, these 2 error reports are identical. To be clear, in that run python2 ran 1787 tests and reported FAILED (errors=3) while python3 ran 1800 tests with a similar count of errors. ĸen -- Truth, in front of her huge walk-in wardrobe, selected black leather boots with stiletto heels for such a barefaced truth. - Unseen Academicals

3 4

Documentation
by Gregory 17 Sep '17

17 Sep '17

A little while ago I though it would be fun to use Sphinx and a theme derived from "Read the Docs" to build the lxml docs, and it worked rather well I think. I put it on github: https://gregoryvigotorres.github.io/lxml_docs/index.html The way I did it breaks some things and has some issues, but I think it can be made to work. Is there any interest in pursuing this further and possibly replacing the endearing though somewhat dated looking and eccentric current documentation? Gregory

4 3

Re: [lxml] Fwd: Reg Libxml2 upgrade
by Christian Fersch 17 Sep '17

17 Sep '17

Hi Swarnalatha, the previous releases were build with 2.9.3, due to 2.9.4 having a bug regarding schema validation. Unfortunately, 2.9.3 has it's own bugs, too. You can see some previous discussion here: https://bugs.launchpad.net/lxml/+bug/1706375 That discussion was then moved to this mailing list – unfortunately all of its web-accessible archives seem to be broken :(. If you have a newsreader, you can have a look at nntp://news.gmane.org/gmane.comp.python.lxml.devel. The thread title is "Should (Linux) binary wheels include libxml2/libxslt". If you need a quick fix, you can just compile lxml yourself with your own version of libxml2 – that's what we ended up doing. On 29.08.17 8:43:15, "Swarnalatha Kannan" <swarna298(a)gmail.com> wrote: > >Team, > >We are using lxml (lxml-3.3.4.win-amd64-py2.7) in my project which gets >shipped with libxml2 2.9.0. We would like to upgrade libxml2 to 2.9.4 >to mitigate security vulnerabilities (CVE-2016-4448 > >). For this upgrade we have to update lxml which has 2.9.4 libxml2. But >latest lxml for 2.7 python 64 bit windows is having 2.9.3. Let me know >how to solve the issue? Thanks. > >Python version used: python-2.7.12.amd64.msi > >

4 8

Fwd: Fwd: Reg Libxml2 upgrade
by Swarnalatha Kannan 06 Sep '17

06 Sep '17

Since Python 2 and 3 are not compatible, we are little worried on doing the code change now. Is there any easy way to do the same? Also i could only see lxml 3.8.0 available only till python 3.4 for windows and it has only 2.9.3 libxml2 version and we required latest libxml2 which is 2.9.4. Kindly let us know if lxml windows installer is available for python 3.6. Also the steps to do the migration from Python 2 to Python3 without much code changes? Thanks. [image: Inline image 1] python installer that we use: pywin32-218.win-amd64-py2.7.exe lxml --> lxml-3.3.4.win-amd64-py2.7.exe (it comes with 2.9.0 version of libxml2) Regards, Swarna On Tue, Sep 5, 2017 at 11:59 PM, Stefan Behnel <stefan_ml(a)behnel.de> wrote: > Hi, > > Swarnalatha Kannan schrieb am 05.09.2017 um 17:55: > > I would like to know if security vulnerabilities on libxml2 holds good > for > > lxml. We are using lxml-3.3.4.win-amd64-py2.7.exe in our project. After > > installation, I could only see .h files inside libxml fodler of lxml. > > (C:\Python27\Lib\site-packages\lxml\includes\libxml). I would like to > know > > about the implementations of these header file functions. Because most of > > the vulnerabilities mentioned are part of .c files. Kindly get back. > > The header files are only there for external code that wants to compile > against lxml. What you are interested in is the libxml2 version that the > Windows binaries include. You can look that up in lxml itself: > > print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION)) > print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION)) > print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION)) > print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION)) > print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION)) > > lxml 3.3.4 is hugely old and probably comes with some 2.7.x version of > libxml2, maybe 2.7.8. These old versions have bugs that most likely include > security relevant ones. > > I recomment switching to Python 3.5 or 3.6. The corresponding lxml wheels > for Windows come with more recent library versions. > > Stefan > _________________________________________________________________ > Mailing list for the lxml Python XML toolkit - http://lxml.de/ > lxml(a)lxml.de > https://mailman-mail5.webfaction.com/listinfo/lxml >

1 0