Mailman 3 July 2018 - lxml - The Python XML Toolkit

ignore duplicate xmlids
by Martin Mueller 11 Jul '18

11 Jul '18

Lxml will abort if there are duplicate xmlids. Which is great except when it is not. Is there a way of disabling this feature on a case-by-case basis? In my case I did something stupid and ended up with lots of elements that have duplicate ids. The simplest way to deal with them would be to process the file again and so something about the bad IDs. But you can’t do that with lxml. I could replace xml:id with some other attribute name, remove the schema, fiddle with the files, change the attribute name back to xml:ids and hope for the best. But there must be a more elegant way. Martin Mueller

1 0

tostring ignores tag attribute, bug ?
by Jamesie Pic 11 Jul '18

11 Jul '18

Hello everybody, Thanks a lot for maintaining this awesome library. I don't understand why tostring ignores the tag attribute of my Element, and enforces using their class name, any recommendation / workaround idea or clue is welcome ! Test script: from lxml import etree class Component(etree.ElementBase): pass class Div(Component): tag = 'div' class Col(Div): def __init__(self, l=None): self.l = l super().__init__() def _init(self): super()._init() if 'class' not in self.attrib: self.attrib['class'] = 'col' else: self.attrib['class'] += ' col' if self.l: self.attrib['class'] += ' l{}'.format(self.l) el = Col(l=2) el.tag = 'div' print('el repr', el) print('expected: ', b'<div class="col l2"/>') print('result: ', etree.tostring(el)) Result: el repr <Element div at 0x7fe6923ada48> expected: b'<div class="col l2"/>' result: b'<Col class="col l2"/>' As you can see, tostring enforces the class name instead of using the tag attribute, like repr does. https://bugs.launchpad.net/lxml/+bug/1780950 Python : sys.version_info(major=3, minor=6, micro=6, releaselevel='final', serial=0) lxml.etree : (4, 2, 3, 0) libxml used : (2, 9, 8) libxml compiled : (2, 9, 8) libxslt used : (1, 1, 32) libxslt compiled : (1, 1, 32)

2 2

Re: [lxml] lxml resolving with https
by Duthie, Roger J.A. 06 Jul '18

06 Jul '18

Hi Anrik, My solution was to implement a resolver which checks for TLS/SSL before either resolving the resource itself or delegating it to libxml2 (I think that's how resolve_filename works, I'm happy to be corrected!). Find a copy of my resolver attached. I've removed my own application logging and added a few comments which hopefully quickly explain the coding decisions. Comments and feedback are welcome. To use this resolver, follow the documentation on the lxml site: https://lxml.de/resolvers.html#uri-resolvers All the best, Roger ________________________________ From: Anrik Drenth <anrikd(a)hotmail.com> Sent: 05 July 2018 22:42:28 To: Duthie, Roger J.A. Subject: lxml resolving with https Hi Roger, Have just spotted your emails to the group. I've also struck issues with lxml/https when loading supporting XML schemas from the main schema. With Chrome's intention of showing http url's as "unsecure" I'm racing to try to find a solution. If you find a solution please let me know. Will post if I find an answer. Thank you Anrik ________________________________ From: lxml <lxml-bounces(a)lxml.de> on behalf of Duthie, Roger J.A. <rogie(a)bas.ac.uk> Sent: Tuesday, 26 June 2018 2:20 PM To: Stefan Behnel; lxml(a)lxml.de Subject: Re: [lxml] Default parser with/without no_network Hi Stefan & the LXML list, Thanks for the feedback on these issues. Once I have more information, I'm happy to help improve the documentation to explain to new users these quirks and gotchas (as long as they're not bugs). For example: Stefan wrote: > Basically, when you define your own custom resolver, it's up to that > resolver to decide whether it should allow network access or not. ^^ This would be useful to have in the docs! However, generally what's going on with the resolvers is not clear from the documentation as it stands. I've already tried to dive into the Cython and (libxml/lixslt) C code and I don't know if I would have much time to help debug this stuff. Is there any developers' documentation, explaining how the package is plumbed? Update on the testing: ---------------------------- I've been testing the redirects with some files hosted on Amazon's S3. So far, with a XMLSchema object at least - i.e. as opposed to an isoschematron object - a validator can be created - i.e. etree.XMLSchema(schema_etree) can resolve all the resources. The test resources feature 301 and 302 redirects. I now suspect that SSL might be to blame, as the [real world] example I had previously had a redirect to an htttps URL. I need to do some tests on this, but setting up an SSL certificate is a little more work. Is it already known that libxml2 cannot handle https? Roger ________________________________ From: lxml <lxml-bounces(a)lxml.de> on behalf of Stefan Behnel <stefan_ml(a)behnel.de> Sent: 25 June 2018 20:19:02 To: lxml(a)lxml.de Subject: Re: [lxml] Default parser with/without no_network Hi! Duthie, Roger J.A. schrieb am 25.06.2018 um 13:38: > Hello again, > > An update on my attempts to work out the behaviour of the parsers/resolvers. This thread (Question 2) involved the validation of XML against a schematron and the apparent inability of the parser to resolve remote resources. > > > It seemed that my [real world] example was throwing up a few issues. Going back to my example code and output: https://gist.github.com/rogie-bas/e157501d12cff8ac0aafc0cc120ce8dc > > (The link in the previous email may have been broken, because the fullstop was added to the URL, the above will hopefully work without modification!) > > > The first test logging (base resolver, no_network=True) shows that the validation fails with an XSLTApplyError, due to a resource not be resolvable (..../gmxCodeLists.xml). Confusingly, it doesn't seem to be the first one on the list of resources that the code has tried to resolve. However, the behaviour is different if no_network=False. It seems that the resource that the code chokes on is a different one (.../SDN:L11::D08). I may have a tentative reason why on this (based on empirical evidence). > > > I broke my schematron file down into single patterns which I deemed targetting parts of the XML file containing various remote resources. This way I could try resolving the various resources with the various parsers, with no_network = {True | False}. The gmxCodeLists.xml resource is not resolved with the base resolver if no_network=True, but is when no_network=False. The resolution of the other resource (SDN:L11::D08) never happens. If using resolve_filename() method rather than the base resolver, the error is different: an OSError is thrown for these resources, as if the resource was deemed to be a file system resource, instead of a remote resource; however, the gmxCodeLists.xml is resolved correctly. The resolver I created using the Requests package doesn't have a problem with this resource. I find it difficult to follow this description, but at least the gist cases 1-4 looked reasonable to me. Whether an OSError is the best exception for a remote URL, ... well, I guess it's not, but I'm not sure it's worth fixing (as it would probably also break someone's existing code). > I've now got a hunch that this is perhaps because the URI for this problematic resource is a redirect (checking with curl -IL, shows that it is a 302/301/200 chain). I can't currently rule out that there is something else about this resource that makes it unresolvable, but when I change the references to this resource, to point at the redirect, the validation completes. My next task is to create some simple external resources with redirects. > > > Is any of this behaviour expected? > > - no_network has no effect apart from with the base resolver Basically, when you define your own custom resolver, it's up to that resolver to decide whether it should allow network access or not. If your resolver passes the lookup on to the default resolver, either by resolving to a filename/URL or by not resolving at all, then the option applied again. Furthermore, "no_network" is a parser option. The fact that it also impacts the document loading in isoschematron (i.e. XSLT) is more of a (reasonable) side-effect because that uses the parser of the input document to load related documents. ("Input document" here is the actual input/validated document for XSLT/isoschematron for lookups related to the input document, and the schema document for lookups related to the schema, e.g. includes etc.) > - redirects are unresolvable by the base resolver and resolve_filename base resolution I don't think I ever tested redirects, but the input layer of libxml2 should handle them. I would expect redirects to work for both the default resolver and after resolving through resolve_filename(). If they don't then that's a bug. OTOH, if custom resolvers are in place then the question is whether each redirect should first call back and ask them. I guess it would be nice if it did, but I'm not sure if this can be done (reasonably easily) with libxml2. > - resolution seemingly going ahead (successive calls to the resolver methods without error - see example test #1 in the linked logging), but really it's not At least the duplicate calls to the resolver seem ... unexpected. I don't see why that would happen. I sense some room for improvements here... :) Stefan _________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml(a)lxml.de https://mailman-mail5.webfaction.com/listinfo/lxml ________________________________ This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ________________________________ ________________________________ This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ________________________________

1 0

lxml 4.2.3 Python wheel for 3.7?
by cataco＠keemail.me 05 Jul '18

05 Jul '18

Hey.. I just discovered that I cannot install lxml with pip on my OS (Windows 10 x64 v1803, Python 3.7.0 x64).. And checking on PyPI, the list of files (https://pypi.org/project/lxml/#file <https://pypi.org/project/lxml/#files>s) seems to confirm this. There is no Wheel (yet) for cp37 as win32 or win_amd64.. (I assume cp37 is CPython 3.7.0..?) Is there something preventing support for this specific platform? Or is it just not done yet? ;-) Kind Regards R.

2 1