Mailman 3 November 2008 - lxml - The Python XML Toolkit

[lxml-dev] Avoiding DTD default attributes in XSLT
by Chuck Bearden 15 Nov '08

15 Nov '08

xsltproc offers the '--nodtdattr' option to cause libxslt not to add default attributes from the DTD to the output document when they aren't present in the source document. How can I achieve the same effect when doing XSLT with lxml? Thanks in advance for your help! Chuck

2 1

[lxml-dev] seg. fault using _Comment
by Frédéric Mantegazza 12 Nov '08

12 Nov '08

Hello, I'm trying to use comments in my xml file. The goal is to be able to enable/disable some entries in a config. file. Parsing comments works fine; when getting a node, I check if it is a _Comment object. If so, I just parse the string contained in the comment. Now, I would like to do the reverse operation: save some entries as comments. But it fails. I instanciate a _Comment object, but as soon as I use it, the application crashed with seg. fault. Is it a known bug? I'm using the 1.1.1 version. Let me know if it can be solved by upgrading. Thanks, -- Frédéric

2 2

[lxml-dev] NotImplementedError for external functions with xsl:variable's
by Dmitri Fedoruk 11 Nov '08

11 Nov '08

Greetings, We've been using lxml for almost a year now. Recently we were stuck with XSLT functionality and eventually started to use external functions, which are pretty useful in many cases. Of course, this makes the template unportable, but we deal with this. So, we register the functions in our namespace for the transformer and use them like this: <?xml version="1.0" encoding="utf-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fe="http://rambler.ru/fe" extension-element-prefixes="fe"> <xsl:template match="/"> query=<xsl:value-of select="fe:quote($query)"/> </xsl:tempalte> Here $query is the external parameter passed to the transformer. Works fine. But when we slightly modify the template and want to use not the external parameter, but xsl:variable, we fail: <xsl:template match="/"> <xsl:variable name="query1">asdfadsfadsf</xsl:variable> query=<xsl:value-of select="fe:quote($query1)"/> </xsl:tempalte> We have lxml 2.0.1 in production and 2.1 on the development machine. The problem occurs, but in slightly different manner, in both situations. 2.0.1 just performs nothing and returns an empty string; 2.1 raises File "xslt.pxi", line 515, in lxml.etree.XSLT.__call__ (src/lxml/lxml.etree.c:90526) File "lxml.etree.pyx", line 233, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:4916) File "extensions.pxi", line 665, in lxml.etree._extension_function_call (src/lxml/lxml.etree.c:82995) File "extensions.pxi", line 513, in lxml.etree._unwrapXPathObject (src/lxml/lxml.etree.c:81828) NotImplementedError As far as I can understand, this situation is handled explicitly in 2.1 and there are reasons for that. So, may I ask if it is implemented sometime later? Cheers, Dmitri

2 4

[lxml-dev] Fwd: Re: lxml Mac installation idea
by Michael Guntsche 09 Nov '08

09 Nov '08

CC'ing lxml-dev -------- Original Message -------- Subject: Re: [lxml-dev] lxml Mac installation idea Date: Sun, 9 Nov 2008 23:12:32 +0100 From: Guntsche Michael <mike(a)it-loops.com> To: Stefan Behnel <stefan_ml(a)behnel.de> On Nov 9, 2008, at 22:59, Stefan Behnel wrote: > Hi, > > > On a normal system, that shouldn't pose a problem at all. But if > lxml.etree > reports that the runtime linker linked it with the correct, up-to-date > library, and then it crashes due to a problem in the outdated system > library, > which should not even have been loaded, I just can't call that the > normal > behaviour of an "operating" system. > I never said that MacosX is normal. :) It is really botched in some places, especially in the library department. > So you mean that you are building lxml manually and you do not use the > macports build, right? Is there anything you have to do at runtime, > or does it > "just work" for you once it's built and installed? Exactly. I just made sure that every related library that comes from macports, in this case libxml, libexslt and libiconv are build as universal binary. The latest builds I have are running through a "make test" without any problems. They even survive the test-suite you posted a few months ago. I still do not understand why the linker picks up the internal libraries during runtime. Maybe if someone with a "broken" setup could give me more information about it (universal binaries, self-compiled from macports etc..) I can figure out what's going on. Kind regards, Michael

1 0

[lxml-dev] setupinfo.py that downloads/builds libxml2/libxslt
by Ian Bicking 07 Nov '08

07 Nov '08

I've attached a version of setupinfo.py that will download and build libxml2 and libxslt. There are some problems with the script, but hopefully this is a start. If you give the build option --build-libxml2xslt then it will download the packages and build them, and set the options to use the xml2-config and xslt-config scripts. I also made it always use --static if you do this. I'm not sure if that's a good idea, but if they aren't built statically then I'm not sure where to install the libraries... though I guess they could be installed in some subdirectory of the lxml package? That would be fine, unless there were problems if those libraries were then moved around (as they are often during the installation process), it might mean that the linker won't find them...? Also adding files to the installation process means some more work. (It just occurred to me... do I have to run "make; make install" for those libraries if it is a static build, or would ./configure be sufficient?) The script will download the newest versions of these libraries, or you can give a specific version with --libxml2-version/--libxslt-version. Also BUILD_LIBXML2XSLT=true, LIBXML2_VERSION, and LIBXSLT_VERSION environmental variables work for these settings. Maybe there can also be a fallback to archives distributed directly with lxml? The tarballs are about 8Mb for the two of them. Or I suppose just some more flags to point to existing source code or tarballs. The biggest problem is that I'm not sure at all where to put this stuff in the setup.py build process. Right now the options to setup() include information about the built versions of these libraries, but simply calling "python setup.py" should not cause those libraries to be built. (Right now if you use --build-libxml2xslt it will build those libraries at setupinfo/setup.py import time.) This is the biggest problem I see with this at the moment. Lastly, I'm having problems building lxml trunk right now, with this error: > cythoning src/lxml/lxml.etree.pyx to src/lxml/lxml.etree.c > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > cdef readonly object localname > cdef readonly object namespace > def __init__(self, text_or_uri_or_element, tag=None): > if not _isString(text_or_uri_or_element): > if isinstance(text_or_uri_or_element, _Element): > text_or_uri_or_element = (<_Element>text_or_uri_or_element).tag > ^ > ------------------------------------------------------------ > > /home/ianb/src/lxml/src/lxml/lxml.etree.pyx:259:51: Declarator should be empty But I got this working with the last lxml release, so that's fine. -- Ian Bicking : ianb(a)colorstudy.com : http://blog.ianbicking.org

3 7

[lxml-dev] lxml XSD Schema validation problem
by Richard Smith 06 Nov '08

06 Nov '08

Hi guys, I'm getting an error using the lxml objectification API and XSD Validation. Am using lxml and from distribution on standard hardy lib versions. Here's the boilerplate version info from the FAQ: lxml.etree: (2, 1, 2, 0) libxml used: (2, 6, 31) libxml compiled: (2, 6, 31) libxslt used: (1, 1, 22) libxslt compiled: (1, 1, 22) The code I'm testing with is as follows: ############################ >8 ############################ #!/usr/bin/python from lxml import objectify from lxml import etree SCHEMA_FILE = 'nvdcve.xsd' DATA_FILE = 'nvdcve-recent.xml' schema_file = open(SCHEMA_FILE) data_file = open(DATA_FILE) schema = etree.XMLSchema(file=schema_file) parser = objectify.makeparser(schema=schema) tree = objectify.parse(data_file, parser) ############################ 8< ############################ The XSD I'm validating against is the XSD provided by NIST for CVE posts and the XML doc is the recent NVDs: http://nvd.nist.gov/schema/nvdcve.xsd http://nvd.nist.gov/download/nvdcve-recent.xml The error I'm getting from lxml is: lxml.etree.XMLSyntaxError: Element '{http://nvd.nist.gov/feeds/cve/1.2}ref', attribute 'url': [facet 'pattern'] The value 'http://www.securitytracker.com/id?1021107' is not accepted by the pattern '(news|(ht|f)tp(s)?)://.+'. The problem is that on every implementation of regex patterns I can think of (Python, Perl, egrep etc) the regex validates... Indeed on my win32 xml editors (LiquidXML and OxygenXML) the data validates with no problems. $ egrep "(news|(ht|f)tp(s)?)://.+" nvdcve-recent.xml | wc -l 497 pcretest comes back fine too: $ pcretest PCRE version 7.4 2007-09-21 re> #(news|(ht|f)tp(s)?)://.+# data> http://www.securitytracker.com/id?1021107 0: http://www.securitytracker.com/id?1021107 1: http 2: ht Is this a libxml2 problem, my code or, dare I say, an evil bug? Thanks -- Richard

2 2

[lxml-dev] lxml Mac installation idea
by Ian Bicking 04 Nov '08

04 Nov '08

So... I hear that lxml installs better on a Mac if it's built along with libxml2/libxslt. That's not what everyone would do, so I was unclear how to enable something like that, and if setup.py would be the right place. A number of attempts to get stuff setup have been tried before external to lxml (like buildouts and now staticlxml). While it's kind of lame, I wonder if enabling this static installation via an environmental variable would be reasonable? It would be easier to apply in a number of circumstances. I imagine it would mean something like, on installation, if a variable like LXML_INSTALL_STATIC (or INSTALL_LIBXML2 or something) was set, it'd download the libxml2 and libxslt libraries, run configure/make/make install with a prefix inside the lxml source directory itself, then build lxml using that library. Another option would be simply a different tarball that contains the libxml2/libxslt source, and its setup.py would always build those. It could be versioned like 2.1static or something, which should keep it from being implicitly used by easy_install, etc. (since 2.1static is considered an earlier version than 2.1). This might be more reasonable? staticlxml is kind of weird, because installing staticlxml installs lxml, which can confuse tools. Maybe the two versions could be arranged with some svn:externals, or just as a build script of some sort (e.g., drop a marker file in the source to make it "static", and have setup.py look for that file and change the sdist/install commands appropriately). One nuisance with any attempt to fix this is that I don't think either myself nor Stefan have ready access to a Mac to test this stuff... are there any Macs we can ssh into for testing? (Obviously patches are also welcome, but this Mac thing has caused so many support problems for me that I really want to get it resolved.) -- Ian Bicking : ianb(a)colorstudy.com : http://blog.ianbicking.org

5 8

Re: [lxml-dev] [XML-SIG] Messages when installing xml
by Stefan Behnel 03 Nov '08

03 Nov '08

Hi, first of all: is there any reason you are not using the latest binary packages of lxml 2.1? Also: the mailing list of lxml is a better place to ask these questions than the more general XML-SIG list. Colin J. Williams wrote: > The log of my install is below: > > Building lxml version 2.2.alpha1-59220. > Building with Cython 0.9.8.1.1. > ERROR: 'xslt-config' is not recognized as an internal or external command, > operable program or batch file. > > ** make sure the development packages of libxml2 and libxslt are installed ** You need to install libxml2 and libxslt and add the xslt-config script that ships with libxslt to your program search path when building lxml. > [...] > building 'lxml.etree' extension > C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\python25\include > -IC:\python25\PC -c src/lxml\lxml.etree.c -o > build\temp.win32-2.5\Release\src\lxml\lxml.etree.o -w It's generally recommended to build lxml statically against libxml2/libxslt on Windows. There are some build instructions on the web page. Stefan

1 0