...

The main feature I'd like to see would be an easy installer that include lxml's dependencies, maybe using easy_install. It's a complex project and the installation is easy to get wrong.

Hmm, I'm not quite sure what could be done better here. What you'd have to do for 0.9 is:

* install libxml2 and libxslt (which lxml can't do for you)

Why not? Other python extensions install their dependencies.

* tar zxf lxml-0.9.tar.gz * cd lxml-0.9 * run "python setup.py install" (or bdist_egg or whatever you run normally)

That doesn't sound very error prone to me...

But then, that's mainly how it works on Linux. You seem to be on Apple, so I imagine what you're looking for is a readily installable darwin port? Maybe even without compilation?

OS X is my main platform, but I actually encountered trouble installing on Windows, where the steps were: 1. Read install instructions for lxml 2. Follow these to libxml site 3. Find download link on crowded page 4. Find download link to Windows binaries 5. Read install instructions and figure out *which* libraries are needed out of: libxml2, libxslt, openssl, iconv, zlib, xmlsec, and xsldbg) and in what order. Appear to need four libraries: openssl, iconv, libxml2, and libxslt in that order. 6. Download and unzip each set of binaries. 7. Get the exes in my %PATH% 8. Collect the headers in ~/include/ and the DLLs in ~/lib/ and point the config file at them 9. run python setup.py install 10. have it fail 11. Run out of time to futz with it and give up

I guess then we'd have to find someone who uses MacOS-X and can provide a port...

On OS X it isn't nearly such a problem, more of an inconvenience. If necessary I can make an egg at some point for OS X.

Stefan

--Dethe Choosing software is not a neutral act. It must be done consciously; the debate over free and proprietary software can’t be limited to the differences in the applications’ features and ergonomics. To choose an operating system, or software, or network architecture is to choose a kind of society. --Lemaire and Decroocq (trans. by Tim Bray)

Dethe Elza wrote:

...

...

The main feature I'd like to see would be an easy installer that include lxml's dependencies, maybe using easy_install. It's a complex project and the installation is easy to get wrong.

Hmm, I'm not quite sure what could be done better here. What you'd have to do for 0.9 is:

* install libxml2 and libxslt (which lxml can't do for you)

Why not? Other python extensions install their dependencies.

Stephan already answered this, but I'll jump in -- I actually looked for a bit at setuptools last year in the hope it could tackle this problem, but there wasn't much that could help there... So, yeah, we're aware that this can be a problem, but it's not easy to fix unfortunately. Contributions in that department are of course very welcome. Regards, Martijn

Paul Everitt wrote:

Before writing and checking in a test, I want to find out if something is known behavior. I have an XSLT that uses XML data in the stylesheet itself. Essentially:

---- xslt preamble stuff ---- <localstuff> <localitem>klkdk</localitem> </localstuff> .....later....

It works with xsltproc but doesn't work with lxml. I wonder if there's something about the resolver that is the issue.

Should I file a case for this, or is this known behavior?

Hmmm, I can reproduce that. This results in an empty root tag 'test': --------------- from lxml import etree xslt = etree.XSLT(etree.XML('''\ http://www.w3.org/1999/XSL/Transform"> <test> </test> ''')) print xslt(etree.XML('<a/>')), --------------- Looks like libxslt doesn't know how to access document('') without an XSLT specific entity loader. You can run xsltproc with the '--load-trace' option to see that the entity solver handles that access, too. According to strace, it even reads in the file twice, so we'd either have to provide the XSLT processor with access to the original XSL file (which is impossible if it is generated in memory) or write a replacement to the entity resolver that handles that special case. I personally consider that a bug in libxslt, though. document('') should be a special case /in the library/.

Also, I'd file a case about not wiring in libxml2's HTMLParser, but I suppose that's not a bug, it's just a missing feature. :^) I *really* need that wired in at some point, but for now, I can just ship some other tool that converts HTML -> well-formed XML.

Would be nice to have - but it's a little too close to 0.9 to implement the interface code, which also has to be ElementTree compatible to a certain extent. Stefan

On 2 Mar 2006 at 16:37, Paul Everitt wrote:

Stefan Behnel wrote:

...

Paul Everitt wrote:

...

Before writing and checking in a test, I want to find out if something is known behavior. I have an XSLT that uses XML data in the stylesheet itself. Essentially:

...

...

Sorry to barge in here.. I've been watching lxml for a while, waiting for it to support a custom resolver so I can use it in Paste and Zope, etc. Currently I'm using libxslt with Paste in my "tal2xslt" project. I rely on document('') as a way to inject "system-wide" constants into the .xsl files that are generated from the input TAL. I register a global resolver with libxslt and tag the URI associated with a given document so it can be tied back to the originating request. I use libxml2.readDoc() to associate a custom URI with each document. Each URI I load uses a custom scheme so I know it's "mine". Anyway, in the resolver if the scheme doesn't match my fake scheme, I just hand it back to libxslt and let it resolve itself. Can you do the same in lxml? I poked around http://codespeak.net/svn/lxml/trunk/src/lxml/ but couldn't find the module where you set the resolver.

Ok, point made. :^) I'm currently working around this by putting the nodes I want into the input document. However, that makes me pay a penalty on every request, instead of just once when the stylesheet is generated.

I can live with it, though. It's already lots faster than other templating approaches. :^)

Yeah, I love it. Edge side includes done via xml/xsl in the client. It's great. -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com AOL-IM or SKYPE: BKClements

On 2 Mar 2006 at 22:16, Stefan Behnel wrote:

...

I poked around http://codespeak.net/svn/lxml/trunk/src/lxml/ but couldn't find the module where you set the resolver.

That's because we don't. :)

Great, at least I know I'm not blind now.

We currently do not care about resolving at all. Everything that works does so because libxml2/xslt handles it itself. Everything that doesn't work - uhm, well, doesn't work.

In that case, I wonder about the original poster's claim that document('') doesn't work. I use xsltproc and xml starlet all the time, I don't think xsltproc sets a resolver at all, but I haven't looked at the source to be certain. Anyway, document('') works in both xsltproc and xml starlet.

ones. We would especially need to see at what granularity you can set them: at a function-local level, at an object level or at a global level?

The last I knew, libxslt only allowed setting the resolver on a per-process level. Also, I am using the Python level interface, not the C level interface. I'll paste below the current hacky code I use. One note, I'm not using the most recent libxslt. In the version of libxslt I'm using now, if a resolver raises an exception, libxslt treats that to mean "continue with default resolution". apparently the semantics have changed in more recent libxslt versions.. Since I don't want default resolution to occur when an exception happens, I return an empty string in that case. I want to point out that this code isn't really correct in that respect. This code is still in development. _resolver_context = {} _scheme = "memory" _resolver_lock = RLock() def _Resolver(URL,ID,ctxt): try: context, new_uri = extract_uri_and_context(URL) if not context: prefix = URL.split(':')[0] if prefix == _scheme: print "no context for %r" % URL return None resolver = get_resolver(context) if not resolver: print "Could not find resolver for context %r, url %r" % (context, URL) return '' return StringIO.StringIO(resolver(new_uri)) except: print "unexpected exception in resolver" traceback.print_exc() return '' def add_context_to_uri(uri, context): """Mangle the uri, adding a fake scheme and context""" return urlparse.urlunsplit((_scheme, context, uri, '', '')) def extract_uri_and_context(uri): """Un-mangle uri, returns (context, uri) or (None, None) if not our scheme""" parts = urlparse.urlsplit(unquote(uri)) context, new_uri = (None, uri) if parts[0] == _scheme: if parts[1]: context = parts[1] new_uri = parts[2] elif parts[2][:2] == '//': # buggy urlsplit sticks netloc in path context, new_uri = parts[2][2:].split('/',1) new_uri = "/" + new_uri return (context, new_uri) def register_resolver(context, resolver): """add this context to list of resolvers""" _resolver_lock.acquire() try: obj = get_resolver(context) if not obj: _resolver_context[context] = (resolver, 1) else: old_resolver, use_count = obj _resolver_context[context] = (resolver, use_count+1) finally: _resolver_lock.release() def unregister_resolver(context): """remove this context from the list of resolvers""" _resolver_lock.acquire() try: obj = get_resolver(context) if not obj: raise ValueError("resolver context %r not currently registered" % context) old_resolver, use_count = obj use_count -= 1 if use_count > 0: _resolver_context[context] = (old_resolver, use_count) else: del _resolver_context[context] finally: _resolver_lock.release() def get_resolver(context): """Return a resolver for the specified context""" _resolver_lock.acquire() try: obj = _resolver_context.get(context) if not obj: return obj resolver, use_count = obj return resolver finally: _resolver_lock.release() libxml2.setEntityLoader(_Resolver) # this is a process-wide change, -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com AOL-IM or SKYPE: BKClements

Paul Everitt wrote:

Stefan Behnel wrote:

...

from lxml import etree

xslt = etree.XSLT(etree.XML('''\ http://www.w3.org/1999/XSL/Transform">

<test> </test> '''))

print xslt(etree.XML('<a/>')), ---------------

Looks like libxslt doesn't know how to access document('') without an XSLT specific entity loader.

Ok, I played with it a bit and found that we can work around this in the case where the XSLT is read in from a file (which should be the majority of cases, I'd say). All we have to do is use the file parser functions from libxml2 in that case, which store the file URL in the document structure. This breaks the case where the document is modified in between (since the changes are not reflected by the file when it is re-read by libxslt), but that is a) a rare case and b) a bug in libxslt, which should recognise the case where the stylesheet itself is referenced. So, I committed the change to the trunk (revision 23950). Please try it to see if it fixes the cases you needed. Stefan