Htmldiff stripping newlines from pre tags
Hello, I'm using lxml's htmldiff function to (surprise) diff two HTML snippets. However I have found that if the snippet includes a <pre> tag then any newlines are stripped from it. For example:
html = "<pre>test\ntest2\ntest3</pre>" print repr(htmldiff(html, html)) u'<pre>test test2 test3</pre>'
Is there a reason for this, or better yet is there a way I can stop this from occurring? I was under the impression that inside a pre tag newlines and whitespace are handled differently and shouldn't be stripped like you can do with other HTML tags. Tom
Le 26/07/2013 14:21, Tom . a écrit :
Hello, I'm using lxml's htmldiff function to (surprise) diff two HTML snippets. However I have found that if the snippet includes a <pre> tag then any newlines are stripped from it. For example:
html = "<pre>test\ntest2\ntest3</pre>" print repr(htmldiff(html, html)) u'<pre>test test2 test3</pre>'
Is there a reason for this, or better yet is there a way I can stop this from occurring? I was under the impression that inside a pre tag newlines and whitespace are handled differently and shouldn't be stripped like you can do with other HTML tags.
IMO htmldiff shouldn’t touch whitespace anywhere. The whitespace handling in pre elements is actually a CSS property that can be applied to any element. -- Simon Sapin
Well perhaps I am going crazy (or making a silly mistake?), but I tested this on my Linux box with Lxml version 3.0.0 through to the latest sources on github with the same results. Here is the test script I used: from lxml.html.diff import htmldiff html = u"""<pre>test test2 test3</pre>""" result = htmldiff(html, html) assert "\n" in result, "Newline not found in %s" % result Inside htmldiff it calls tokenize() on both inputs, which appears to loose all notion of whitespace:
tokenize("""<pre>test ... test2 ... test3 ... """) [token(u'test', ['<pre>'], []), token(u'test2', [], []), token(u'test3', [], ['</pre>'])]
It then calls htmldiff_tokens, which produces output like this: ['<pre>', u'test ', u'test2 ', u'test3', '</pre>'] It then joins those with an empty string, which results in a loss of whitespace. On 26 July 2013 14:49, Simon Sapin <simon.sapin@exyr.org> wrote:
Le 26/07/2013 14:21, Tom . a écrit :
Hello,
I'm using lxml's htmldiff function to (surprise) diff two HTML snippets. However I have found that if the snippet includes a <pre> tag then any newlines are stripped from it. For example:
html = "<pre>test\ntest2\ntest3</pre>**" print repr(htmldiff(html, html)) u'<pre>test test2 test3</pre>'
Is there a reason for this, or better yet is there a way I can stop this from occurring? I was under the impression that inside a pre tag newlines and whitespace are handled differently and shouldn't be stripped like you can do with other HTML tags.
IMO htmldiff shouldn’t touch whitespace anywhere. The whitespace handling in pre elements is actually a CSS property that can be applied to any element.
-- Simon Sapin ______________________________**______________________________**_____ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de https://mailman-mail5.**webfaction.com/listinfo/lxml<https://mailman-mail5.webfaction.com/listinfo/lxml>
Le 26/07/2013 15:44, Tom . a écrit :
Well perhaps I am going crazy (or making a silly mistake?), but I tested this on my Linux box with Lxml version 3.0.0 through to the latest sources on github with the same results. [...]
I’m not doubting your results :) I agree that it’s a bug that should be fixed. I was also saying that the <pre> should not be treated specially, but it doesn’t appear to be. -- Simon Sapin
Tom ., 26.07.2013 15:21:
I'm using lxml's htmldiff function to (surprise) diff two HTML snippets. However I have found that if the snippet includes a <pre> tag then any newlines are stripped from it. For example:
html = "<pre>test\ntest2\ntest3</pre>" print repr(htmldiff(html, html)) u'<pre>test test2 test3</pre>'
Is there a reason for this, or better yet is there a way I can stop this from occurring? I was under the impression that inside a pre tag newlines and whitespace are handled differently and shouldn't be stripped like you can do with other HTML tags.
Thanks for your patch, I merged it into current master. https://github.com/lxml/lxml/pull/124 I think the change is a good thing, but playing with it a bit, it doesn't really feel completely right yet. Consider this example: >>> print(htmldiff('<p> first\nsecond\nthird</p>', ... '<p> first\n second\nthird </p>')) <p>first second third </p> It still drops the whitespace at the beginning, but not at the end. It also seems to copy the content from the second argument, not the first. Not sure if that's good or bad, but it seems surprising. What do others think here? Stefan
I've modified my fork to preserve whitespace after a tag. its a bit hacky ( https://github.com/orf/lxml/blob/master/src/lxml/html/diff.py#L731) but it to works. I've sent a pull request with the changes on Github. Tom On 1 August 2013 05:29, Stefan Behnel <stefan_ml@behnel.de> wrote:
Tom ., 26.07.2013 15:21:
I'm using lxml's htmldiff function to (surprise) diff two HTML snippets. However I have found that if the snippet includes a <pre> tag then any newlines are stripped from it. For example:
html = "<pre>test\ntest2\ntest3</pre>" print repr(htmldiff(html, html)) u'<pre>test test2 test3</pre>'
Is there a reason for this, or better yet is there a way I can stop this from occurring? I was under the impression that inside a pre tag newlines and whitespace are handled differently and shouldn't be stripped like you can do with other HTML tags.
Thanks for your patch, I merged it into current master.
https://github.com/lxml/lxml/pull/124
I think the change is a good thing, but playing with it a bit, it doesn't really feel completely right yet. Consider this example:
>>> print(htmldiff('<p> first\nsecond\nthird</p>', ... '<p> first\n second\nthird </p>')) <p>first second third </p>
It still drops the whitespace at the beginning, but not at the end. It also seems to copy the content from the second argument, not the first. Not sure if that's good or bad, but it seems surprising.
What do others think here?
Stefan
_________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de https://mailman-mail5.webfaction.com/listinfo/lxml
participants (3)
-
Simon Sapin -
Stefan Behnel -
Tom .