Question regarding HTMLParser module.

[Carl Banks]

Adonis wrote:
> When parsing my html files, I use handle_pi to capture some embedded python
> code, but I have noticed that in the embedded python code if it contains
> html, HTMLParser will parse it as well, and thus causes an error when I exec
> the code, raises an EOL error. I have a work around for this as I use
> different set of characters rather that <tag> use something like (tag) then
> revert it back to <tag> via another function, I was wondering if there is a
> way to tell HTMLParser to ignore the embedded tags or another alternative?
> 
> Any help would be greatly appreciated.
> And another note, I am well aware of Zope, Webware, CherryPy, etc... for
> py/html embedding options, but I want this to be a learning experience.


Unfortunately, HTMLParser (and the similar sgmllib) miserably fail to
process inline text.  I know this very well; I have an HTML-generating
package that uses a lot of scripting and verbatim text.

What's happening in your case is that HTMLParser, when processing a <?
tag, simply and naively inputs text up to the next ">".  HTMLParser
thinks the > in <tt> closes your <? tag.  (It should at least have a
flag indicating whether it should read up to "?>" or just ">".)

A workaround is to do something like this:

<? print '<tt\x29monospaced</tt\x29' >

where obviously, \x29 is the hex code for >.  That's not quite as bad
as replacing characters, although it's still not perfect.

Another possibility is to use sgmllib, but that's probably way more
trouble than it's worth, and still far from perfect.  Basically,
sgmllib parsers have an method called verbatim, that turns of HTML tag
processing, although entities and closing tags are still processed.
(Entities and closing tags you can kind of reconstruct into the
original text, although the whitespace is lost.)  This is what I do in
my own HTML-generating package.

I'll probably contribute some badly-needed remedies to HTMLParser
sometime, as the limitations of it and sgmllib are starting to get on
my nerves.


-- 
CARL BANKS