<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Sep 16, 2010, at 7:34 PM, Barry Warsaw wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On Sep 16, 2010, at 06:11 PM, Glyph Lefkowitz wrote:<br><br><blockquote type="cite">That may be a handy way to deal with some grotty internal<br></blockquote><blockquote type="cite">implementation details, but having a 'decode()' method is broken. The<br></blockquote><blockquote type="cite">thing I care about, as a consumer of this API, is that there is a<br></blockquote><blockquote type="cite">clearly defined "Message" interface, which gives me a uniform-looking<br></blockquote><blockquote type="cite">place where I can ask for either characters (if I'm displaying them to<br></blockquote><blockquote type="cite">the user) or bytes (if I'm putting them on the wire). I don't<br></blockquote><blockquote type="cite">particularly care where those bytes came from. I don't care what<br></blockquote><blockquote type="cite">decoding tricks were necessary to produce the characters.<br></blockquote><br>But first you have to get to that Message interface. This is why the current<br>email package separates parsing and generating from the representation model.<br>You could conceivably have a parser that rot13's all the payload, or just<br>parses the headers and leaves the payload as a blob of bytes. But the parser<br>tries to be lenient in what it accepts, so that one bad header doesn't cause<br>it to just punt on everything that follows. Instead, it parses what it can<br>and registers a defect on that header, which the application can then reason<br>about, because it has a Message object. If it were to just throw up its hands<br>(i.e. raise an exception), you'd basically be left with a blob of useless crap<br>that will just get /dev/null'd.<font class="Apple-style-span" color="#000000"><font class="Apple-style-span" color="#144FAE"><br></font></font></div></blockquote><div><br></div><div>Oh, absolutely. Please don't interpret anything I say as meaning that the email API should not handle broken data. I'm just saying that you should not expect broken data to round-trip through translation to characters and back, any more than you should expect a broken PNG to round-trip through a translation to a 2d array of pixels and back.</div><br><blockquote type="cite"><div><blockquote type="cite">Now, it may be worthwhile to have specific normalization /<br></blockquote><blockquote type="cite">debrokenifying methods which deal with specific types of corrupt data<br></blockquote><blockquote type="cite">from the wire; encoding-guessing, replacement-character insertion or<br></blockquote><blockquote type="cite">whatever else are fine things to try. It may also be helpful to keep<br></blockquote><blockquote type="cite">around a list of errors in the message, for inspection. But as we<br></blockquote><blockquote type="cite">know, there are lots of ways that MIME data can go bad other than<br></blockquote><blockquote type="cite">encoding, so that's just one variety of error that we might want to<br></blockquote><blockquote type="cite">keep around.<br></blockquote><br>Right. The middle ground IMO is what the current parser does. It recognizes<br>the problem, registers a defect, and tries to recover, but it doesn't fix the<br>corrupt data. So for example, if you had a valid RFC 2047 encoded Subject but<br>a broken X-Foo header, you'd at least still end up with a Message object. The<br>value of the good headers would be things from which you can get the unicode<br>value, the raw bytes value, parse its parameters, munge it, etc. while the bad<br>header might be something you can only get the raw bytes from.<br></div></blockquote></div><div><br></div>My take on this would be that you should always be able to get bytes or characters, but characters are always suspect, in that once you've decoded, if you had invalid bytes, then they're replacement characters (or your choice of encoding fix).</body></html>