<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Mar 25, 2016 at 12:00 AM, Andrey Vlasovskikh <span dir="ltr"><<a href="mailto:andrey.vlasovskikh@gmail.com" target="_blank">andrey.vlasovskikh@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Upon further investigation of the problem I've come up with an alternative idea that looks simpler and yet still capable of finding most text/binary conversion errors.<br></blockquote><div>... </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
## TL;DR<br>
<br>
* Introduce `typing.Text` for text data in Python 2+3<br>
* `bytes`, `str`, `unicode`, `typing.Text` in type hints mean whatever they<br>
 mean at runtime for Python 2 or 3<br>
* Allow `str -> unicode` and `unicode -> str` promotions for Python 2<br></blockquote><div><br></div><div>I'm against this, as it would seem to make str and unicode pretty much the same type in Python 2, and thus Python 2 mode seems much weaker than necessary. I wrote a more detailed reply in the mypy issue tracker (<a href="https://github.com/python/mypy/issues/1141#issuecomment-201799761">https://github.com/python/mypy/issues/1141#issuecomment-201799761</a>). I'm not copying it all here since much of that is somewhat mypy-specific and related to the rest of the discussion on that issue, but I'll summarize my main points here.</div><div><br></div><div>I prefer the idea of doing better type checking in Python 2 mode for str and unicode, though I suspect we need to implement a prototype to decide whether it will be practical.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
* Type checking for Python 2 *and* Python 3 actually finds most text/binary<br>
 errors<br></blockquote><div><br></div><div>This may be true, but I'm worried about usability for Python 2 code bases. Also, the effort needed to pass type checking in both modes (which is likely pretty close to the effort of a full Python 3 migration, if the entire code will be annotated) might be impractical for a large Python 2 code base.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
## Summary for authors of type checkers<br>
<br>
The semantics of types `bytes`, `str`, `unicode`, `typing.Text` and the type<br>
checking rules for them should match the *runtime behavior* of these types in<br>
Python 2 and Python 3 depending on Python 2 or 3 modes. Using the runtime<br>
semantics for the types is easy to understand while it still allows to catch<br>
most errors. The Python 2+3 compatibility mode is just a sum of Python 2 and<br>
Python 3 warnings.<br></blockquote><div><br></div><div>At least for mypy, the Python 2+3 compatibility mode would likely that twice as much CPU to run, which is a pretty high cost as type checking speed is one of the biggest open issues we have right now.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">## Runtime type compatibility<br></blockquote><div>... </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Each cell contains two characters: the result in Python 2 and in Python 3<br>
respectively. Abbreviations:<br></blockquote><div>... </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">* `*` — types are compatible, ignoring implicit ASCII conversions<br></blockquote><div><br></div><div>Am I reading this right if I understand this as "considered valid during type checking but may fail at runtime"?</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
For non-ASCII text literals passed to functions that expect `Text` or `str` in<br>
Python 2 a type checker can analyze the contents of the literal and show<br>
additional warnings based on this information. For non-ASCII data coming from<br>
sources other than literals this check would be more complicated.<br></blockquote><div><br></div><div>I wonder what would the check look like in the latter case? I can't imagine how this would work for non-literals.</div><div><br></div><div>Jukka</div></div></div></div>