On Fri, Jan 22, 2016 at 1:40 PM, Andrew Barnert
On Jan 22, 2016, at 13:11, Guido van Rossum
wrote: Interesting. PEP 484 defines an IO generic class, so you can write IO[str] or IO[bytes]. Maybe introducing separate helper functions that open files in text or binary mode can complement this to get a solution?
The runtime types are a little weird here as well.
In 3.x, open returns different types depending on the value, rather than the type, of its inputs. Also, TextIOBase is a subclass of IOBase, even though it isn't a subtype in the LSP sense, so you have to test isinstance(IOBase) and not isinstance(TextIOBase) to know that read() is going to return bytes. That's all a little wonky, but not impossible to deal with.
Agreed. At this level it's really hard to fix. :-(
In 2.x, most file-like objects--including file itself, which open returns--don't satisfy either ABC, and most of them can return either type from read.
Well, the type returned by the builtin open() never returns Unicode. For duck types (and even StringIO) it's indeed a crapshoot. :-(
Having a different function for open-binary instead of a mode flag would solve this, but it seems a little late to be adding that now. You'd have to go through all your 2.x code and change every open to one of the two new functions just to statically type your code, and then change it again for 3.x. Plus, you'd need to do the same thing not just for the builtin open, but for every library that provides an open-like method.
Yeah, painful. Though in most cases you can also patch up individual calls using cast(IO[str], open(...)) etc.
Maybe this special case is special enough that static type checkers just have to deal with it specially? When the mode flag is a literal, process it; when it's forwarded from another function, it may be possible to get the type from there; otherwise, everything is just unicode|bytes and the type checker can't know any more unless you explicitly tell it (by annotating the variable the result of open is stored in).
That would be a lot of work too. We have so many other important-but-not-urgent things already that I would really like to push back on this until someone has actually tried the alternative and tells us how bad it is (like Ben Darnell did for @overload). -- --Guido van Rossum (python.org/~guido)