[Python-Dev] Python3 "complexity"

[Paul Moore]

On 9 January 2014 10:15, Kristján Valur Jónsson <kristjan at ccpgames.com> wrote:
> Also, the problem I'm describing has to do with real world stuff.
> This is the python 2 program:
> with open(fn1) as f1:
>     with open(fn2, 'w') as f2:
>         f2.write(process_text(f1.read())
>
> Moving to python 3, I found that this quickly caused problems.

You don't say what problems, but I assume encoding/decoding errors. So
the files apparently weren't in the system encoding. OK, at that point
I'd probably say to heck with it and use latin-1. Assuming I was sure
that (a) I'd never hit a non-ascii compatible file (e.g., UTF16) and
(b) I didn't have a decent means of knowing the encoding.

One thing that genuinely is difficult is that because disk files don't
have any out-of-band data defining their encoding, it *can* be hard to
know what encoding to use in an environment where more than one
encoding is common. But this isn't really a Python issue - as I say,
I've hit it with GNU tools, and I've had to explain the issue to
colleagues using Java on many occasions. The key difference is that
with grep, people blame the file, whereas with Python people blame the
language :-) (Of course, with Java, people expect this sort of problem
so they blame the perverseness of the universe as a whole... ;-))

Paul.