[Python-Dev] PEP 383 (again)

[Stephen J. Turnbull]

Thomas Breuel writes:

 > PEP 383 doesn't make it any easier; it just turns one set of
 > problems into another.

That's false.  There is an interesting class of problems of the form
"get a list of names from the OS and allow the user to select from it,
and retrieve corresponding content."  People are *very* often able to
decode complete gibberish, as long as it's the only gibberish in a
list.  Ditto partial gibberish.  In that case, PEP 383 allows the
content retrieval operation to complete.

There are probably other problems that this PEP solves.

 > Actually, it makes it worse,

Again, it gives you different problems, which may be better and may be
worse according to the user's requirements.  Currently, you often get
an exception, and running the program again is no help.  The user must
clean up the list to make progress.  This may or may not be within the
user's capacity (eg, read-only media).

 > since any problems that show up now show up far from the source of
 > the problem, and since it can lead to security problems and/or data
 > loss.

Yes.  This is a point I have been at pains to argue elsewhere in this
thread.  However, it is "mission creep": Martin didn't volunteer to
write a PEP for it, he volunteered to write a PEP to solve the
"roundtrip the value of os.listdir()" problem.  And he succeeded, up
to some minor details.

 > The problem may well be with the program using the wrong encodings or
 > incorrectly ignoring encoding information.  Furthermore, even if it is user
 > error, the program needs to validate its inputs and put up a meaningful
 > error message, not mangle the disk.  To detect such program bugs, it's
 > important that when Python detects an incorrect encoding that it doesn't
 > quietly continue with an incorrect string.

I agree.  Guido, however, responded that "Practicality beats purity"
to a similar point in the PEP 263 discussion.

Be aware that you're fighting an uphill battle here.