[Python-Dev] a suggestion ... Re: PEP 383 (again)

Glenn Linderman v+python at g.nevcal.com
Thu Apr 30 09:38:29 CEST 2009

On approximately 4/29/2009 10:17 PM, came the following characters from 
the keyboard of Martin v. Löwis:
>> I don't understand the proposal and issues. I see a lot of people 
>> claiming that they do, and then spending all their time either 
>> talking past each other, or disagreeing. If everyone who claims they 
>> understand the issues actually does, why is it so hard to reach a 
>> consensus?
> Because the problem is difficult, and any solution has trade-offs.
> People disagree on which trade-offs are worse than others.
>> I'd like to see some real examples of how things can break in the 
>> current system
> Suppose I create a new directory, and run the following script
> in 3.x:
> py> open("x","w").close()
> py> open(b"\xff","w").close()
> py> os.listdir(".")
> ['x']


py> os.listdir(b".")
['x', '\xff']

> If I quit Python, I can now do
> martin at mira:~/work/3k/t$ ls
> ?  x
> martin at mira:~/work/3k/t$ ls -b
> \377  x
> As you can see, there are two files in the current directory, but
> only one of them is reported by os.listdir. The same happens to
> command line arguments and environment variables: Python might swallow
> some of them.

There is presently no solution for command line and environment 
variables, I guess... which adds some amount of urgency to the 
implementation of _something_, even if not this PEP.

>> and I'd like any potential solution to be made 
>> available as a third-party package before it goes into the standard 
>> library (if possible).
> Unfortunately, at least for my solution, this isn't possible. I need
> to change the implementation of the existing file IO APIs.

Other than initializing them to use UTF-8b instead of UTF-8, and to use 
the new python-escape handler?  I'm sure if I read the code for that, 
I'd be able to figure out the answer...  I don't find any documented way 
of adding an encoding/decoding handler to the file IO encoding 
technique, though which lends credence to your statement, but then that 
could also be an oversight on my part.

One could envision a staged implementation: the addition of the ability 
to add encoding/decoding handlers to the file IO encoding/decoding 
process, and the external selection of your new python-escape handler 
during application startup.  That way, the hooks would be in the file 
system to allow your solution to be used, but not require that it be 
used; competing solutions using similar technology could be implemented 
and evaluated.

Glenn -- http://nevcal.com/
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

More information about the Python-Dev mailing list