[Python-3000] PEP 3137 plan of attack

[Brett Cannon]

On 10/8/07, Guido van Rossum <guido at python.org> wrote:
> On 10/8/07, Brett Cannon <brett at python.org> wrote:
> > On 10/7/07, Guido van Rossum <guido at python.org> wrote:
> > [SNIP]
> > > PS. Is there anyone who understands test_urllib2net and can fix it?
> > > It's been failing for weeks (maybe months) now.
> >
> > I don't understand it but I fixed it in r58378.  =)
> >
> > When ftplib.FTP was converted over to Py3K it was given a default
> > encoding of ASCII on all read data, but that doesn't work as the stuff
> > on the other end could be latin1 (and it was).  So I just changed the
> > default encoding.
>
> Cool. Though how do you know it was really latin1? Is there anything
> standardized about the encoding used by FTP?

See, now I had to go and look stuff up.  So much work for a holiday.  =)

According to the spec, data transfers can be anything based on data
transfer format specified.  ASCII is one of them, but so is Local
which can be anything.

Turns out that ftplib.FTP.connect() reads from the socket using
socket.makefile('r', encoding), so it starts off in text mode.  So
that makes restricting the encoding to bytes < 128 a bad thing as not
all possible data transfers would be legal.

Basically it sounds like the ftplib module might need a thorough
rewrite to use bytes/buffers so that the proper decoding happens at
the last second.  But I am not the person to do that rewrite.  =)

-Brett