[Python-3000] PEP 3137 plan of attack

[Guido van Rossum]

On 10/8/07, Brett Cannon <brett at python.org> wrote:
> On 10/8/07, Guido van Rossum <guido at python.org> wrote:
> > On 10/8/07, Brett Cannon <brett at python.org> wrote:
> > > On 10/7/07, Guido van Rossum <guido at python.org> wrote:
> > > [SNIP]
> > > > PS. Is there anyone who understands test_urllib2net and can fix it?
> > > > It's been failing for weeks (maybe months) now.
> > >
> > > I don't understand it but I fixed it in r58378.  =)
> > >
> > > When ftplib.FTP was converted over to Py3K it was given a default
> > > encoding of ASCII on all read data, but that doesn't work as the stuff
> > > on the other end could be latin1 (and it was).  So I just changed the
> > > default encoding.
> >
> > Cool. Though how do you know it was really latin1? Is there anything
> > standardized about the encoding used by FTP?
>
> See, now I had to go and look stuff up.  So much work for a holiday.  =)
>
> According to the spec, data transfers can be anything based on data
> transfer format specified.  ASCII is one of them, but so is Local
> which can be anything.
>
> Turns out that ftplib.FTP.connect() reads from the socket using
> socket.makefile('r', encoding), so it starts off in text mode.  So
> that makes restricting the encoding to bytes < 128 a bad thing as not
> all possible data transfers would be legal.
>
> Basically it sounds like the ftplib module might need a thorough
> rewrite to use bytes/buffers so that the proper decoding happens at
> the last second.  But I am not the person to do that rewrite.  =)

Thanks. Mind filing a bug for someone to find? It sounds like the
rewrite might be easier once we have immutable bytes. (So this
conversation is not entirely off-topic for this thread. ;-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)