[Python-Dev] very bad network performance

[Curt Hagenlocher]

On Mon, Apr 14, 2008 at 4:19 PM, Guido van Rossum <guido at python.org> wrote:
>
> But why was imaplib apparently specifying 10MB? Did it know there was
> that much data? Or did it just not want to bother looping over all the
> data in smaller buffer increments (e.g. 64K, which is probably the max
> of what most TCP stacks will give you)?

I'm going to guess that the code in question is

                size = int(self.mo.group('size'))
                if __debug__:
                    if self.debug >= 4:
                        self._mesg('read literal size %s' % size)
                data = self.read(size)

It's reading however many bytes are reported by the server as the size.

> If I'm right with my hunch that the TCP stack will probably clamp at
> 64K, perhaps we should use min(system limit, max(requested size,
> buffer size))?

I have indeed missed the point of the read buffer size.  This would work.

--
Curt Hagenlocher
curt at hagenlocher.org