[Python-Dev] Patch to telnetlib.py

[Guido van Rossum]

On Sat, Mar 13, 2010 at 9:24 AM, gregory dudek <dudek at cim.mcgill.ca> wrote:
> The Telnet module telnetlib.py can be
> very slow -- unusably slow -- for large automated data transfers.  There are typically done in raw mode.
>
> The attached patch greatly increased the speed of telnet interactions in raw mode.  I submitted this a couple of year ago, but it was for an older branch of python.

For which Python version is your new patch?

> There are 2 key things being done:
>  1) concatenations string with string.join instead of '+'  (which is probably a minor issue)

This looks like superstition in the patch below -- there is no reason
why "".join((a, b)) should be any faster than a+b. (The join trick is
important when joining *many* strings, but for two it doesn't make a
difference, both strings have to be copied at least once no matter
what you try.)

>  2) wholesale appending the raw and processed buffers when the IAC character is not found.  The should be examined
>     carefully since I am not an expert in the Telnet protocol, but it seems to work very well giving me a 5x speedup.

This makes sense and is likely the reason your patch is faster.

You should really submit this to the bug tracker at bugs.python.org.

> --- installed/telnetlib.py      2010-02-02 22:57:58.000000000 -0500
> +++ telnetlib.py        2010-03-13 12:17:02.000000000 -0500
> @@ -30,6 +30,7 @@
>  - timeout should be intrinsic to the connection object instead of an
>   option on one of the read calls only
>
> +Modified by G. Dudek for greater efficiency.

You're gonna need to submit a Python contributor agreement
(http://www.python.org/psf/contrib/contrib-form/) if/when your patch
gets accepted. Welcome to the club!

>  """
>
>
> @@ -420,6 +421,14 @@
>         """
>         buf = ['', '']
>         try:
> +            if self.rawq:
> +                if not IAC in self.rawq:
> +                    # speed hack, no IAC just grab whole queue. --Dudek
> +                    buf[self.sb] = "".join((buf[self.sb] , self.rawq))
> +                    self.cookedq = "".join((self.cookedq , buf[0] ))
> +                    self.sbdataq = "".join((self.sbdataq , buf[1] ))
> +                    self.rawq_flush()
> +                    return
>             while self.rawq:
>                 c = self.rawq_getchar()
>                 if not self.iacseq:
> @@ -428,7 +437,7 @@
>                     if c == "\021":
>                         continue
>                     if c != IAC:
> -                        buf[self.sb] = buf[self.sb] + c
> +                        buf[self.sb] = "".join((buf[self.sb] , c))
>                         continue
>                     else:
>                         self.iacseq += c
> @@ -480,8 +489,14 @@
>             self.iacseq = '' # Reset on EOF
>             self.sb = 0
>             pass
> -        self.cookedq = self.cookedq + buf[0]
> -        self.sbdataq = self.sbdataq + buf[1]
> +        self.cookedq = "".join((self.cookedq , buf[0] ))
> +        self.sbdataq = "".join((self.sbdataq , buf[1] ))
> +
> +
> +    def rawq_flush(self):
> +        """ Set the queue to empty status """
> +        self.rawq = ''
> +        self.irawq = 0
>
>     def rawq_getchar(self):
>         """Get next char from raw queue.
> @@ -516,7 +531,7 @@
>         buf = self.sock.recv(50)
>         self.msg("recv %r", buf)
>         self.eof = (not buf)
> -        self.rawq = self.rawq + buf
> +        self.rawq = "".join((self.rawq,buf))
>
>     def sock_avail(self):
>         """Test whether data is available on the socket."""
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)