Performance evaluation of HTTPS library

[Ashish]

On Oct 13, 11:11 am, Ashish <amvya... at gmail.com> wrote:
> On Oct 12, 6:33 pm, Antoine Pitrou <solip... at pitrou.net> wrote:> On Tue, 12 Oct 2010 05:40:43 -0700 (PDT)
>
> > Ashish Vyas <amvya... at yahoo.com> wrote:
> > > Another observation that I have made is with 10 parallel HTTPS connection each
> > > trying 1 transaction per second from 2 different machines (effectively same load
> > > on server), the response time is again reducing to .17 secs.
> > > However if I run two instances of the tool with 10 parallel HTTPS connection
> > > each trying 1 transaction per second from from same machine, the response time
> > > is again shooting up to 1.1 seconds.
>
> > Is the client machine at 100% CPU when you do that?
>
> With HTTP, I see client CPU at appx. 97%. However with HTTPS, it stays
> at 53-55%.
>
> > > So the question is does anyone here have any idea or some data about performance
> > > limitation of HTTPS implementation in Python 3.1?
>
> > Which API are you using? urlopen()?
> > The HTTPS implementation is basically the same as the HTTP
> > implementation, except for the additional SSL layer. So if indeed
> > Python is responsible for the slowdown, it may be because of excessive
> > overhead brought by the SSL layer.
>
> I am doing something like this:-
>
> self.conn = AsyncHTTPSConnection(self.URL, HTTPS_PORT)
>
> self.conn.putrequest('POST', WEBSERVER_IP)
> self.conn.putheader('Cookie', cookie)
> self.conn.putheader('Content-Length', reqLen)
> ..
> self.conn.endheaders()
> self.conn.send(str.encode(reqest))
>
> and AsyncHTTPSConnection class is something like this:-
>
> class AsyncHTTPSConnection(client.HTTPConnection):
>     default_port = HTTPS_PORT
>     def __init__(self, host, port=HTTPS_PORT, key_file=None,
> cert_file=None,
>                      strict=None,
> timeout=socket._GLOBAL_DEFAULT_TIMEOUT):
>         """ Init has same eparameters as HTTPSConnection. """
>         client.HTTPConnection.__init__(self, host, port, strict,
> timeout)
>         self.key_file = key_file
>         self.cert_file = cert_file
>
>     def connect(self):
>         try:
>             log.mjLog.LogReporter ("Model", "info",
> "AsyncHTTPSConnection::connect trying to connect... "+ str(self.host)
> + ":"+ str(self.port))
>             sock = socket.create_connection((self.host, self.port),
> self.timeout)
>             sock2 = ssl.wrap_socket(sock, self.key_file,
> self.cert_file)
>             self.sock = CBSocket(sock2)
>         except:
>             log.mjLog.LogReporter ("Model", "critical",
> "AsyncHTTPSConnection::connect Failed to connect to the GWS")
>
> > It would be nice if you tried the just-released Python 3.2 alpha,
> > because some changes have been made to the SSL wrapper:http://python.org/download/releases/3.2/
>
> Let me try to use this, I will come back with my observations.

Well, I tried python3.2a2 and the average response time for 20 HTTPS
tps reduced from about 1.1 seconds to about .97 seconds.
This is a noticeable change but not enough I feel.
Also when I tried running same test client on XEON, I am able to see
average response time appx. 0.23 seconds.

>
> > Also, there's a feature request to reduce overhead of SSL
> > connections, but it needs implementing:http://bugs.python.org/issue8106
>
> Well good to know this. Do we have any date when this will be
> available? I feel like contributing to this but kind of over occupied
> with several activities right now.
>
>
>
> > Regards
>
> > Antoine.
>
> Thanks a lot,
> Ashish