Performance evaluation of HTTPS library

Ashish amvyas82 at gmail.com
Thu Oct 14 14:06:30 CEST 2010


On Oct 13, 6:12 pm, Antoine Pitrou <solip... at pitrou.net> wrote:
> On Wed, 13 Oct 2010 05:27:29 -0700 (PDT)Ashish <amvya... at gmail.com> wrote:
>
> > Well, CBSocket is socket implementation that calls my callback on
> > data.
> > Both my classes AsyncHTTPSConnection and AsyncHTTPConnection use it
> > and use it the same way ( self.sock = CBSocket(sock2) ).
> > The implemetation of AsyncHTTPConnection differs from
> > AsyncHTTPSConnection only in connect method: sock2 =
> > ssl.wrap_socket(sock, self.key_file, self.cert_file)
>
> > class CBSocket(asynchat.async_chat):
>
> [...]
>
> Ok, this won't work as expected. The first issue is that
> ssl.wrap_socket() is a blocking operation, where your client will send
> data and wait for the server reply (it's the SSL's handshake),
> *before* the socket has been set in non-blocking mode by asyncore. It
> means that your client will remain idle a lot of time, and explains
> that neither the client nor the server reach 100% CPU utilization.
>
> The second issue is that combining SSL and asyncore is more complicated
> than that; there are various situations to consider which your code
> doesn't address. The stdlib right now doesn't provide SSL support for
> asyncore (seehttp://bugs.python.org/issue10084), so you would have to
> do it yourself. I don't think it's worth the trouble, and would
> recommend switching your client to a simple thread-based approach,
> where you handle each HTTP(S) connection in a separate thread and stick
> to blocking I/O.
>
> Regards
>
> Antoine.

I am impressed by the knowledge and also thankful to you for helping
me out.

I thought threads will be costly to use and if I go for say 200
parallel connections with 200 total threads (+ a few more I have in my
tool), it may not be efficient either. Let me try to change the
implementation to use threads + blocking i/o and get back with
results.

One more question: If I run the tool from multicore machine, will
python3.1 or 3.2 be able to actually use multicore? or it will be
running only on one core?

Thanks
Ashish.



More information about the Python-list mailing list