Performance evaluation of HTTPS library

[Ashish]

On Oct 13, 6:12 pm, Antoine Pitrou <solip... at pitrou.net> wrote:
> On Wed, 13 Oct 2010 05:27:29 -0700 (PDT)Ashish <amvya... at gmail.com> wrote:
>
> > Well, CBSocket is socket implementation that calls my callback on
> > data.
> > Both my classes AsyncHTTPSConnection and AsyncHTTPConnection use it
> > and use it the same way ( self.sock = CBSocket(sock2) ).
> > The implemetation of AsyncHTTPConnection differs from
> > AsyncHTTPSConnection only in connect method: sock2 =
> > ssl.wrap_socket(sock, self.key_file, self.cert_file)
>
> > class CBSocket(asynchat.async_chat):
>
> [...]
>
> Ok, this won't work as expected. The first issue is that
> ssl.wrap_socket() is a blocking operation, where your client will send
> data and wait for the server reply (it's the SSL's handshake),
> *before* the socket has been set in non-blocking mode by asyncore. It
> means that your client will remain idle a lot of time, and explains
> that neither the client nor the server reach 100% CPU utilization.
>
> The second issue is that combining SSL and asyncore is more complicated
> than that; there are various situations to consider which your code
> doesn't address. The stdlib right now doesn't provide SSL support for
> asyncore (seehttp://bugs.python.org/issue10084), so you would have to
> do it yourself. I don't think it's worth the trouble, and would
> recommend switching your client to a simple thread-based approach,
> where you handle each HTTP(S) connection in a separate thread and stick
> to blocking I/O.
>
> Regards
>
> Antoine.

I am impressed by the knowledge and also thankful to you for helping
me out.

I thought threads will be costly to use and if I go for say 200
parallel connections with 200 total threads (+ a few more I have in my
tool), it may not be efficient either. Let me try to change the
implementation to use threads + blocking i/o and get back with
results.

One more question: If I run the tool from multicore machine, will
python3.1 or 3.2 be able to actually use multicore? or it will be
running only on one core?

Thanks
Ashish.