[Python-Dev] OpenSSL thread safety when reading files?

[Bill Janssen]

I'm seeing a number of malloc (actully, free) errors, now that I'm
pounding on the OpenSSL server/client setup with lots of server
threads and client threads.  They all look like either

(gdb) bt
#0  0x9010b807 in malloc_printf ()
#1  0x900058ad in szone_free ()
#2  0x90005588 in free ()
#3  0x9194e508 in CRYPTO_free ()
#4  0x91993e77 in ERR_clear_error ()
#5  0x919b1884 in PEM_X509_INFO_read_bio ()
#6  0x9197a692 in X509_load_cert_crl_file ()
#7  0x9197a80e in by_file_ctrl ()
#8  0x919d6e2e in X509_STORE_load_locations ()
[...]

or (much more frequently)

(gdb) bt
#0  0x9010b807 in malloc_printf ()
#1  0x900058ad in szone_free ()
#2  0x90005588 in free ()
#3  0x9194e508 in CRYPTO_free ()
#4  0x91993e77 in ERR_clear_error ()
#5  0x949fcf11 in SSL_CTX_use_certificate_chain_file ()
[...]

Always in ERR_clear_error(), always from some frame that's reading a
certificate file for some purpose.

If I disable Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS around the
places where the C code reads the certificate files, all these free
errors go away.

ERR_clear_error() is supposed to be thread-safe; it operates on a
per-thread error state structure (which I make sure is initialized in
my C code).  But it sure looks like the client and server threads are
both working with the same error state.

Bill