[New-bugs-announce] [issue21564] Declaration of EVP_MD_CTX causes crash when switching between OpenSSL 0.9 and 1.0

[Ryan Calhoun]

New submission from Ryan Calhoun:

OpenSSL, specifically libcrypto, includes functions EVP_MD_CTX_create() and EVP_MD_CTX_destroy(), such that the application code only needs to forward declare the EVP_MD_CTX* pointer type.

Declaring the EVP_MD_CTX variable type directly requires compile-time knowledge of the size of the structure. This knowledge will be wrong when compiling with headers from OpenSSL 0.9.8 and dynamically linking at run time against OpenSSL 1.0.1. Result is a SIGSEGV as follows:

(gdb) bt
#0  0x00007ffff03b71a0 in EVP_PKEY_CTX_dup () from /usr/lib64/libcrypto.so
#1  0x00007ffff03a90cd in EVP_MD_CTX_copy_ex () from /usr/lib64/libcrypto.so
#2  0x00007ffff110da8a in EVPnew (name_obj=0x7ffff7ef45a8, digest=0x0, initial_ctx=0x7ffff130fbc0, cp=0x0, len=0)
    at /home/ryan/vaas/python/2.7.6/build/src/Python-2.7.6/Modules/_hashopenssl.c:436
#3  0x00007ffff110de10 in EVP_new_md5 (self=<value optimized out>, args=<value optimized out>)
    at /home/ryan/vaas/python/2.7.6/build/src/Python-2.7.6/Modules/_hashopenssl.c:540

The attached patch updates all declarations in _hashopenssl.c to be pointers initialized by calling EVP_MD_CTX_create(). The patch is done against source version 3.4.1. I have a similar patch for version 2.7.6, but can only attach one file here?

----------
components: Extension Modules
files: EVP_MD_CTX-python3.4.patch
keywords: patch
messages: 218995
nosy: Ryan.Calhoun
priority: normal
severity: normal
status: open
title: Declaration of EVP_MD_CTX causes crash when switching between OpenSSL 0.9 and 1.0
type: crash
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file35330/EVP_MD_CTX-python3.4.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21564>
_______________________________________