[issue9146] Segfault in hashlib in OpenSSL FIPS mode using non-FIPS-compliant hashes, if "ssl" imported before "hashlib"

Dave Malcolm report at bugs.python.org
Fri Jul 2 21:10:10 CEST 2010

New submission from Dave Malcolm <dmalcolm at redhat.com>:

Having run:
  prelink --undo --all
the following works OK:
 OPENSSL_FORCE_FIPS_MODE=1 python -c "import hashlib; m = m = hashlib.md5(); m.update('abc')"

but the following segfaults:
 OPENSSL_FORCE_FIPS_MODE=1 python -c "import ssl; import hashlib; m = m = hashlib.md5(); m.update('abc')"

and the following gives the same segfault, in a simpler way (strictly speaking one shouldn't directly import _ssl, but this most directly reproduces the crash):
 OPENSSL_FORCE_FIPS_MODE=1 python -c "import _ssl; import hashlib; m = m = hashlib.md5(); m.update('abc')"

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007fffee978736 in EVP_hash (self=0xaed3c0, vp=0x95b6a4, len=3) at /home/david/coding/python-svn/trunk-fips/Modules/_hashopenssl.c:112
#2  0x00007fffee978cb5 in EVP_update (self=0xaed3c0, args=('abc',)) at /home/david/coding/python-svn/trunk-fips/Modules/_hashopenssl.c:247
#3  0x0000000000567faa in PyCFunction_Call (func=<built-in method update of _hashlib.HASH object at remote 0xaed3c0>, arg=('abc',), kw=0x0)
    at Objects/methodobject.c:81
and the segfault is due to EVP_DigestUpdate calling through the ctx->update function pointer, which in this case is NULL.

(gdb) p self->ctx
$2 = {digest = 0x7ffff0c955a0, engine = 0x0, flags = 0, md_data = 0x0, pctx = 0x0, update = 0}

self->ctx->update == NULL and self->ctx->digest == &bad_md, as set up by:

The setup is here: (different run of gdb, so different addresses):

(gdb) bt
#0  EVP_DigestInit_ex (ctx=0x7fffecdc7b80, type=0x7fffefc6fc60, impl=0x0) at
#1  0x00007fffecbc53ce in init_hashlib () at

                  if (FIPS_mode())
                          if (!(type->flags & EVP_MD_FLAG_FIPS)
                           && !(ctx->flags & EVP_MD_CTX_FLAG_NON_FIPS_ALLOW))
  =>                              ctx->digest = &bad_md;
                                  return 0;

(seen on x86_64 Linux; Fedora 13, with openssl-1.0.0-1.fc13.x86_64 on SVN trunk, and on other builds)

Clearly, the Python process should not segfault or abort.

I don't think it's clear what the behavior should be here, though - how is the Python standard library to be used in conjunction with OpenSSL's FIPS mode?

>From page 18 of the OpenSSL's FIPS guide: http://ftp.openssl.org/docs/fips/UserGuide.pdf
"By design, the OpenSSL API attempts to disable non­FIPS algorithms, when in FIPS mode, at the 
EVP level and via most low level function calls.  Failure to check the return code from low level 
functions could result in unexpected behavior.  Note also that sufficiently creative or unusual use of 
the API may still allow the use of non­FIPS algorithms.  The non­FIPS algorithm disabling is 
intended as a aid to the developer in preventing the accidental use of non­FIPS algorithms in FIPS 
mode, and not as an absolute guarantee. It is the responsibility of the application developer to 
ensure that no non­FIPS algorithms are used when in FIPS mode."

It seems odd that the behavior of "md5" within "hashlib" can vary depending on whether "_ssl" was imported first.  Reducing surprises for the programmer suggests to me that the behavior for these two cases should be harmonized.

It also seems odd that SSL can refuse the usage of MD5 whilst other implementations exist and are available; my understanding of FIPS mode is that it's intended for locked-down environments that wish to ensure that only known-good crypto is used (and e.g. MD5 is known to be be broken for some uses, see: http://www.kb.cert.org/vuls/id/836068)

Possible approaches:
  (A) change hashlib so that it continues to support md5 if ssl/_ssl is imported first, even in FIPS mode
  (B) change hashlib so that in FIPS mode, it fails to support md5 even if ssl/_ssl is not imported first
    (B.1) by not recognizing "md5" as a digest, leading to NameError exceptions and similar (I dislike this approach, as it leads to obscure failures)
    (B.2) by raising a new exception e.g. "ProhibitedInFIPSMode" or somesuch (so that the failure is obvious and searchable)
  (C.*) as per (B.*), but support an explicit override: hashlib.md5(non_fips_allow=True)


components: Library (Lib)
messages: 109121
nosy: dmalcolm
priority: normal
severity: normal
status: open
title: Segfault in hashlib in OpenSSL FIPS mode using non-FIPS-compliant hashes, if "ssl" imported before "hashlib"
type: crash
versions: Python 2.7

Python tracker <report at bugs.python.org>

More information about the Python-bugs-list mailing list