[Python-Dev] AIX, test_ssl in particular, but also AIX failed tests in general - getting to the 'STABLE' side

[Michael]

Dear all,

Last week I experimented with test_ssl. My expectation was that test
failures was caused by the openssl.base provided by IBM for AIX not
having a default certificate file (CApath). However, that is not the case.

The tests that fail are similar to:
self.assertRaisesRegex(ssl.SSLError, "PEM lib")

I started out by testing with something as:
if not AIX:
    with self.assertRaisesRegex(ssl.SSLError, "PEM lib"):
        ctx.load_cert_chain(BADCERT)
else:
    with self.assertRaises(ssl.SSLError):
        ctx.load_cert_chain(BADCERT)

This is after an analysis where I saw that calls too SSL were returning
an non-success status (!= 1) while ERR_peek_last_error() regularly
returned 0. Hence, the frequent 'AssertionError: "PEM lib" does not
match "unknown error ...' with "unknown error" the string Python provides.

While above might remove the 'fail messages' it did not satisfy me. So,
I downloaded openssl (1.0.2p) and compiled - with no optimization! And
now, even from Python3.6 I see:

test_ssl passed in 1 min 23 sec

== Tests result: SUCCESS ==

1 test OK.

In short, the failures of test_ssl may be ignored - as far as raising an
exception goes.

a) I am running a bot for Python, and once the argument
"-with-openssl=/opt/aixtools" is added my bot will stop showing these
errors. I mention this so that it is clear why they suddenly disappear
on my bot (but not elsewhere). Also to alert Python-Dev that the AIX
platform, regarding ssl.py, _ssl.c and test_ssl.py functions 'stable'
but is not as friendly when it comes to saying why WHEN (my guess) a
heavily optimized (I am thinking -O3 to -O5) library is used.

b) With this feedback - MAYBE - the team from IBM might review the way
they package openssl and make sure the messages are visible via
ERR_peek_last_error() et al. Ideally, IBM will notice and work on it
without prompting. One can dream :)

c) In the meantime - I am curious to know what this 'proof' means to
Python-Dev.

I have a simple goal - work through the tests that AIX has been failing
historically and figure out why they fail and fix the tests. To that end
I have submitted several PR's - starting back In January, then nothing
as noone ever seemed to notice, and the last weeks several additional
ones. Victor has been kind enough to say he will look at the tests as he
has time (and back from vacation). But we are all, or most, working on
our time. My goal, rephrased, is to see AIX in the 'stable' column so
that when a test fails it is because there is a regression that needs
addressing - either in the test or in the proposed code change. So I
would be grateful if others were also looking.

I am not trying to re-invent the wheel and will not be surprised if my
'test fix' is not done in the 'Python' way. I'll learn over time - but
this calls for instructive (and critical) comments. "bij voorbaat dank"
aka Thanks in Advance.

So, hoping this helps - I'll continue as I can. Time and resources are
limited. And, I am very curious re: point c) above.

Great Days! everyone,

Michael


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180827/e3363d6f/attachment.sig>