[issue8108] test_ftplib fails with OpenSSL 0.9.8m

[Darryl Miles]

Darryl Miles <darryl.miles at darrylmiles.org> added the comment:

I am unable to get "make test" to run from an unpatched version in SVN (details below of make output).

Please find attached an updated patch for your consideration (and testing, as I can't test it due to 'make test' SIGSEGV on CentOS 5.4 i386).


Patch Notes:

1) Some thing that concern me, the "unwrap()" philosophy looks to be used to remove SSL from the Python high-level socket handle, so you can go back to plaintext mode.  You can ONLY perform an unwrap() AFTER an SSL_shutdown()==1 has been observed (you need to wait for the other end to do something voluntarily).

So you must retry the SSL_shutdown() over and over while you sleep-wait for IO, so this is akin to calling the ssl.shutdown(ssl.SSL_SHUTDOWN_MODE_BOTH) and getting back success.

Also if it is your intention to properly implement an unwrap() like this you should disable IO read-ahead mode before calling shutdown for the second time, SSL_set_read_ahead(ssl, 0).  This stops OpenSSL from eating too many bytes accidentally (probably from the kernel into its own buffers), from the inbound IO stream, which may not be SSL protocol data, it maybe plain text data (behind the last byte of SSL protocol data).


2) Due to the IO waiting it looks also necessary to copy the setup of SSL_set_nbio() from the read/write paths so the check_socket_and_wait_for_timeout() works in sympathy to the callers IO timeout reconfiguration.


3) My patch presumes the allocation of the type "struct PySSLObject" uses calloc() or some other memory zeroing strategy.  There is a new member in that struct to track if SSL_shutdown() has previously returned a zero value.


4) The SSL_peek() error path needs checking to see if the error return is consistent with the Python paradigm.


5) Please check I have understand the VARARGS method correctly.  I have made the default to SSL_SHUTDOWN_MODE_SENT (despite backward compatibly being SSL_SHUTDOWN_MODE_ONCE), this is because I would guess that most high-level applications did not intend to use it in raw mode; nor be  bothered with the issues surrounding correct usage.  I would guess high-level applications wanted Python to take the strain here.


6) I suspect you need to address your unwrap() policy a little better, the shutdown operation and the unwrap() are two different matters.  The shutdown() should indicate success or not (in respect of the mode being requested, raw mode is a tricky one as the caller would want to the exact error return so it can do the correct thing), unwrap() should itself call ssl.shutdown(ssl.SSL_SHUTDOWN_MODE_BOTH) until it sees success and then remove the socket (and deallocate SSL objects).

As things stand SSL_SHUTDOWN_MODE_ONCE does not work in a useful way since the error returns are not propagated to the caller, because "unwrap" is mixed into this.  So that would still need fixing.



building works ok, testing fails with SIGSEGV.  Is this something to do with no having _bsddb built ?  I have db-4.3 working.  Maybe someone can reply by email on the matter.

# make
running build
running build_ext
building dbm using gdbm

Python build finished, but the necessary bits to build these modules were not found:
bsddb185           sunaudiodev                        
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

running build_scripts




# make test
running build
running build_ext
building dbm using gdbm

Python build finished, but the necessary bits to build these modules were not found:
bsddb185           sunaudiodev                        
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

running build_scripts
find ./Lib -name '*.py[co]' -print | xargs rm -f
./python -Wd -3 -E -tt ./Lib/test/regrtest.py -l 
== CPython 2.7a4+ (trunk:79902M, Apr 11 2010, 16:38:55) [GCC 4.1.2 20080704 (Red Hat 4.1.2-46)]
==   Linux-2.6.18-164.15.1.el5-i686-with-redhat-5.4-Final
==   /root/python-svn/build/test_python_29248
test_grammar
test_opcodes
test_dict
test_builtin
test_exceptions
test_types
test_unittest
test_doctest
test_doctest2
test_MimeWriter
test_SimpleHTTPServer
test_StringIO
test___all__
/root/python-svn/Lib/test/test___all__.py:10: DeprecationWarning: in 3.x, the bsddb module has been removed; please use the pybsddb project instead
  import bsddb
/root/python-svn/Lib/bsddb/__init__.py:67: PendingDeprecationWarning: The CObject type is marked Pending Deprecation in Python 2.7.  Please use capsule objects instead.
  import _bsddb
make: *** [test] Segmentation fault

----------
Added file: http://bugs.python.org/file16872/Modules__ssl.c.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8108>
_______________________________________