python3.10 compilation on OpenBSD: running into ssl issues
Trying to compile python3.10 on openbsd 7.0 on Pi4. It seems to run into several openssl issue. I have installed openssl as I couldn't find libreSSL in the package manager. The configure seems to passthe ssl test. configure:17559: checking whether compiling and linking against OpenSSL works Trying link with OPENSSL_LDFLAGS=; OPENSSL_LIBS= -lssl -lcrypto; OPENSSL_INCLUDES= configure:17581: cc -pthread -o conftest conftest.c -lssl -lcrypto -lpthread -lutil -lm >&5 configure:17581: $? = 0 configure:17583: result: yes I get following errors:
don't know why this happens ldd: /usr/lib/libreadline.a: not an ELF executable
Errors when building _ssl module
cc -pthread -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/include -I. -I/usr/local/include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0 -c /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c -o build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.o In file included from /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:390: /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl/debughelpers.c:180:5: error: implicit declaration of function 'SSL_CTX_set_keylog_callback' is invalid in C99 [-Werror,-Wimplicit-function-declaration] SSL_CTX_set_keylog_callback(self->ctx, NULL); ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:1842:29: error: implicit declaration of function 'SSL_get0_verified_chain' is invalid in C99 [-Werror,-Wimplicit-function-declaration] STACK_OF(X509) *chain = SSL_get0_verified_chain(self->ssl); ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:1842:21: warning: incompatible integer to pointer conversion initializing 'struct stack_st_X509 *' with an expression of type 'int' [-Wint-conversion] STACK_OF(X509) *chain = SSL_get0_verified_chain(self->ssl); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2349:18: error: implicit declaration of function 'SSL_write_ex' is invalid in C99 [-Werror,-Wimplicit-function-declaration] retval = SSL_write_ex(self->ssl, b->buf, (size_t)b->len, &count); /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2349:18: note: did you mean 'SSL_write'? /usr/include/openssl/ssl.h:1415:6: note: 'SSL_write' declared here int SSL_write(SSL *ssl, const void *buf, int num); ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2501:18: error: implicit declaration of function 'SSL_read_ex' is invalid in C99 [-Werror,-Wimplicit-function-declaration] retval = SSL_read_ex(self->ssl, mem, (size_t)len, &count); ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2501:18: note: did you mean 'SSL_read'? /usr/include/openssl/ssl.h:1413:6: note: 'SSL_read' declared here int SSL_read(SSL *ssl, void *buf, int num); ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:3542:30: error: implicit declaration of function 'SSL_CTX_get_num_tickets' is invalid in C99 [-Werror,-Wimplicit-function-declaration] return PyLong_FromSize_t(SSL_CTX_get_num_tickets(self->ctx)); ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:3560:9: error: implicit declaration of function 'SSL_CTX_set_num_tickets' is invalid in C99 [-Werror,-Wimplicit-function-declaration] if (SSL_CTX_set_num_tickets(self->ctx, num) != 1) { ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:3574:28: error: implicit declaration of function 'SSL_CTX_get_security_level' is invalid in C99 [-Werror,-Wimplicit-function-declaration] return PyLong_FromLong(SSL_CTX_get_security_level(self->ctx)); ^ cc -pthread -shared -fPIC build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_testcapimodule.o -L/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/lib -L/usr/local/lib -o build/lib.openbsd-7.0-arm64-3.10/_ testcapi.cpython-310.so
and errors building _hashlib module
cc -pthread -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/include -I. -I/usr/local/include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0 -c /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c -o build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.o building '_sha256' extension cc -pthread -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/include -I. -I/usr/local/include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0 -c /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/sha256module.c -o build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/sha256module.o -DPy_BUILD_CORE_MODULE /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:158:10: error: use of undeclared identifier 'NID_sha3_224' case NID_sha3_224: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:161:10: error: use of undeclared identifier 'NID_sha3_256' case NID_sha3_256: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:164:10: error: use of undeclared identifier 'NID_sha3_384' case NID_sha3_384: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:167:10: error: use of undeclared identifier 'NID_sha3_512' case NID_sha3_512: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:172:10: error: use of undeclared identifier 'NID_shake128' case NID_shake128: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:175:10: error: use of undeclared identifier 'NID_shake256' case NID_shake256: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:180:10: error: use of undeclared identifier 'NID_blake2s256' case NID_blake2s256: ^ /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:183:10: error: use of undeclared identifier 'NID_blake2b512' case NID_blake2b512: Any suggesstions? Thanks -S
On 05/10/2021 16.40, Sandeep Gupta wrote:
Trying to compile python3.10 on openbsd 7.0 on Pi4. It seems to run into several openssl issue. I have installed openssl as I couldn't find libreSSL in the package manager.
Your installation is picking up header files from LibreSSL. Python 3.10 requires a fully OpenSSL 1.1.1 API compliant OpenSSL. LibreSSL is missing a bunch of required features. Please follow the instruction https://docs.python.org/3.10/using/unix.html#custom-openssl to compile, install, and use a custom installation of OpenSSL. I recommend that you use the latest OpenSSL 1.1.1 version. Regards, Christian
Tried with openssl. Some progress but no success. The configure checks went through find.
configure:17536: checking for openssl/ssl.h in /home/kabira/DrivingRange//project_versa/Build s/openssl-1.1.1l configure:17543: result: yes configure:17559: checking whether compiling and linking against OpenSSL works Trying link with OPENSSL_LDFLAGS=-L/home/kabira/DrivingRange//project_versa/Builds/openssl-1. 1.1l/lib; OPENSSL_LIBS=-lssl -lcrypto; OPENSSL_INCLUDES=-I/home/kabira/DrivingRange//project_ versa/Builds/openssl-1.1.1l/include
But for some reason module could not be imported. I could find any errors related to import. There were no compilation errors:
Following modules built successfully but were removed because they could not be imported: _hashlib _ssl readline
There area number of warnings regarding type conversions. Just attaching that for sanity check. This readline error is particulary wierd. ==> ldd: /usr/lib/libreadline.a: not an ELF executable
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Python/pytime.c:160:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] if (!_Py_InIntegralTypeRange(time_t, intpart)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange' #define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
~~ ^~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax' #define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0) ` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ Python/pytime.c:213:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] if (!_Py_InIntegralTypeRange(time_t, intpart)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange' #define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type)) ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax' #define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ Python/pytime.c:398:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] if (!_Py_InIntegralTypeRange(_PyTime_t, d)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange' #define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type)) ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax' #define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ 3 warnings generated. ./Modules/_threadmodule.c:1644:26: warning: implicit conversion from '_PyTime_t' (aka 'long long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-const-int-float-conversion] double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6; ^~~~~~~~~~~~~~~~~~~~~~~~~ ~ 1 warning generated. ldd: /usr/lib/libreadline.a: not an ELF executable /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/readline.c:1258:37: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers] completer_word_break_characters = ^ 1 warning generated. *** WARNING: renaming "readline" since importing it failed: dynamic module does not define module export function (PyInit_readline) *** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object *** WARNING: renaming "_hashlib" since importing it failed: Cannot load specified object renaming build/scripts-3.10/pydoc3 to build/scripts-3.10/pydoc3.10 renaming build/scripts-3.10/idle3 to build/scripts-3.10/idle3.10 renaming build/scripts-3.10/2to3 to build/scripts-3.10/2to3-3.10 *** Warning in /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0: '$(LLVM_PROF_MERGER)' expands to '' while building build_all_merge_profile (Makefile:539) Python/pytime.c:160:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] if (!_Py_InIntegralTypeRange(time_t, intpart)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange' #define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type)) ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax' #define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ Python/pytime.c:213:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] if (!_Py_InIntegralTypeRange(time_t, intpart)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange' #define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type)) ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax' #define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ Python/pytime.c:398:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] if (!_Py_InIntegralTypeRange(_PyTime_t, d)) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange' #define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type)) ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~ ./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax' #define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ 3 warnings generated. ./Modules/_threadmodule.c:1644:26: warning: implicit conversion from '_PyTime_t' (aka 'long long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-const-int-float-conversion] double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6; ^~~~~~~~~~~~~~~~~~~~~~~~~ ~ 1 warning generated. ldd: /usr/lib/libreadline.a: not an ELF executable /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/readline.c:1258:37: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers] completer_word_break_characters = ^ 1 warning generated. *** WARNING: renaming "readline" since importing it failed: dynamic module does not define module export function (PyInit_readline) *** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object *** WARNING: renaming "_hashlib" since importing it failed: Cannot load specified object renaming build/scripts-3.10/pydoc3 to build/scripts-3.10/pydoc3.10 renaming build/scripts-3.10/idle3 to build/scripts-3.10/idle3.10 renaming build/scripts-3.10/2to3 to build/scripts-3.10/2to3-3.10
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
On Tuesday, October 5, 2021, Christian Heimes
On 05/10/2021 16.40, Sandeep Gupta wrote:
Trying to compile python3.10 on openbsd 7.0 on Pi4. It seems to run into several openssl issue. I have installed openssl as I couldn't find libreSSL in the package manager.
Your installation is picking up header files from LibreSSL. Python 3.10 requires a fully OpenSSL 1.1.1 API compliant OpenSSL. LibreSSL is missing a bunch of required features.
Please follow the instruction https://docs.python.org/3.10/u sing/unix.html#custom-openssl to compile, install, and use a custom installation of OpenSSL. I recommend that you use the latest OpenSSL 1.1.1 version.
Regards, Christian
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archiv es/list/python-dev@python.org/message/BK22LJ5LICXUMIU6MYB746FSZ57KJX2T/ Code of Conduct: http://python.org/psf/codeofconduct/
On 06/10/2021 09.06, Sandeep Gupta wrote:
Tried with openssl. Some progress but no success. The configure checks went through find.
configure:17536: checking for openssl/ssl.h in /home/kabira/DrivingRange//project_versa/Build s/openssl-1.1.1l configure:17543: result: yes configure:17559: checking whether compiling and linking against OpenSSL works Trying link with OPENSSL_LDFLAGS=-L/home/kabira/DrivingRange//project_versa/Builds/openssl-1. 1.1l/lib; OPENSSL_LIBS=-lssl -lcrypto; OPENSSL_INCLUDES=-I/home/kabira/DrivingRange//project_ versa/Builds/openssl-1.1.1l/include
But for some reason module could not be imported. I could find any errors related to import. There were no compilation errors:
Following modules built successfully but were removed because they could not be imported: _hashlib _ssl readline
[...]
*** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object *** WARNING: renaming "_hashlib" since importing it failed: Cannot load specified object
OpenBSD uses clang C compiler. The new --with-openssl-rpath=auto option was only tested with GCC. It turned out that Python's distutils package didn't support rpath with clang. The bugfix https://bugs.python.org/issue45371 will be available in upcoming release Python 3.10.1. In the mean time you either need to apply the patch from the issue manually or figure out the right environment variables to add correct rpath yourself. I'm sorry for the inconvenience. We don't have any CI for OpenBSD. Apparently this feature was never tested on OpenBSD during the release candidate phase either. Christian
Thank you for super quick fix ). Works like charm.
Thanks
S
On Wednesday, October 6, 2021, Christian Heimes
On 06/10/2021 09.06, Sandeep Gupta wrote:
Tried with openssl. Some progress but no success. The configure checks went through find.
configure:17536: checking for openssl/ssl.h in /home/kabira/DrivingRange//project_versa/Build s/openssl-1.1.1l configure:17543: result: yes configure:17559: checking whether compiling and linking against OpenSSL works Trying link with OPENSSL_LDFLAGS=-L/home/kabira /DrivingRange//project_versa/Builds/openssl-1. 1.1l/lib; OPENSSL_LIBS=-lssl -lcrypto; OPENSSL_INCLUDES=-I/home/kabir a/DrivingRange//project_ versa/Builds/openssl-1.1.1l/include
But for some reason module could not be imported. I could find any errors related to import. There were no compilation errors:
Following modules built successfully but were removed because they could not be imported: _hashlib _ssl readline
[...]
*** WARNING: renaming "_ssl" since importing it failed: Cannot load
specified object *** WARNING: renaming "_hashlib" since importing it failed: Cannot load specified object
OpenBSD uses clang C compiler. The new --with-openssl-rpath=auto option was only tested with GCC. It turned out that Python's distutils package didn't support rpath with clang. The bugfix https://bugs.python.org/issue4 5371 will be available in upcoming release Python 3.10.1.
In the mean time you either need to apply the patch from the issue manually or figure out the right environment variables to add correct rpath yourself.
I'm sorry for the inconvenience. We don't have any CI for OpenBSD. Apparently this feature was never tested on OpenBSD during the release candidate phase either.
Christian
participants (2)
-
Christian Heimes -
Sandeep Gupta