Windows package for new SSL package?
I can't figure out how to build a Windows package for ssl-1.1.tar.gz, and probably don't have the tools to do it anyway. I presume that both a Windows machine and Visual Studio (because there's a C extension) is required? Anyone out there who's interested in the challenge? It's at http://www.parc.com/janssen/transient/ssl-1.1.tar.gz. Incidentally, there's no documentation in the package; instead, just use the development documentation at http://docs.python.org/dev/library/ssl.html. Bill
I can't figure out how to build a Windows package for ssl-1.1.tar.gz, and probably don't have the tools to do it anyway. I presume that both a Windows machine and Visual Studio (because there's a C extension) is required?
Anyone out there who's interested in the challenge?
It's at http://www.parc.com/janssen/transient/ssl-1.1.tar.gz.
I had a bit of a look at this. I think I managed to get it building: * find_ssl() is along way from working on Windows. Python itself uses magic to locate an SSL directory in the main Python directory's parent. On my system, this is c:\src\openssl-0.9.7e, but obviously that could be almost anywhere, and with almost any name. See PCBuild\build_ssl.py and PCBuild\_ssl.mak for the gory details. I'm not sure how you would like to approach this (insist on an environment variable for the top-level SSL dir name?), but in the meantime I hacked find_ssl() to: ssl_incs = [r"\src\openssl-0.9.7e\inc32",] ssl_libs = [r"\src\openssl-0.9.7e\out32"] return ssl_incs, ssl_libs, ["libeay32", "ssleay32", "gdi32", "wsock32"] * The call to find_ssl() appears to discard the 3rd param: ssl_incs, ssl_libs, libs = find_ssl() ... ext_modules=[Extension('ssl._ssl2', ['ssl/_ssl2.c'], include_dirs = ssl_incs + [socket_inc], library_dirs = ssl_libs, libraries = ['ssl', 'crypto'], depends = ['ssl/socketmodule.h'])], The 'libraries =' line probably means to pass 'libs' rather than the literal. * The "depends = ['ssl/socketmodule.h']" fails for me - no header of that name exists in the ssl directory in your archive. After those changes I was able to get it built and tested: """ Ran 15 tests in 3.157s OK """ Hope this helps, Mark
Thanks, Mark (and David, who replied to me personally). I'll update the setup.py files with your suggestions and do a 1.2 (with more metadata in it, too). Looks like the functionality is working for people, even if the build is still a bit flakey. Bill
* find_ssl() is along way from working on Windows. Python itself uses magic to locate an SSL directory in the main Python directory's parent. On my system, this is c:\src\openssl-0.9.7e, but obviously that could be almost anywhere, and with almost any name. See PCBuild\build_ssl.py and PCBuild\_ssl.mak for the gory details. I'm not sure how you would like to approach this (insist on an environment variable for the top-level SSL dir name?)
Can't we look in the registry for this? We have a working Python; perhaps we can just use a Windows-specific registry lookup to find OpenSSL? (I'm just blue-skying here; I have no clue how things work on Windows.) Bill
* find_ssl() is along way from working on Windows. Python itself uses magic to locate an SSL directory in the main Python directory's parent. On my system, this is c:\src\openssl-0.9.7e, but obviously that could be almost anywhere, and with almost any name. See PCBuild\build_ssl.py and PCBuild\_ssl.mak for the gory details. I'm not sure how you would like to approach this (insist on an environment variable for the top-level SSL dir name?)
Can't we look in the registry for this? We have a working Python; perhaps we can just use a Windows-specific registry lookup to find OpenSSL? (I'm just blue-skying here; I have no clue how things work on Windows.)
Not really. Python itself, when building _ssl, doesn't look for a binary install of openssl, but instead a source directory and a working perl interpreter so an openssl can be built suitable for linking with Python. This source directory is just downloaded and unzipped - no registration takes place, and any binaries that may be built are ignored (we just want the .h and .lib files) It might be possible to try and use build_ssl.py to locate the openssl directory, but this will still require that someone building it has Python built from source - I'm fairly sure that someone installing a Python binary will not have build_ssl.py, nor are they likely to have a suitable openssl directory or installation just "hanging around" either. Mark
Can't we look in the registry for this? We have a working Python; perhaps we can just use a Windows-specific registry lookup to find OpenSSL? (I'm just blue-skying here; I have no clue how things work on Windows.)
Not really. Python itself, when building _ssl, doesn't look for a binary install of openssl, but instead a source directory and a working perl interpreter so an openssl can be built suitable for linking with Python. This source directory is just downloaded and unzipped - no registration takes place, and any binaries that may be built are ignored (we just want the .h and .lib files)
In that case, I think your idea of just hard-coding a path is probably the right thing to do. I'll add a note that this is how you need to do it if you are going to try "python setup.py build". Presumably the binary then built with "python setup.py bdist" will install on a Windows machine regardless of where OpenSSL is installed? Bill
Bill Janssen <janssen@parc.com> writes:
In that case, I think your idea of just hard-coding a path is probably the right thing to do. I'll add a note that this is how you need to do it if you are going to try "python setup.py build". Presumably the binary then built with "python setup.py bdist" will install on a Windows machine regardless of where OpenSSL is installed?
Yes (though typically bdist_wininst for the Windows installer), but perhaps not for the reason you think. I think where there's probably a small disconnect here is that, there really isn't an OpenSSL "installed" on the end user's machine. Well, there could be, but Python isn't using it. The OpenSSL library is statically linked as part of the _ssl.pyd module, as it will be with your _ssl2.pyd module. (That's also why there is no OpenSSL to "find" in your setup even with Python installed - at least not any libraries you can use). In other words, both the standard and your extension module on Windows bring along their own OpenSSL. -- David
On 13/09/2007, David Bolen <db3l.net@gmail.com> wrote:
Bill Janssen <janssen@parc.com> writes:
In that case, I think your idea of just hard-coding a path is probably the right thing to do. I'll add a note that this is how you need to do it if you are going to try "python setup.py build". Presumably the binary then built with "python setup.py bdist" will install on a Windows machine regardless of where OpenSSL is installed?
Yes (though typically bdist_wininst for the Windows installer), but perhaps not for the reason you think.
I think where there's probably a small disconnect here is that, there really isn't an OpenSSL "installed" on the end user's machine. Well, there could be, but Python isn't using it. The OpenSSL library is statically linked as part of the _ssl.pyd module, as it will be with your _ssl2.pyd module. (That's also why there is no OpenSSL to "find" in your setup even with Python installed - at least not any libraries you can use).
That's not 100% true, is it? If I use mingw and Gnuwin32 openssl, I believe the default is a dynamic link of openssl (it depends on the import library used, and gnuwin32 supplies dynamic libs by default). So the openssl DLLs need to be on the user's PATH for the extension module to work.
In other words, both the standard and your extension module on Windows bring along their own OpenSSL.
For the extension, you may need to (1) document that the user needs to have the openssl DLLs on their PATH and possibly (1a) supply a zipfile with the necessary DLLs as a supplemental download, or (2) arrange for the openssl DLLs to be included in the extension installer, and installed alongside the .pyd file. Alternatively, it *may* be possible with setup.py magic to force a static openssl link (but would that need hard coding for the gnuwin32 naming conventions?) Paul.
On 9/13/07, Paul Moore <p.f.moore@gmail.com> wrote:
On 13/09/2007, David Bolen <db3l.net@gmail.com> wrote:
I think where there's probably a small disconnect here is that, there really isn't an OpenSSL "installed" on the end user's machine. Well, there could be, but Python isn't using it. The OpenSSL library is statically linked as part of the _ssl.pyd module, as it will be with your _ssl2.pyd module. (That's also why there is no OpenSSL to "find" in your setup even with Python installed - at least not any libraries you can use).
That's not 100% true, is it? If I use mingw and Gnuwin32 openssl, I believe the default is a dynamic link of openssl (it depends on the import library used, and gnuwin32 supplies dynamic libs by default). So the openssl DLLs need to be on the user's PATH for the extension module to work.
That's a fair point - my comments are all related to the standard Python distribution and building extensions with the VS.NET compiler (including the binary installer I had built for Bill).
For the extension, you may need to (1) document that the user needs to have the openssl DLLs on their PATH and possibly (1a) supply a zipfile with the necessary DLLs as a supplemental download, or (2) arrange for the openssl DLLs to be included in the extension installer, and installed alongside the .pyd file.
If we're talking about the construction of binary Windows installers, I'd just suggest that they get built as the built-in SSL module does, including the static linking with a pure Windows OpenSSL build, which is a bit simpler for the typical end user and has no other external requirements. Of course, that certainly doesn't stop a person who has already set up their system for using mingw for extensions from doing their own compilation, although it does raise a question as to whether the setup.py would need some further adjustments to cover that case most cleanly. -- David
On 13/09/2007, David Bolen <db3l.net@gmail.com> wrote:
That's a fair point - my comments are all related to the standard Python distribution and building extensions with the VS.NET compiler (including the binary installer I had built for Bill). [...] If we're talking about the construction of binary Windows installers, I'd just suggest that they get built as the built-in SSL module does, including the static linking with a pure Windows OpenSSL build, which is a bit simpler for the typical end user and has no other external requirements.
OK. Building with mingw is a bit of a hobby horse of mine, as the requirement for the (expensive) VS.NET compiler forces many users to rely on binary builds. I know other alternatives can be made to work, but it's often too much pain to bother. I'd much rather extensions which *can* be built using free tools, support actually doing so out of the box.
Of course, that certainly doesn't stop a person who has already set up their system for using mingw for extensions from doing their own compilation, although it does raise a question as to whether the setup.py would need some further adjustments to cover that case most cleanly.
And that's my point. I'd rather work to ensure that mingw works out of the box, than leave things requiring VS.NET for a clean build. It's not relevant here, but I've certainly been in a situation with other extensions where I can't upgrade a Python install because the distributors of a particular extension haven't produced a build for the new version yet (*cough* mod_python *cough*) - and I live in fear of support for some extensions dying, as I'll then have to move away from them or stop upgrading Python. Anyway, philosophy aside, I'll try to make some time in the next few days to get a working setup.py for the SSL package using mingw. Hopefully, Bill will then integrate this and we'll have mingw as a supported option. Paul.
"Mark Hammond" <mhammond@skippinet.com.au> writes:
It might be possible to try and use build_ssl.py to locate the openssl directory, but this will still require that someone building it has Python built from source - I'm fairly sure that someone installing a Python binary will not have build_ssl.py, nor are they likely to have a suitable openssl directory or installation just "hanging around" either.
Yep - even if a Windows user has an appropriate development environment in general (and can build most standalone extensions with just a binary Python install), as you say the odds are pretty small they'd have an OpenSSL source tree around, with libraries built. At the same time, I suspect that only a small percentage of Windows users will want to rebuild the extension - rather they'll just want a binary installer, something not uncommon to be published for Windows users of many extension modules. So that pushes the problem upstream a bit where having a Python development tree might be more common or familiar. Rather than a lot of complexity to cater to that small percentage, I'd probably just make setup.py need an explicit configuration - editing, or perhaps environment variable - for the location of the root of the OpenSSL source tree. As you say, there's no guaranteed way to find it otherwise, although I suppose it might try checking relative to the Python executable (along the same lines as build_ssl.py) in case it's being built from within the source tree. Adding some comments that following instructions to build Python from source (or at least the standard _ssl module) will yield just such a tree should be a simple enough as a reference for those who need it. The setup.py does also need to understand the different library names (and required system libraries) to build properly under Windows, as you've already highlighted, but that should be relatively easy to vary by platform. -- David
On 13/09/2007, David Bolen <db3l.net@gmail.com> wrote:
"Mark Hammond" <mhammond@skippinet.com.au> writes:
It might be possible to try and use build_ssl.py to locate the openssl directory, but this will still require that someone building it has Python built from source - I'm fairly sure that someone installing a Python binary will not have build_ssl.py, nor are they likely to have a suitable openssl directory or installation just "hanging around" either.
Yep - even if a Windows user has an appropriate development environment in general (and can build most standalone extensions with just a binary Python install), as you say the odds are pretty small they'd have an OpenSSL source tree around, with libraries built.
It is possible to build extensions on Windows using the mingw gcc toolchain. Users doing this may well have some or all of the gnuwin32 (http://gnuwin32.sf.net) utilities installed. Gnuwin32 includes openssl (both headers, link libraries, and DLLs). It seems to me a perfectly reasonable option for someone wanting to build the SSL extension to grab mingw and gnuwin32 openssl. I tried building with this config last night, but didn't have the time to deal with hacking the setup.py - I see someone else has covered this. I'll have another go with the new version when I get a chance.
At the same time, I suspect that only a small percentage of Windows users will want to rebuild the extension - rather they'll just want a binary installer, something not uncommon to be published for Windows users of many extension modules. So that pushes the problem upstream a bit where having a Python development tree might be more common or familiar.
Agreed. I assume Windows binary builds will be published, so it's only early adopters, or people who want to work with their own builds for some other reason, who might care. Paul.
participants (4)
-
Bill Janssen -
David Bolen -
Mark Hammond -
Paul Moore