Hi, On Fri, Dec 10, 2004 at 12:06:01PM +0000, Paul Moore wrote:

For most C extensions, the best free option is mingw.

Sorry, I was not aware that mingw supports the new VC7.1-type of runtime that is needed for the extension module to load with the official Python 2.4 distribution. Note that compiling with Microsoft's free compiler (after the huge downloads and the manual patching) worked just fine, so I think it is worth a note somewhere. Another note: can you report on whether building libpython24.a can be skipped for mingw? I'm thinking about the specific situation where we want on-site compilation of extension modules with a minimal number of things to install first. E.g. if we need to compile libpython24.a it means we need to fetch the Python sources themselves first. (A use case is for prorgams that dynamically produce and compile extension modules, as "weave" or the PyPy test suite do.) A bientot, Armin.

On Dec 10, 2004, at 1:05 PM, Phillip J. Eby wrote:

At 05:19 PM 12/10/04 +0000, Armin Rigo wrote:

...

Another note: can you report on whether building libpython24.a can be skipped for mingw? I'm thinking about the specific situation where we want on-site compilation of extension modules with a minimal number of things to install first. E.g. if we need to compile libpython24.a it means we need to fetch the Python sources themselves first.

Actually, no, you don't need the sources. See:

http://mail.python.org/pipermail/python-dev/2004-January/041676.html

for a script that builds libpython24.a from the python24.lib distributed with Python for Windows.

Shouldn't this be distributed with binary distributions of Python, to save people the trouble? Or, if it can be skipped, the procedure for doing a mingw build with the .lib should be documented and that'd be the end of it. -bob

Phillip J. Eby wrote:

The Python developers who produce the Windows binaries don't use mingw/cygwin, so this would put a maintenance burden on them.

If this were completely automatic (e.g. part of msi.py), I'd happily add all utilities needed to integrated this into the build process. For this to happen: 1. Somebody else needs to do the actual work. I.e. write a patch to msi.py that invokes all the necessary magic, and test that patch. 2. The patch should avoid manual configuration of the "use cygwin" case. I.e. it should try to find the cygwin on the path, or in their standard location, or use any other mechanism available to locate cygwin. It should produce an error message if cygwin tools cannot be found, and then proceed without out creating libpython24.a Regards, Martin

...

E.g. if we need to compile libpython24.a it means we need to fetch the Python sources themselves first.

Actually, no, you don't need the sources. See:

http://mail.python.org/pipermail/python-dev/2004-January/041676.html

for a script that builds libpython24.a from the python24.lib distributed with Python for Windows.

Previously I built libpython24.a; it can be downloaded from my website. See http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/cygwin.html --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

On Fri, 10 Dec 2004 20:40:10 +0000, Paul Moore wrote:

On Fri, 10 Dec 2004 17:19:21 +0000, Armin Rigo wrote:

...

Another note: can you report on whether building libpython24.a can be skipped for mingw?

A first attempt seems to almost work. There is a problem with structures, however - I get an error about unresolved references to _imp___Py_NoneStruct. My guess is that there's some issue with resolving references to data (as opposed to functions) from DLLs. I'll investigate further and then report back.

OK. After some more extensive investigation: 1. You cannot use python24.lib (ie, skip building libpython24.a). If you do, data values exported from python24.dll (specifically, Py_NoneStruct, which is quite important :-)) do not get exported correctly. I don't know exactly why - I guess it's something to do with how the MS LIB format works. 2. You can, however, link direct to python24.dll, which works fine. Unfortunately, there are 2 problems with this - first, distutils doesn't support it as things stand, and second, it isn't possible to get the full path of the Python DLL without using Mark Hammond's pywin32, or custom C code :-( (You need to get from sys.dllhandle to the filename, and the API to do this isn't exposed). 3. In some cases, an extension built with mingw still contains references to msvcrt (I think Martin has noted this). This seems *not* to be an issue. What is happening, is that any symbols referenced from user code are satisfied from msvcr71, and so match the Python DLL. However, if the mingw startup code uses symbols which have not been satisfied by the time all references from user code are resolved, then these references are satisfied from msvcrt. But I can't see any way that these references are going to do any harm, as they are startup code, which by its nature will have been written to work cleanly in this situation (linking like this is explicitly supported by mingw). Martin, as the CRT compatibility expert, does this sound reasonable? The mingw people certainly seem to think that the situation is OK... You could probably incorporate Philip Eby's script, posted here a while ago, which he mentioned earlier in this thread, into a setup.py so that if libpython24.a didn't already exist, the setup script would build it. This would give you a reasonably clean way of making sure it was present. Philip's script doesn't need anything but mingw installed, AIUI. Longer term, I'll see if I can put together a distutils patch to make it reference pythonXX.dll directly, so removing the need for libpythonXX.a in the future. Obviously, this is only going to be relevant for Python 2.5 and later, though. Sorry for the long mail, but I thought it would be useful to get the details in the archives... Paul.

Armin Rigo wrote:

In other words, if you want 3rd parties to compile Windows binaries for 2.4, tell them how.

The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke python setup.py bdist_wininst That's not too hard to do, I think. Regards, Martin

Armin Rigo wrote:

Hi Martin,

On Sat, Dec 11, 2004 at 12:45:01AM +0100, "Martin v. Löwis" wrote:

...

The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke

python setup.py bdist_wininst

That's not too hard to do, I think.

Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest time and money on Windows tools just to compile my extension module for Windows users...

Maybe we can set this up as a service? I have the compiler and could do something like doing a checkout, build, and upload new binary for a number of projects. What is needed is a simple but secure notification method. Probably I need one windows machine which is always online. ciao - chris -- Christian Tismer :^) mailto:tismer@stackless.com tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

On Sat, 11 Dec 2004 19:57:55 +0100, Christian Tismer wrote:

Armin Rigo wrote:

...

Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest time and money on Windows tools just to compile my extension module for Windows users...

First of all, I'm assuming this is a timing issue. If I understood your initial posting, your concern was that people wanted Windows build of your extension *now*, and it was going to take you time to make it available. That's a different issue from you not having the facilities to build the Windows installers at all, where you rely on 3rd parties making builds available. As Martin points out, ultimately the provision of Windows binaries is an issue for the extension project - is the demand high enough to justify the effort, can you find tools and/or a helper, etc etc. But the former issue (how quickly you can provide binaries, assuming that you will do so ultimately) is more relevant for python-dev. Specifically, because lack of binary extensions can be a barrier to take-up of the new version. Certainly, in the past, you could pretty much guarantee that there would be very few Windows users testing beta releases, because pywin32 binaries weren't available. With 2.4, I have at least one system I'd upgrade *now* but for the lack of a critical extension in binary form (I haven't yet had the time to try to adapt the build process to mingw for myself).

Maybe we can set this up as a service?

That sounds like a good idea. What I'd suggest is needed is a website where the following can take place: 1. People have a way of posting rquests for particular modules. 2. Installers can be uploaded to satisfy those requests. 3. There is somewhere to put step-by-step "build it yourself" instructions, using free components, so that people *without* access to VS.NET can make progress themselves. Obviously, if a particular extension can't be built with free compilers, then binaries or access to VS.NET are the only options. The installers should be clearly noted as having no warranty or support implied, to encourage people to offer binaries they have built without feeling that they are taking on a support burden. Conversely, as soon as "official" binaries are available from the extension project, the binaries available here should be removed (and replaced with a link to the official site) to reinforce the "unofficial" nature of the binaries provided here. The biggest potential issue with such a site is clearly validation. I've no idea how to make something like this work without it being a major virus risk. Which may, sadly, be enough to kill the idea :-( Paul.

On Sun, 12 Dec 2004 16:40:22 +0100, Martin v. Löwis wrote:

If none of your users volunteers to do the build for you, I would stop worrying about the Windows users.

Sorry, Martin. I understand your point, but I think you are not being realistic. I for myself took the decision to use only free tools for my own development, but I still have to suport my Windows customers. I can't force them to change to Linux. I don't own a copy of MSVC. Also, one of the reasons to choose a third part module is to save time. The moment I am required to recompile everything myself I'm losing this convenience. This of course impacts my ability to focus on my own work, and so the story goes. I'm not saying that module authors should work for free just to save me some time & hassle. It's fair if an author decides to release a Linux-only module. But again -- this is not realistic. The world has a lot of Windows users, and I depend on them for my own income. If I can't find a good set of Python tools for my projects, what should I do? Picking another language is not a choice, mind you :-) All in all, I sincerely hope that this discussion end up in a high note. I'm not qualified to talk about the particulars of C compilers & development environments, but as a Python user, I have hope that a good solution will be found to make the process of building Python extensions for Windows more convenient. The dream scenario is not to require recompiling, at least inside the same major release (2.4 to 2.5, for example). That would be really great. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com

On Sun, Dec 12, 2004 at 04:40:22PM +0100, "Martin v. L?wis" wrote:

Armin Rigo wrote:

...

Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest time and money on Windows tools just to compile my extension module for Windows users...

If you don't have Windows at all, you cannot create Windows installers for your users, anyway.

Actually you are wrong there! We cross compile our major application (about 500k lines of C/C++) on linux/x86 for various architectures, Windows/x86 being one of them. We create the Windows installer using the Nullsoft installer (which we run under Wine) and compile with a gcc / mingw cross compiler. This all works very well and means we can build all our architectures every night on a single machine. I'm following this thread with interest because we are considering embedding Python into this project, and I'm wondering whether we can cross compile python using mingw (almost certainly by the sound of it), but probably harder would be to make python module build and install system work cross-wise. -- Nick Craig-Wood -- http://www.craig-wood.com/nick