Building a Windows MSI for Python /trunk
Hi all, I got an MSI build working on my WinXP VM just now, and I wanted to touch base with whomever it is that is maintaining this (wonderful!) set of scripts... I ran into three problems, and I managed to figure out two of them; the third wasn't fatal. Note, the diff of my fixed checkout is attached. First, the script that finds & builds the external dependencies has two minor problems. * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in tcl8.* and tk8.* to grab the license text. I changed the glob strings appropriately and that seemed to work. * Tix isn't downloaded/installed/built automatically like everything else, and msi.py looks for its license file, too. I just removed the Tix reference. I can't figure out how to build Tix appropriately; any tips? Second, the buildmsi.bat file refers to python26a3.hhp instead of python27a0.hhp. Third, I could not get _tkinter to build properly, although it wasn't fatal to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although tcl85g.lib existed. Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I had to figure out with google -- things like nasm/nasmw, for example. Are these documented somewhere, or would it be helpful to document them? I think I had to install: - Microsoft HTML Help Compiler - cygwin with make and python2.5 to build the docs - nasm (and copy nasm.exe to nasmw.exe) - cabarc Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;) I'd love to get this build process working completely automatically and 100% correctly, too. Hat tip to Trent Nelson, who helped me figure out where the scripts are and what other things I needed... cheers, --titus -- C. Titus Brown, ctb@msu.edu
First, the script that finds & builds the external dependencies has two minor problems.
* it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in tcl8.* and tk8.* to grab the license text. I changed the glob strings appropriately and that seemed to work.
Not sure what "it" is that put it there; perhaps you are referring to the Tools/buildbot scripts? Unfortunately, the naming convention that these scripts establish doesn't quite work, as Tix would fail to build under these conventions. So in my own checkout, I manually renamed the trees, and that's what msi.py refers to.
* Tix isn't downloaded/installed/built automatically like everything else, and msi.py looks for its license file, too. I just removed the Tix reference. I can't figure out how to build Tix appropriately; any tips?
See above. I keep forgetting how to build Tix; one set of command lines is in PCbuild/build_tkinter.py.
Second, the buildmsi.bat file refers to python26a3.hhp instead of python27a0.hhp.
Yes, that script hasn't been updated for a long time, ever since we stopped having automated builds. Ideally, the script would find the Python version on its own.
Third, I could not get _tkinter to build properly, although it wasn't fatal to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although tcl85g.lib existed.
You'll need to build release versions of Tcl/Tk/Tix.
Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I had to figure out with google -- things like nasm/nasmw, for example. Are these documented somewhere, or would it be helpful to document them?
See PCbuild/readme.txt.
Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)
Thanks, fixed.
I'd love to get this build process working completely automatically and 100% correctly, too.
Hat tip to Trent Nelson, who helped me figure out where the scripts are and what other things I needed...
Feel free to submit patches. There may be a misunderstanding, though, in that buildmsi.bat isn't actually used for anything, and isn't "meant" to be run by users, but instead by build slaves. One consequence is that these scripts build in debug mode (perhaps except for buildmsi), whereas end users would typically want to build in release mode. Another consequence is that different committers actually use different procedures. Trent created much of the Tools/buildbot setup, so he is obviously in favor of that strategy. Christian Heimes created PCbuild/build_tkinter, so he would probably prefer to use that instead. When I took over Windows builds from Tim Peters, PCbuild/readme.txt was the official reference, and I try to stick to that. Regards, Martin
On Thu, Nov 26, 2009 at 09:54:35AM +0100, "Martin v. L?wis" wrote:
First, the script that finds & builds the external dependencies has two minor problems.
* it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in tcl8.* and tk8.* to grab the license text. I changed the glob strings appropriately and that seemed to work.
Not sure what "it" is that put it there; perhaps you are referring to the Tools/buildbot scripts?
Yes, sorry, Tools/buildbot/external*.bat, as in the diff.
Unfortunately, the naming convention that these scripts establish doesn't quite work, as Tix would fail to build under these conventions. So in my own checkout, I manually renamed the trees, and that's what msi.py refers to.
OK, thanks!
* Tix isn't downloaded/installed/built automatically like everything else, and msi.py looks for its license file, too. I just removed the Tix reference. I can't figure out how to build Tix appropriately; any tips?
See above. I keep forgetting how to build Tix; one set of command lines is in PCbuild/build_tkinter.py.
Ok, thanks!
Second, the buildmsi.bat file refers to python26a3.hhp instead of python27a0.hhp.
Yes, that script hasn't been updated for a long time, ever since we stopped having automated builds.
Ideally, the script would find the Python version on its own.
Right...
Third, I could not get _tkinter to build properly, although it wasn't fatal to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although tcl85g.lib existed.
You'll need to build release versions of Tcl/Tk/Tix.
Yes, I saw a reference to that online, but I wasn't sure if that was the problem -- is the naming convention really that 'tcl85g.lib' is the debug lib!?
Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I had to figure out with google -- things like nasm/nasmw, for example. Are these documented somewhere, or would it be helpful to document them?
See PCbuild/readme.txt.
OK, thanks.
I'd love to get this build process working completely automatically and 100% correctly, too.
Hat tip to Trent Nelson, who helped me figure out where the scripts are and what other things I needed...
Feel free to submit patches. There may be a misunderstanding, though, in that buildmsi.bat isn't actually used for anything, and isn't "meant" to be run by users, but instead by build slaves.
Well, yes, although wouldn't we want the scripts to run without any user editing, for the build slaves?
One consequence is that these scripts build in debug mode (perhaps except for buildmsi), whereas end users would typically want to build in release mode.
Right. Is there a reason the buildbot builds were stopped? I've added binary file uploads to pony-build, so that platform-specific files can be uploaded to build results -- see, e.g. http://lyorn.idyll.org/ctb/pb-dev/p/python/10768/files/ or http://lyorn.idyll.org/ctb/pb-dev/p/pygr/10805/files/ and I thought daily -latest builds for Python latest on Windows and Mac would be a good demo... Incidentally, the pony-build results & file upload scripts can be used from within buildbot, if people are interested.
Another consequence is that different committers actually use different procedures. Trent created much of the Tools/buildbot setup, so he is obviously in favor of that strategy. Christian Heimes created PCbuild/build_tkinter, so he would probably prefer to use that instead. When I took over Windows builds from Tim Peters, PCbuild/readme.txt was the official reference, and I try to stick to that.
OK, I see. Thanks! So if I can bring the scripts into concordance with that it might be valuable? --titus -- C. Titus Brown, ctb@msu.edu
You'll need to build release versions of Tcl/Tk/Tix.
Yes, I saw a reference to that online, but I wasn't sure if that was the problem -- is the naming convention really that 'tcl85g.lib' is the debug lib!?
Yes; Tcl apparently follows Unix conventions here rather than Windows conventions (where the debug build would end with "d").
Feel free to submit patches. There may be a misunderstanding, though, in that buildmsi.bat isn't actually used for anything, and isn't "meant" to be run by users, but instead by build slaves.
Well, yes, although wouldn't we want the scripts to run without any user editing, for the build slaves?
Sure. For the scripts that are in actual use, that's also the case. I stopped working on buildmsi because it was just too tedious to get right.
Is there a reason the buildbot builds were stopped?
It was too difficult to get right, plus nobody was interested in using the daily MSI files. At some point, it was broken for several months (IIRC), with nobody reporting that breakage. So when we noticed, we just turned the service off.
and I thought daily -latest builds for Python latest on Windows and Mac would be a good demo...
I'll guess that you will find the same: nobody is really interested in *using* daily builds of Python.
OK, I see. Thanks! So if I can bring the scripts into concordance with that it might be valuable?
Most certainly, yes. Regards, Martin
"Martin v. Löwis" <martin@v.loewis.de> writes:
It was too difficult to get right, plus nobody was interested in using the daily MSI files.
At some point, it was broken for several months (IIRC), with nobody reporting that breakage. So when we noticed, we just turned the service off.
Sounds right - the daily MSIs were built on my slave from Sep 2007 into 2008 somewhere, but it had been failing for 6 months or more when the task was turned off last December. I'm not sure precisely when the failures started (the buildbot status page was also down for a while starting in March), but it had been failing for a long time. I believe the failures at the time were tcl related - maybe some of the MSI buildbot-related scripts getting out of date with ongoing development or something. Sounds like it might be the same stuff Titus is running into. It didn't seem worth the effort to track down at the time, given that nobody had seemed to notice the lack of builds. -- David
participants (3)
-
"Martin v. Löwis" -
C. Titus Brown -
David Bolen