...

Is there or will there be support for python merge modules so we can include python in our installer?

I haven't planned any. Contributions are welcome.

...

But has there been thought towards migrating away from msilib and using platform standard tools such as wix (used by ms office, visual studio, etc.)?

Are you sure visual studio uses wix?

The visual studio regularly contributes to the wix toolset. They and the .net framework team contributed a lot of the wix bootstrapper (burn) foundational code.

I wouldn't call it a "platform standard tool".

It's becoming that as more and more projects are adopting it. It was considered to be shipped w/ VS 2010 but didn't make it due to management/time issues.

The Installer COM object is the platform standard mechanism, and that's what msilib uses.

Why maintain a lib when there's (better), free alternatives out there that are maintained by Microsoft itself? (okay, a group at Microsoft that works on their free time, but has significant contributions by many teams at Microsoft -- thus they have a vested interested in its success).

I really see no need to move away from that - it can create arbitrary MSI files.

Can it create merge modules? Transform files (e.g. localization)? Bootstrappers? There's a lot more to ms installers than the msi itself. Wouldn't it be easier to maintain an XML file than an entire lib dedicated to something that someone else has already solved?

If Python was starting at ground zero, and the choices were to create a library or to use WiX, the answer might have been different. However with a mature enough library to suit all the needs that anyone has been willing to author, it's certainly more work to create the WiX install and maintain it than it is to merely maintain the existing scripts.

Sounds reasonable.

As far as the possibility of distributing Python as a merge module? I'd recommend against it. Shared location merge modules are a maintenance nightmare, and private location merge modules may not offer the benefit you need. Better to just install the main Python msi as part of a suite with your installer, whether you build that installer in WiX and burn, or not.

I'm not familiar w/ shared location merge modules vs. private location merge modules. Are you referring to the possibility of having multiple python apps trying to use their own python versions? How relocatable is python? The maintenance nightmare isn't on your end (in my case), it would be on me to provide patches/upgrades. I do agree that patches w/ merge modules are a pain/nightmare. But what about providing both a merge module and msi so developers have a choice? I work on open source projects myself, and we always provide both a merge module and a normal msi installer. It's very little extra work (in WiX at least) to create both. But I can tell from this discussion that it would require changes in msilib and since I don't have the time to work on it, it would most likely not happen. Perhaps I could generate enough fervor in the community for a merge module, though, to drive msilib development. :p At any rate, where could I find the script used to generate the msi package? Perhaps I can translate it into WiX if I thought it'd be used (but why bother if no one's interested?).

On Thu, Feb 3, 2011 at 00:30, "Martin v. Löwis" <martin@v.loewis.de> wrote:

Another challenge with shared location merge modules is upgrades: the Python installer currently doesn't use stable component IDs; I think this would cause problems for users of the merge module. Providing stable component IDs is a challenge since it's difficult to version the files on disk.

Not sure why Michael thinks that a private location merge module would provide no benefits to the user of the merge module.

I hadn't thought it through fully, but the preceding paragraph really gets to the core of the problem. The maintenance nightmare is security updates for private location installations by third parties. The only MSI-friendly way to update that code is through releasing an updated merge module and having the consuming application also release an update that uses it. Since Python's components use fresh GUIDs each time, this would require a "major" upgrade; "minor" upgrades would cause Windows Installer to throw fits. Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components. For contrast, I don't see a shared-location merge module as offering benefits beyond a silently redistributable msi package. The shared location is better about following component code rules (re-use in private areas is an allowed gray area), and there are people out there who consider the reference counting through merge modules to be superior. I find the resulting complexity in the consuming package's installation to be more of a down-side.

...

I work on open source projects myself, and we always provide both a merge module and a normal msi installer. It's very little extra work (in WiX at least) to create both.

But what's the quality of these? Ideally, I'd like to create a single merge module which, at the option of the user of the merge module, produces either a shared or a private installation. Is that still only little extra work in Wix?

I've never tried to make a configurable merge module in WiX, but I think that's the only option if you want a single merge module to allow both. It should be a one-time authoring overhead with [1]. Using them is pretty straightforward within the Merge elements [2]. [1] http://wix.sourceforge.net/manual-wix3/wix_xsd_configuration.htm [2] http://wix.sourceforge.net/manual-wix2/wix_xsd_configurationdata.htm Michael

At work we keep the required stable UUIDs in an ConfigParser-format file checked in to the VCS for that purpose.

FWIW our build system uses WiX (v2) currently and if I where to redo it, I'd change to msilib and not WiX v3. But never change working systems.

No need to do that if you're using heat -- I haven't used WiX v2 so I can't speak to its relative merits, but v3.5 (which was just officially released) is pretty good and much more feature complete (according to rob mensching's blog). I'd recommend re-evaluating it. I'm not a Microsoft fanboy, btw. But I am getting my group (or trying to, at least) to migrate away from older, stale installer projects (e.g. Visual Studio will be dropping support for any further installer project improvements in the future) and into WiX because that's where the momentum is and it also keeps up-to-date w/ the latest msi changes. That and I was tired of the install looking like an intern did it (no offense to interns -- I was one once. (c: ). A good, polished product should (IMO) also have a good, polished installer. Especially since that's one of the customer's first views/impressions of your product.

Python.org supplied merge modules would be nice though, if the upgrade/security issue can be resolved.

Good to know I'm not entirely alone. (c:

I hadn't thought it through fully, but the preceding paragraph really gets to the core of the problem. The maintenance nightmare is security updates for private location installations by third parties. The only MSI-friendly way to update that code is through releasing an updated merge module and having the consuming application also release an update that uses it. Since Python's components use fresh GUIDs each time, this would require a "major" upgrade; "minor" upgrades would cause Windows Installer to throw fits.

That's exactly right and why I said earlier that patches w/ merge modules are a pain/nightmare. But I also said that the security patching w/ a merge module would then depend on the party who has integrated the merge module into their product -- not on you guys. I don't think most (or any) parties are trying to do incremental minor updates w/ python right now anyway. With us, we just want a single, well-tested and known python implementation installed w/ our product -- we basically don't trust the user to get the right version and install it correctly. You obviously can't hold their hand every step of the way, at some point you have to let go -- but this is one thing that we could possibly control.

Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components.

The general recommendation regarding msi packages is that you always, always do single-file components (one of the major reasons being for patching purposes). If you use WiX' heat application to autogenerate WiX files from directories, you'll see that it always produces single-file components.

For contrast, I don't see a shared-location merge module as offering benefits beyond a silently redistributable msi package. The shared location is better about following component code rules (re-use in private areas is an allowed gray area), and there are people out there who consider the reference counting through merge modules to be superior. I find the resulting complexity in the consuming package's installation to be more of a down-side.

I think a merge module (shared or private) generally reduces complexity unless you already have a bootstrapper for other packages. Including one in WiX is very simple (if you're already familiar w/ msi's): <Feature Id="Full"> <MergeRef Id="Python" /> </Feature> <Directory Id="TARGETDIR"> <Merge Id="Python" SourceFile="python.msm" Language="1033" DiskId="1" /> </Directory> If you use something like the VS installer projects, you just have to use the GUI to add a reference to it and you're done. A shared merge module might pose a problem if you want to have multiple python versions installed. At the least, you'd need to change the component guid w/ each major release (e.g. 2.x -> 3.x, but not 2.7 -> 2.8 -> 2.9, etc.). I'd recommend w/ sticking to a private one that doesn't modify the PATH (should you choose to create one) and then you're free to keep or change the component guids. Can python operate inside a sandboxed C:\Program Files\<My Product Here>\ directory?

I've never tried to make a configurable merge module in WiX, but I think that's the only option if you want a single merge module to allow both. It should be a one-time authoring overhead with [1]. Using them is pretty straightforward within the Merge elements [2].

[1] http://wix.sourceforge.net/manual-wix3/wix_xsd_configuration.htm [2] http://wix.sourceforge.net/manual-wix2/wix_xsd_configurationdata.htm

I wouldn't try to make it configurable (nor does it really need to be) beyond what it already should do -- that is, just be able to set the base target directory, which is already done for you. And as I just suggested, I wouldn't go for both -- declare the merge module to be private and that's that.

Can you please cite a source for that recommendation? Preferably some MSDN documentation.

http://msdn.microsoft.com/en-us/library/aa368269(v=vs.85).aspx http://wix.sourceforge.net/manual-wix3/add_a_file.htm Specifically, starting in bold, where it says "In general, you should restrict yourself to a single file per component. The Windows Installer is designed to support thousands of components in a single installer, so unless you have a very good reason, keep to one file per component. Every component must have its own unique GUID. Failure to follow these two basic rules can lead to many problems down the road when it comes to servicing." A little before that, it states: "The component element describes a set of resources (usually files, registry entries, and shortcuts) that need to be installed as a single unit. This is separate from whether the set of items consist of a logical feature the user can select to install ... While it may not seem like a big deal when you are first authoring your installer, components play a critical role when you decide to build patches at a later date." I didn't say it's a must, but experience lends you to following the recommendation.

...

The Installer COM object is the platform standard mechanism, and that's what msilib uses.

Why maintain a lib when there's (better), free alternatives out there that are maintained by Microsoft itself?

Using msilib is easier than using Wix. It's also more flexible. All you have to know is how the MSI schema works.

...

I really see no need to move away from that - it can create arbitrary MSI files.

Can it create merge modules? Transform files (e.g. localization)?

It could easily be extended to do so, in a straight-forward manner.

Bootstrappers?

Not sure what this is - if it is what I think it is, there is no need to (actually, there is no need to create the other two types of files, either, as it stands).

There's a lot more to ms installers than the msi itself. Wouldn't it be easier to maintain an XML file than an entire lib dedicated to something that someone else has already solved?

Definitely not. Python is easier than XML. If you think otherwise, feel free to provide a proof in the form of a Wix installation generator for Python. Regards, Martin

On Wed, 02 Feb 2011 17:30:43 -0800, "Hoyt, David" <hoyt6@llnl.gov> wrote:

...

Definitely not. Python is easier than XML.

I disagree.

Just as an FYI, I believe that most people in the Python community find XML much more of a pain than Python. Many of us (especially those of us who are not web developers) avoid XML whenever possible, and when we do have to deal with it we tend to abstract it behind easier to work with Python code. The obvious exception to that would be web templating languages, but I personally prefer to avoid those, as well :) -- R. David Murray www.bitdance.com

Martin v. Löwis <martin@v.loewis.de> wrote:

The Installer COM object is the platform standard mechanism, and that's what msilib uses. I really see no need to move away from that - it can create arbitrary MSI files.

I've used it to package UpLib for Windows -- see http://uplib.parc.com/hg/uplib/file/e29e36f751f7/win32/build-msi-installer.p.... I've generalized some of Martin's code to create a Packager class which supports non-Python pre and post install scripts. I found it much easier to use than WiX, which I also tried. Bill