Thinking about a Windows Installer
Hi, I'm currently thinking about a bdist_winexe command for distutils. (Is there a better name?) This command will create a single executable file similar to those created with Wise or Installshield installers (except that you don't need to have these _commercial_ programs installed). This exe-File consists of a C-program embedding python, appended is a zip-compatible archive of the modules to distribute, the setup.py script, as well as an ini-file "setup.ini" or something like that which contains the distribution meta-data in a format easily readable by the C-program. This exe-File, when run, simply unpacks the archive into a temporary directory, displays a nice GUI, runs the setup.py script, and cleans up the temp-dir. (The design is inspired by previous work: Gordons installer, squeeze and whatever) Comments? Thomas Heller
On 17 May 2000, Thomas Heller said:
I'm currently thinking about a bdist_winexe command for distutils. (Is there a better name?)
How about "bdist_wininst"?
This command will create a single executable file similar to those created with Wise or Installshield installers (except that you don't need to have these _commercial_ programs installed).
There is some sort of informal arrangement that lets Python developers use Wise to create installers of free Python extensions for free. I think the same arrangement lets Guido create Python installers for free. So we probably shouldn't toss Wise out the window just because it's commercial; it might be a good (and zero-cost, if not entirely free) option for Python-on-Windows developers.
This exe-File consists of a C-program embedding python, appended is a zip-compatible archive of the modules to distribute, the setup.py script, as well as an ini-file "setup.ini" or something like that which contains the distribution meta-data in a format easily readable by the C-program.
Whether you want to embed Python really depends on your audience. Let's face it, installing Python on Windows is not that hard. If a programmer wants to use some collection of modules, it's not too bad to say, "Download and install Python, then download and install the module distribution". Painless * 2 is not too bad. For end-user applications, though, embedding a Python interpreter (and however much of the standard library as is needed) is probably the right thing to do. I think it should be up to the module developer/packager to decide he distributes a "Python-less" installer or a big fat one (or both). Now, if you *do* end up writing your own installer GUI (instead of using Wise), how much of it can be written in Python rather than C? I assume doing this would require the presence of the Win32 extensions, so "painless * 2" might become "painless * 3". Hmmm... I'd be reluctant to make someone install Python *and* the Win32 stuff just to run an installer.
This exe-File, when run, simply unpacks the archive into a temporary directory, displays a nice GUI, runs the setup.py script, and cleans up the temp-dir.
I think the mode of operation should be as close to the other bdist commands as possible: distribute a pseudo-installation tree and just copy it in place. That is, the setup script shouldn't enter into it here, *unless* you need it to extract package meta-data. (Unless that's put into another file when you run "bdist_wininst".) Greg -- Greg Ward - just another /P(erl|ython)/ hacker gward@python.net http://starship.python.net/~gward/ All programmers are playwrights and all computers are lousy actors.
From: "Greg Ward" <gward@python.net> To: <Distutils-SIG@python.org> Sent: Saturday, May 20, 2000 4:41 PM Subject: Re: [Distutils] Thinking about a Windows Installer
On 17 May 2000, Thomas Heller said:
I'm currently thinking about a bdist_winexe command for distutils. (Is there a better name?)
How about "bdist_wininst"? OK.
This command will create a single executable file similar to those created with Wise or Installshield installers (except that you don't need to have these _commercial_ programs installed).
There is some sort of informal arrangement that lets Python developers use Wise to create installers of free Python extensions for free. I think the same arrangement lets Guido create Python installers for free. So we probably shouldn't toss Wise out the window just because it's commercial; it might be a good (and zero-cost, if not entirely free) option for Python-on-Windows developers.
OK, will submit an 'bdist_wise' sometime. Or maybe 'bdist_wisescript' because it will create an xxx.wse script, which has to be compiled with Wise32.exe. (See below for more about Wise).
This exe-File consists of a C-program embedding python, appended is a zip-compatible archive of the modules to distribute, the setup.py script, as well as an ini-file "setup.ini" or something like that which contains the distribution meta-data in a format easily readable by the C-program.
Whether you want to embed Python really depends on your audience. Let's face it, installing Python on Windows is not that hard. If a programmer wants to use some collection of modules, it's not too bad to say, "Download and install Python, then download and install the module distribution". Painless * 2 is not too bad.
Sure. But: Windows users are (normally) not used to the command line (neither will they read the README file :-). They are used to installer executables.
For end-user applications, though, embedding a Python interpreter (and however much of the standard library as is needed) is probably the right thing to do. I think it should be up to the module developer/packager to decide he distributes a "Python-less" installer or a big fat one (or both).
I did not think about end-user applications here. Is distutils headed in this direction after all?
Now, if you *do* end up writing your own installer GUI (instead of using Wise), how much of it can be written in Python rather than C? I assume doing this would require the presence of the Win32 extensions, so "painless * 2" might become "painless * 3". Hmmm... I'd be reluctant to make someone install Python *and* the Win32 stuff just to run an installer.
This exe-File, when run, simply unpacks the archive into a temporary directory, displays a nice GUI, runs the setup.py script, and cleans up the temp-dir.
I think the mode of operation should be as close to the other bdist commands as possible: distribute a pseudo-installation tree and just copy it in place. That is, the setup script shouldn't enter into it here, *unless* you need it to extract package meta-data. (Unless that's put into another file when you run "bdist_wininst".)
I was more thinking about a zip-selfextractor, extracting the archive into the python installation directory automatically. The GUI is/will be not more than a dialog box displaying some meta-info (name, version,...) of the installed modules. The GUI is written in C. So it is not neccessary to embed python (Maybe for compiling pyc files?) More about Wise: Wise is a full fledged windows installer. These programs handle the following and more: - Check the access rights to install the program and its utilities - Let the user select an installation directory - Create shortcuts in the start menu so that the user can start the program easily - Care about incrementing the ref-count of shared files (usually dlls) - Create file associations for program specific data files - Install accessories needed for the program: database access, device drivers, system services - Create registry entries - Create some meta data for uninstallation and install an uninstaller - Care about replacing in-use files (common under windows) Nearly all of this is not needed for python modules and extensions, and if it is, IMO it should be handled by distutils. The price to pay for this flexibility is that the overhead of an empty wise script to the final installer program is about 130 kBytes (which may be MUCH for a collection of a handfull python modules), while a zip-selfextractor can be 20 - 30 kB. Thomas
On 23 May 2000, Thomas Heller said:
OK, will submit an 'bdist_wise' sometime. Or maybe 'bdist_wisescript' because it will create an xxx.wse script, which has to be compiled with Wise32.exe.
Great! A bit of calibration: "bdist_rpm" exists mainly to create the .spec file, but at the end of the job it runs "rpm -b..." for your convenience. It has --spec-only, --source-only, and --binary-only options for finer control. I think "bdist_wise" should do roughly the same: create the .wse file, and then (optionally, but by default) compile it with Wise32.exe.
Whether you want to embed Python really depends on your audience. Let's face it, installing Python on Windows is not that hard. If a programmer wants to use some collection of modules, it's not too bad to say, "Download and install Python, then download and install the module distribution". Painless * 2 is not too bad.
Sure. But: Windows users are (normally) not used to the command line (neither will they read the README file :-). They are used to installer executables.
No no, I'm not arguing for a command-line installer. (Except for the hard-code Pythonistas who want to build everything from source, and think the current Distutils interface on Windows is just fine.) I'm arguing that Joe's Handy Python Modules should *just* include joes/module1.py, joes/module2.py, and related files -- ie. pretty much what goes into other Distutils-generated built distributions ("dumb" tarballs or zip files, or "smart" binary RPMs). The boundary between Joe's Handy Python Modules and Jim's Humongous Python-Based Application (which probably should include an embedded Python interpreter and library) is fuzzy -- I'm not sure where eg. NumPy falls. That should be the developer's decision to make. (Hmmm, now why did the name "Jim" come to mind when thinking of humongous Python-based applications...)
I did not think about end-user applications here. Is distutils headed in this direction after all?
Well, yes, eventually. We have to keep it in mind. Support for "building" and installing scripts is already partly there. (When someone can download Zope and run "python setup.py install", and setup.py is a reasonably normal Distutils-style setup script, then I will hang up my keyboard and go home.) [I outline my vision]
I think the mode of operation should be as close to the other bdist commands as possible: distribute a pseudo-installation tree and just copy it in place. That is, the setup script shouldn't enter into it here, *unless* you need it to extract package meta-data. (Unless that's put into another file when you run "bdist_wininst".)
[Thomas describes his]
I was more thinking about a zip-selfextractor, extracting the archive into the python installation directory automatically. The GUI is/will be not more than a dialog box displaying some meta-info (name, version,...) of the installed modules. The GUI is written in C. So it is not neccessary to embed python (Maybe for compiling pyc files?)
We're not contradicting each other. By "mode of operation" I was more concerned with how "installers" or "built distributions" or whatever are created, and what's included in them, not how the user uses them. Of *course* on Windows it should be an executable -- that's the local convention. And if a self-extracting zip file is the best way to do it, then do it! A few questions/comments: * the default installation directory should obviously be where Python will find it: sys.prefix. So you'll either have to run "python -c 'import sys; print sys.prefix'" or get this information from the registry. I'm guessing the latter is preferable and doable -- as long as you can put arbitrary code into the self-extractor. * what if there are multiple Python installations -- are they sufficiently differentiated in the registry? Eg. I might have Python 1.5.2 and 1.6a2 installed, and I would have to choose which one to install to. What if it's a question of 1.6a1 vs 1.6a2 -- are Python's registry entries sufficently fine-grained to resolve this? * should we allow writing to a different directory than the default (sys.prefix)? (This would require writing a .pth file in sys.prefix.) Or is this getting too close to a "real" installer? * how would the actual installer be created? I'm hoping that the CCompiler abstraction would be good enough -- just create yourself a CCompiler (which could be a MSVCCompiler, BCCompiler, Mingw32CCompiler, ...) object, compile a few things, and link an executable. * interesting data point/existence proof: the Java installer- generator InstallAnywhere let me create an executable Windows installer *entirely* on my Solaris box -- I think it carries around a "stub" executable, and somehow generates the real executable by munging it with my Java code (and other stuff to install). Needless to say, this is *very* cool; it is also an incredibly fat, slow, and heavyweight product. If you can let me generate Windows installers under Solaris or Linux, I'll buy you *two* beers at the next Python conference. (I already owe you one.) ;-)
More about Wise: Wise is a full fledged windows installer. These programs handle the following and more: - Check the access rights to install the program and its utilities - Let the user select an installation directory - Create shortcuts in the start menu so that the user can start the program easily - Care about incrementing the ref-count of shared files (usually dlls) - Create file associations for program specific data files - Install accessories needed for the program: database access, device drivers, system services - Create registry entries - Create some meta data for uninstallation and install an uninstaller - Care about replacing in-use files (common under windows) Nearly all of this is not needed for python modules and extensions, and if it is, IMO it should be handled by distutils.
Agreed, except for the somewhat mythical Jim's Humongous Python-Based Application -- there, all of these "real installer" problems are undoubtedly a concern. That's not a big concern for Distutils 1.0, though -- unless bdist_wise is really easy, I would focus on bdist_wininst, as it sounds like it would be a lot lighter weight, and not require any additional software other than Python/Distutils. Wow, this sounds *neat*! And I thought being able to build RPMs was cool (no offense, Harry...) Greg -- Greg Ward - Linux nerd gward@python.net http://starship.python.net/~gward/ I wonder if I ought to tell them about my PREVIOUS LIFE as a COMPLETE STRANGER?
From: "Greg Ward" <gward@python.net>
(When someone can download Zope and run "python setup.py install", and setup.py is a reasonably normal Distutils-style setup script, then I will hang up my keyboard and go home.) Does not sound impossible, because: Zope is not more than some python-modules and some c-extensions. Well, python itself is also included (on Windows, have to check this on unix). Mark Hammonds win32all extensions are more of a challange (but that would not interest YOU very much, until they are ported to *nix ;-)
A few questions/comments:
* the default installation directory should obviously be where Python will find it: sys.prefix. So you'll either have to run "python -c 'import sys; print sys.prefix'" or get this information from the registry. I'm guessing the latter is preferable and doable -- as long as you can put arbitrary code into the self-extractor. I think it should use the python installation directory from the registry. How should the installer find python otherwise?
* what if there are multiple Python installations -- are they sufficiently differentiated in the registry? Eg. I might have Python 1.5.2 and 1.6a2 installed, and I would have to choose which one to install to.
Would read the registry entries and ask the user which version to use. If this is possible at all: .pyc files or compiled c-extensions are specific to the python version (Not necessarily for pyc files, but see the current discussion on python-dev). Probably the default should be the python version used to build the installation. Question: The installer should probably compile the .py files to .pyc/.pyo? If this has to be done, we have no other choice than to embed (or call) python.
What if it's a question of 1.6a1 vs 1.6a2 -- are Python's registry entries sufficently fine-grained to resolve this?
No. Registry entries are for 1.5 or 1.6.
* should we allow writing to a different directory than the default (sys.prefix)? (This would require writing a .pth file in sys.prefix.) Or is this getting too close to a "real" installer?
Not complicated.
* how would the actual installer be created? I'm hoping that the CCompiler abstraction would be good enough -- just create yourself a CCompiler (which could be a MSVCCompiler, BCCompiler, Mingw32CCompiler, ...) object, compile a few things, and link an executable.
build_wininst will NOT generate c-code needed to be compiled.
* interesting data point/existence proof: the Java installer- generator InstallAnywhere let me create an executable Windows installer *entirely* on my Solaris box -- I think it carries around a "stub" executable, and somehow generates the real executable by munging it with my Java code (and other stuff to install). Needless to say, this is *very* cool; it is also an incredibly fat, slow, and heavyweight product. If you can let me generate Windows installers under Solaris or Linux, I'll buy you *two* beers at the next Python conference. (I already owe you one.) ;-)
Yes, I'm heading in this direction. build_wininst will certainly append metadata and files to be installed to a prebuilt stub program. Thomas
On 25 May 2000, Thomas Heller said:
Zope is not more than some python-modules and some c-extensions. Well, python itself is also included (on Windows, have to check this on unix).
And the Linux kernel is some C files. ;-) (BTW, I believe Unix binary distributions include Python, but I've only built Zope from source.)
Mark Hammonds win32all extensions are more of a challange (but that would not interest YOU very much, until they are ported to *nix ;-)
Ahh, I hadn't thought about Mark's stuff. You're right, must be my Unix bias.
I think it should use the python installation directory from the registry. How should the installer find python otherwise?
Good point. My understanding is that we can assume "installation directory from the registry" == sys.prefix always. True?
Would read the registry entries and ask the user which version to use. If this is possible at all: .pyc files or compiled c-extensions are specific to the python version (Not necessarily for pyc files, but see the current discussion on python-dev).
Oohh, ick! That affects *all* built distributions -- RPMs, Debian packages, FreeBSD packages, dumb tarballs, executable installers, *everything*. I totally forgot about that. This means that we need to record the Python version (and/or the API version and marshall magic number) when we create the built distribution. Damn... wonder what the best way to do that is. I'm reluctant to jam one more piece of information into the filename: it's ugly, it's fragile, and it's may not be possible to retrieve when it's needed. I was just about to kill off the kludgey "package_data" file that Harry Gebel introduced to make bdist_rpm work... maybe something like that is needed after all, but to convey information from the packager's installation to the installer when running on the end-user's system.
Probably the default should be the python version used to build the installation.
Yes.
Question: The installer should probably compile the .py files to .pyc/.pyo? If this has to be done, we have no other choice than to embed (or call) python.
The way things are going is this: * the bdist_* command runs the install command, "redirecting" the installation into an alternate location. (It does this by setting the 'root' option, eg. to "build/bdist.linux2-i586/rpm", or "build/bdist.nt/wininst".) * by default, install_lib (called by install) byte-compiles .py files. (Soon, it'll probably create both .pyc and .pyo.) Thus, .pyc (and maybe .pyo) files will be included in the built distribution for free. This makes the installer's job easier, less prone to mysterious errors, etc. etc. Apart from increasing the size of the installer by a factor of ~2-3, it's a win. I realize this is back-pedalling on a decision made many moons ago, the result of which is that byte-compilation -- when building from source! -- is done at install-time rather than build-time. I have been known to change my mind six times before breakfast, though, so this is not a huge departure for me.
build_wininst will NOT generate c-code needed to be compiled. [...] Yes, I'm heading in this direction. build_wininst will certainly append metadata and files to be installed to a prebuilt stub program.
I hate to sound like I want *everything*, but it might be nice to make compiling the installer an option. As long as the stub isn't too big (30-50k?), getting a workable stub is more important -- especially if it lets me generate Windows installers on my Linux box! Greg -- Greg Ward - just another Python hacker gward@python.net http://starship.python.net/~gward/ I appoint you ambassador to Fantasy Island!!!
participants (2)
-
Greg Ward -
Thomas Heller