There are two features which could be very useful.
1) If the install command had a '--force' option, which lets install ignore file dates, you could overwrite manipulated installations with the original files. Currently if there are changed some files for any reason, you have to find and delete them and then reinstall the package. (Distutils doesn't overwrite newer files.) (Using this feature you could also install an older version of the package.)
2) If you are using two or more different python versions on your computer, and you want to install a package which contains C-compiled extensions, then you have to delete the build directory before you can install the package with another version of python or you use build with the '--force' option. In any case you have to force distutils to rebuild all extensions because they were compiled using a wrong version of python.
If you only build this package once, this may be not a problem, but if you develop a package on your own, you have to compile it more often, and then you don't want to compile everything new to test it. You could use here different build directories, this is not too much work. But you could forget these options, so why doesn't let distutils do it for us. We simply have to change the default build directory to something like 'build_' + sys.version[0:3]
There is attached a patch for it. (build.patch)
(Using this, I found 'build_rpm' using a different version of python than I have used to start the setup. I probably never had spotted it, if both had used the same build directory.)
Kind regards
Rene Liebscher
diff -BurN --exclude=*.pyc --minimal distutils.orig/distutils/command/build.py distutils.patched/distutils/command/build.py --- distutils.orig/distutils/command/build.py Sat Jun 24 03:23:37 2000 +++ distutils.patched/distutils/command/build.py Tue Sep 12 12:05:46 2000 @@ -48,7 +48,7 @@ ]
def initialize_options (self): - self.build_base = 'build' + self.build_base = 'build_' + sys.version[0:3] # these are decided only after 'build_base' has its final value # (unless overridden by the user or client) self.build_purelib = None
On 12 September 2000, Rene Liebscher said:
- If the install command had a '--force' option, which lets install ignore file dates, you could overwrite manipulated installations with the original files.
Good idea, especially since the "install" command needs to get *more* paranoid about overwriting old files.
...OK, I've just made that change to install*.py. Boy, we need a better way to add an option to a whole family of commands. ;-(
Anyways, it works for me when installing the Distutils. Please give it a try on something that includes data, scripts, etc. Hmmm, maybe I'll give it a shot on my Distutil-ized IDLE... that's got modules, data files, and a script!
...OK, it works, but it revealed a subtle bug in my IDLE setup script. D'ohh! Oh well, this is an excuse to repost said setup script to python-dev and prod Guido about distutil-izing IDLE. ;-) Good demo of packaging a real Python app, if nothing else.
- If you are using two or more different python versions on your computer, and you want to install a package which contains C-compiled extensions, then you have to delete the build directory before you can install the package with another version of python or you use build with the '--force' option. In any case you have to force distutils to rebuild all extensions because they were compiled using a wrong version of python.
Hmmm, good point. I'm always wary of over-differentiation leading to unworkably long directory names, but I think this one is worthy. However, I don't like the idea of having multiple top-level "build" directories; I'd rather the sub-directories of "build" incorporated the Python version along with the platform info that they already include.
This means there should probably be a function somewhere to take a plain vanilla directory name and "differentiate" it -- eg. build/lib -> build/lib-1.6 or build/temp -> build/temp.linux-i586-1.6. Or something like that. Can you rework your patch to do this? I think the commads to hit are build_* and bdist_*.
Greg
Greg Ward wrote:
On 12 September 2000, Rene Liebscher said:
- If the install command had a '--force' option, which lets install ignore file dates, you could overwrite manipulated installations with the original files.
Good idea, especially since the "install" command needs to get *more* paranoid about overwriting old files.
...OK, I've just made that change to install*.py. Boy, we need a better way to add an option to a whole family of commands. ;-(
Anyways, it works for me when installing the Distutils. Please give it a try on something that includes data, scripts, etc. Hmmm, maybe I'll give it a shot on my Distutil-ized IDLE... that's got modules, data files, and a script!
...OK, it works, but it revealed a subtle bug in my IDLE setup script. D'ohh! Oh well, this is an excuse to repost said setup script to python-dev and prod Guido about distutil-izing IDLE. ;-) Good demo of packaging a real Python app, if nothing else.
- If you are using two or more different python versions on your computer, and you want to install a package which contains C-compiled extensions, then you have to delete the build directory before you can install the package with another version of python or you use build with the '--force' option. In any case you have to force distutils to rebuild all extensions because they were compiled using a wrong version of python.
Hmmm, good point. I'm always wary of over-differentiation leading to unworkably long directory names, but I think this one is worthy. However, I don't like the idea of having multiple top-level "build" directories; I'd rather the sub-directories of "build" incorporated the Python version along with the platform info that they already include.
This means there should probably be a function somewhere to take a plain vanilla directory name and "differentiate" it -- eg. build/lib -> build/lib-1.6 or build/temp -> build/temp.linux-i586-1.6. Or something like that. Can you rework your patch to do this? I think the commads to hit are build_* and bdist_*.
It is only build.py, all other command get their paths from build. I changed only the platform specific directories.
lib.linux2 -> lib-1.5.linux2 temp.linux2 -> temp-1.5.linux2
If you try this with distutils, you will not see any changes, because distutils is a pure-python package. (file: lib-x.x.patch)
----------------------------------- In the CVS are the source files for the windows installer. So everyone can create his own, I think most people would like to change the bitmap (PythonPowered) and use a more specific bitmap for their package. Creating the installer executable is not the problem, but to use it you have to change bdist_wininst.py. (If you only maintain one package this is not a problem but if there are more, you had to change it everytime you want to create an installer for the other package.)
So I added an 'install-exe' option to bdist_wininst, which can be used to specify the executable which is used as base for the installer. Using this you can have own installers for every package. (You could set it in setup.cfg.) This should also help to test new installers, I'm right, Thomas?
(file: wininst.patch)
-------------------------------------- Another problem:
python setup.py bdist --formats=zip,gztar
doesn't work. It deletes the build tree after creating the first archive.
There is a simple solution for that. First find out all commands to run. Then execute all commands, check for every command if it has to be run later again (eg. bdist_dumb with other format), if it has, check if there is a keep_tree option and set it.
This way you keep the tree for following runs, only the last one will remove the tree because there is no following command and the keep_tree will not be set.
(file: bdist.patch)
After this you could even run this command:
python setup.py bdist --formats=rpm,gztar,bztar,ztar,tar,wininst,zip
All, except rpm and wininst, would use the same tree.
Kind regards
Rene Liebscher
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/build.py distutils.patched/distutils/command/build.py --- distutils.orig/distutils/command/build.py Sat Jun 24 03:23:37 2000 +++ distutils.patched/distutils/command/build.py Wed Sep 13 13:58:56 2000 @@ -74,7 +74,7 @@ self.build_purelib = os.path.join (self.build_base, 'lib') if self.build_platlib is None: self.build_platlib = os.path.join (self.build_base, - 'lib.' + self.plat) + 'lib-' + sys.version[0:3] + '.' + self.plat)
# 'build_lib' is the actual directory that we will use for this # particular module distribution -- if user didn't supply it, pick @@ -89,7 +89,7 @@ # "build/temp.<plat>" if self.build_temp is None: self.build_temp = os.path.join (self.build_base, - 'temp.' + self.plat) + 'temp-' + sys.version[0:3] + '.' + self.plat) if self.build_scripts is None: self.build_scripts = os.path.join (self.build_base, 'scripts') # finalize_options ()
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/bdist_wininst.py distutils.patched/distutils/command/bdist_wininst.py --- distutils.orig/distutils/command/bdist_wininst.py Tue Sep 12 11:50:01 2000 +++ distutils.patched/distutils/command/bdist_wininst.py Wed Sep 13 18:05:13 2000 @@ -31,6 +31,8 @@ "do not compile .py to .pyo (optimized) on the target system"), ('dist-dir=', 'd', "directory to put final built distributions in"), + ('install-exe=', None, + "executable used as base for installer"), ]
def initialize_options (self): @@ -40,6 +42,7 @@ self.no_target_optimize = 0 self.target_version = None self.dist_dir = None + self.install_exe = None
# initialize_options()
@@ -59,6 +62,10 @@
self.set_undefined_options('bdist', ('dist_dir', 'dist_dir'))
+ if self.install_exe: + if not os.path.isfile(self.install_exe): + raise DistutilsFileError, "%s not found." % self.install_exe + # finalize_options()
@@ -193,8 +200,11 @@ # create_exe()
def get_exe_bytes (self): - import base64 - return base64.decodestring (EXEDATA) + if self.install_exe: + return open(self.install_exe,"rb").read() + else: + import base64 + return base64.decodestring (EXEDATA)
# class bdist_wininst
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/bdist.py distutils.patched/distutils/command/bdist.py --- distutils.orig/distutils/command/bdist.py Tue Sep 12 11:50:01 2000 +++ distutils.patched/distutils/command/bdist.py Wed Sep 13 15:39:08 2000 @@ -109,16 +109,25 @@
def run (self):
+ commands = [] + # which commands do we need for format in self.formats: try: - cmd_name = self.format_command[format][0] + commands.append(self.format_command[format][0]) except KeyError: raise DistutilsOptionError, \ "invalid format '%s'" % format
+ # execute commands + for i in range(len(self.formats)): + cmd_name = commands[i] sub_cmd = self.reinitialize_command(cmd_name) if cmd_name not in self.no_format_option: - sub_cmd.format = format + sub_cmd.format = self.formats[i] + # Do we need this command later again? Does it have a keep_tree + # option? If it has, keep the tree for next run of this command. + if hasattr(sub_cmd,"keep_tree") and (cmd_name in commands[i+1:]): + sub_cmd.keep_tree = 1 self.run_command (cmd_name)
# run()
[Rene]
In the CVS are the source files for the windows installer. So everyone can create his own, I think most people would like to change the bitmap (PythonPowered) and use a more specific bitmap for their package.
I would happily accept a better (general) bitmap!
Creating the installer executable is not the problem, but to use it you have to change bdist_wininst.py. (If you only maintain one package this is not a problem but if there are more, you had to change it everytime you want to create an installer for the other package.)
Recompiling the installer executable only to change the bitmap seems overkill to me. You could use win32api calls (if available) to replace the bitmap in the executable without need for a compiler, better would be to pack the bitmap into the installer archive at runtime. This could also be done on Linux.
Thomas
Thomas Heller wrote:
[Rene]
In the CVS are the source files for the windows installer. So everyone can create his own, I think most people would like to change the bitmap (PythonPowered) and use a more specific bitmap for their package.
I would happily accept a better (general) bitmap!
What about a kind of contest? Let's collect some drafts, put them on a web page and let the people decide which looks at best. Anyone who wants to manage this? (For people who want create such bitmaps some information. There are four dialogs, they can use the same bitmap or different. 1. "This Wizard will install %s on your computer. ..." 2. "Select python installation to use:" 3. "Click Next to begin the installation. ..." 4. "Click the Finish button to exit the Setup wizard." Maximum size seems to be 98(width)x155(height) pixels. At least are these the values in the install.rc file for the bitmap. )
Creating the installer executable is not the problem, but to use it you have to change bdist_wininst.py. (If you only maintain one package this is not a problem but if there are more, you had to change it everytime you want to create an installer for the other package.)
Recompiling the installer executable only to change the bitmap seems overkill to me. You could use win32api calls (if available) to replace the bitmap in the executable without need for a compiler, better would be to pack the bitmap into the installer archive at runtime. This could also be done on Linux.
Replacing the bitmap was bad example. What I wanted to say was that people sometimes want to use a modified installer and need a simple way to get bdist_wininst use this installer instead its default.
The new installer had to compiled only once. And this could even be done with the help of a short python script which uses distutils compiler classes. (I have already written such a script but it uses my new compiler interface (executable_filename(),link()) so you have to wait until distutils also supports it.)
kind regards
Rene Liebscher
On 14 September 2000, Rene Liebscher said:
It is only build.py, all other command get their paths from build. I changed only the platform specific directories.
lib.linux2 -> lib-1.5.linux2 temp.linux2 -> temp-1.5.linux2
If you try this with distutils, you will not see any changes, because distutils is a pure-python package. (file: lib-x.x.patch)
OK, thanks -- I reworked your patch a bit and checked it in. Note that with the recent changes to util.get_platform(), those directories are actually named (eg.) "lib-1.5-linux-i586" and "temp-1.5-linux-i586". (I changed your . to a -.)
Two things to note: * if we ever start byte-compiling at build-time, rather than install- time, then the Python version will matter for pure Python distributions * the "bdist" command also generates "platform-dependent" directory names; I can't think of any reason to put the Python version in them, but I could be missing something. It's a wee bit inconsistent to have "lib-1.5-linux-i586" next to "bdist.linux-i586". Arg! I just noticed more -/. confusion there. Now it's "lib.linux-i586-1.5".
In the CVS are the source files for the windows installer. So everyone can create his own, I think most people would like to change the bitmap (PythonPowered) and use a more specific bitmap for their package.
I agree, but it does seem silly to create a new executable for every bitmap. Why not fix things so you can supply a bitmap file? (But being able to specify the exe file seems useful in its own right, so don't throw that patch away!)
I'll let you and Thomas work it out, and Thomas can check it in when he's happy with it.
python setup.py bdist --formats=zip,gztar
doesn't work. It deletes the build tree after creating the first archive.
Son of a gun, so it doesn't. I thought I tested that... sigh...
There is a simple solution for that.
Unfortunately, the bug is deeper than that. Your patch is a nice optimization, but this should work without it (redoing the build and install for each format). I think the blame lies on bdist_dumb, but it needs some help from Command and Distribution to get everything right. Working on it now, but I need sleep...
If I get it working, then I'll see about applying your optimization patch.
Greg
On 14 September 2000, Rene Liebscher said:
Another problem:
python setup.py bdist --formats=zip,gztar
doesn't work. It deletes the build tree after creating the first archive.
OK, I've fixed the bug. The problem was that when you ask to reinitialize the "install" command, it doesn't reinitialize "install_lib", "install_headers", etc. Part of reinitializing a command is resetting its "have run" flag to 0, so on the second run of bdist_dumb, it looked like "install" had not run, but "install_lib" had. So nothing actually got installed, and the directory that was supposed to become the root of the archive didn't even exist.
The solution was to add a 'reinit_subcommands' flag to Distribution's 'reinitialize_command()' method. Making this work required promoting the "sub-commands" machinery -- which already had informal standing in the "install" command -- up to the Command base class. Now the Distutils have formal recognition of command "families", something that I've known for a while is needed, but hadn't gotten around to until now.
There is a simple solution for that. First find out all commands to run. Then execute all commands, check for every command if it has to be run later again (eg. bdist_dumb with other format), if it has, check if there is a keep_tree option and set it.
OK, I applied your patch as an optimization. It's not essential at all, but what the hell.
Also, I renamed --keep-tree to --keep-temp in the two bdist commands that had it (bdist_dumb and bdist_wininst). And I changed bdist_rpm so it, too, has a --keep-tree option (it used to have --clean/--no-clean). Now, all bdist commands must support the --keep-tree option, so there's no need for bdist.py to check if it exists before assigning to it.
Greg
participants (3)
-
Greg Ward -
Rene Liebscher -
Thomas Heller