[ANN] setuptools post install script for bdist_wininst
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi. Following the suggestion to use a post installation script in order to install dependencies for a package installed using bdist_wininst, I wrote this reusable script: http://paste.pocoo.org/show/260644/ In order to use it, the script must be declared as a script in the setup.py file. In order to ensure setuptools is installed, the same should be do with the ez_setup bootstrap script. The ez_post_install script will do: 1) Get the installed project name by parsing the bdist_wininst file name 2) Make sure setuptools is installed (if ez_setup is available) 3) Install all dependencies of the installed package, using easy_install 4) Optionally add shortcuts of all gui scripts in the Start Menu\Programs\<project name> special folder 5) Optionally add shortcuts of all gui scripts in the Desktop Problems ======== Since the post install script may take some time downloading and installing packages, the UI of the Windows installer is frozen, without giving user any feedback. This is a problem with the installer, since stdout and stderr are fully buffered. TODO ==== In case a dependency have extension code that must compiled, in 99% of the cases the post install script will fail, since the compiler or some library is not available. This should not happen; the post install script should, instead, check if in the package download_url there is a bdist_wininst executable, download it and execute it. Suggestions are welcome. Manlio Perillo -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyLlWAACgkQscQJ24LbaURBJwCgkU87bIiGGt2uj3qBcxZ1NIS6 07AAn2mXk9kQvtjgEXtWb3uwiQT69mwZ =l8M2 -----END PGP SIGNATURE-----
At 04:42 PM 9/11/2010 +0200, Manlio Perillo wrote:
This is a problem with the installer, since stdout and stderr are fully buffered.
You probably need to pop up some sort of dialog and direct stdout/stderr there. Alternately, you could run easy_install in a separate process as a console script.
TODO ====
In case a dependency have extension code that must compiled, in 99% of the cases the post install script will fail, since the compiler or some library is not available.
This should not happen; the post install script should, instead, check if in the package download_url there is a bdist_wininst executable, download it and execute it.
easy_install already does this, sort of: it downloads bdist_wininst executables and converts them into eggs, then installs the eggs.
Suggestions are welcome.
In general, I'm not sure that this sort of thing is wise to do in a bdist_wininst postinstall without some kind of user prompting, since: 1. A primary reason people have for using bdist_wininst installers is uninstall support, and this approach will leave dependencies behind. 2. Another big reason people have for using these installers instead of easy_install is that they don't want things being downloaded from the net, because e.g. they have no access or are behind a proxy, so this is just going to annoy them. I would strongly recommend having some sort of dialog that simply tells the user what dependencies are missing, and then asks if they would like to have them installed automatically. (So the user can then say no if they need uninstallability or have limited network access.)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Il 11/09/2010 20:10, P.J. Eby ha scritto:
At 04:42 PM 9/11/2010 +0200, Manlio Perillo wrote:
This is a problem with the installer, since stdout and stderr are fully buffered.
You probably need to pop up some sort of dialog and direct stdout/stderr there.
Not easy, but I will give it a try.
Alternately, you could run easy_install in a separate process as a console script.
TODO ====
In case a dependency have extension code that must compiled, in 99% of the cases the post install script will fail, since the compiler or some library is not available.
This should not happen; the post install script should, instead, check if in the package download_url there is a bdist_wininst executable, download it and execute it.
easy_install already does this, sort of: it downloads bdist_wininst executables and converts them into eggs, then installs the eggs.
Sorry, I was referring to "all in one" installers, that is, installers that also install external shared libraries. As an example: http://www.riverbankcomputing.com/static/Downloads/PyQt4/PyQt-Py2.6-gpl-4.7.... The PyQt4 installer put external DLLs (and some executables like designer.exe) in C:\Python26\Lib\site-packages\PyQt4\bin Is it possible to put required DLLs inside an egg? For PyQt4 there seems to be only the "all in one" installers and source distributions. I'm also having some issues with PyQt4 and easy_install.
Suggestions are welcome.
In general, I'm not sure that this sort of thing is wise to do in a bdist_wininst postinstall without some kind of user prompting, since:
1. A primary reason people have for using bdist_wininst installers is uninstall support, and this approach will leave dependencies behind.
As far as I know, all dependencies are correctly handled. Shortcuts are correctly removed by the uninstall.
2. Another big reason people have for using these installers instead of easy_install is that they don't want things being downloaded from the net, because e.g. they have no access or are behind a proxy, so this is just going to annoy them.
For me the main reason is the problem users have at using the command prompt and setting environment variables. easy_install is perfectly fine, IMHO, it only needs an usable GUI for Windows. I recently discovered that ActivePython have PyPM. I will do some tests.
I would strongly recommend having some sort of dialog that simply tells the user what dependencies are missing, and then asks if they would like to have them installed automatically. (So the user can then say no if they need uninstallability or have limited network access.)
Thanks for the suggestion. By the way, I tried to install pywin32, but I got: C:\>easy_install pywin32 Searching for pywin32 Reading http://pypi.python.org/simple/pywin32/ Reading http://sf.net/projects/pywin32 Reading http://sourceforge.net/project/showfiles.php?group_id=78018 Best match: pywin32 214 Downloading http://sourceforge.net/projects/pywin32/files/pywin32/Build%20214/py win32-214.win32-py2.6.exe/download Processing download error: Couldn't find a setup script in c:\docume~1\manlio\impost~1\temp\easy_install-ni9nay\download It is also strange that path components are also reported using short names... Thanks Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyL3SgACgkQscQJ24LbaUQEqgCglYphT5RQU1XBRN0dXuedqamx LgsAoIgsKfZsKzeygpvwP79uWVxmkSHX =B/Zu -----END PGP SIGNATURE-----
At 09:48 PM 9/11/2010 +0200, Manlio Perillo wrote:
Is it possible to put required DLLs inside an egg?
Yes, but they have to be adjacent to any Python extensions (.pyd's) that use them.
1. A primary reason people have for using bdist_wininst installers is uninstall support, and this approach will leave dependencies behind.
As far as I know, all dependencies are correctly handled. Shortcuts are correctly removed by the uninstall.
But the dependencies installed by easy_install will not get corresponding uninstall entries in the Windows control panel, and so will not be uninstalled when the user uninstalls the main application or library.
By the way, I tried to install pywin32, but I got:
AFAIK, pywin32 is not easy_install-able due to some post-install incompatibilities.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Il 11/09/2010 23:22, P.J. Eby ha scritto:
At 09:48 PM 9/11/2010 +0200, Manlio Perillo wrote:
Is it possible to put required DLLs inside an egg?
Yes, but they have to be adjacent to any Python extensions (.pyd's) that use them.
Good to know, thanks. But how can I create such a egg? And if this is possible, what do you think is the reasons why packages like pyqt do not release such eggs?
1. A primary reason people have for using bdist_wininst installers is uninstall support, and this approach will leave dependencies behind.
As far as I know, all dependencies are correctly handled. Shortcuts are correctly removed by the uninstall.
But the dependencies installed by easy_install will not get corresponding uninstall entries in the Windows control panel, and so will not be uninstalled when the user uninstalls the main application or library.
Ah, right; I was thinking to created shortcuts. It should not be hard to track installed dependencies, as an example using the --record option of easy_install, and then registering all installed files (and directories) with `file_created` and `directory_created` functions. The only problem is that created directories are not reported in the record, but if files are always listed ordered, it should not be a problem. Some other related questions: * Do you think you can change ez_setup.py so that the code used to check if setuptools must be installed is available as a separate function? I need to check if setuptools must be installed so that I can pop a message box. For normal package dependencies (and assuming setuptools is available) I can just use working_set.__contains__(dist) * Is it safe to call `ez_setup.use_setuptools` setting `to_dir` parameter to `sysconfig.get_python_lib()`? By default the setuptools egg is copied to the same directory where the bdist_wininst file is, and this is not safe since the user may delete these files. * Is it possible to get the ID of the main window of the bdist_wininst application? I need it because MessageBox function requires the ID of the parent window. It is possible to use the Desktop as parent, but this may cause problems (message box displayed in the wrong position, with incorrect focus, or who know) * Is it possible to modify bdist_wininst (in a future release) so that, when executing the post install script, the name of the project being installed is passed in argv[2]? This change does not break existing code. Thanks Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyMyUsACgkQscQJ24LbaUTAKACfQJDRgrPqkTo+IhTXTZ/zQyop 7Z0AniXpSCvyGpADk87Ybmu1RT0noBO1 =kSwm -----END PGP SIGNATURE-----
At 02:36 PM 9/12/2010 +0200, Manlio Perillo wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Il 11/09/2010 23:22, P.J. Eby ha scritto:
At 09:48 PM 9/11/2010 +0200, Manlio Perillo wrote:
Is it possible to put required DLLs inside an egg?
Yes, but they have to be adjacent to any Python extensions (.pyd's) that use them.
Good to know, thanks.
But how can I create such a egg?
In the same way that you'd create a distutils-based package that installs shared libraries next to Python extensions. (i.e., not easily.)
And if this is possible, what do you think is the reasons why packages like pyqt do not release such eggs?
Because major C extensions (e.g. wx, numpy, pywin32) tend to have special build requirements for their libraries that aren't handled well by the distutils, or the build process predated the distutils, or the build process wasn't originally designed to be used with Python, etc. etc.
It should not be hard to track installed dependencies, as an example using the --record option of easy_install, and then registering all installed files (and directories) with `file_created` and `directory_created` functions.
The only problem is that created directories are not reported in the record, but if files are always listed ordered, it should not be a problem.
That's not the only problem: if you install A, then B, and both depend on C, then if you uninstall A you will break B. You would have to also track shared use of dependencies. This situation is a big mess, and I haven't needed it for myself or any client as yet, so I've avoided the whole subject.
Some other related questions:
* Do you think you can change ez_setup.py so that the code used to check if setuptools must be installed is available as a separate function?
Grab the bit that looks like this: try: import pkg_resources except ImportError: # it's not installed else: try: pkg_resources.require('setuptools>='+version) except pkg_resources.VersionConflict, e: # older version is installed except pkg_resources.DistributionNotFound: # it's not installed else: # it's installed Season to taste. ;-)
For normal package dependencies (and assuming setuptools is available) I can just use working_set.__contains__(dist)
No, you need to use require() or resolve() or some other API that: 1. Checks versions, and 2. Handles recursive dependencies
* Is it safe to call `ez_setup.use_setuptools` setting `to_dir` parameter to `sysconfig.get_python_lib()`?
It's safe for setuptools, but whether it's safe in the context of a bdist_wininst I'm not sure.
By default the setuptools egg is copied to the same directory where the bdist_wininst file is, and this is not safe since the user may delete these files.
You could also fix that by calling easy_install(['--always-copy', 'setuptools']), in the case where you've had to install setuptools.
* Is it possible to get the ID of the main window of the bdist_wininst application? I need it because MessageBox function requires the ID of the parent window. It is possible to use the Desktop as parent, but this may cause problems (message box displayed in the wrong position, with incorrect focus, or who know)
* Is it possible to modify bdist_wininst (in a future release) so that, when executing the post install script, the name of the project being installed is passed in argv[2]?
This change does not break existing code.
I don't know the answers to these questions.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Il 12/09/2010 20:22, P.J. Eby ha scritto:
At 02:36 PM 9/12/2010 +0200, Manlio Perillo wrote: [...]
It should not be hard to track installed dependencies, as an example using the --record option of easy_install, and then registering all installed files (and directories) with `file_created` and `directory_created` functions.
The only problem is that created directories are not reported in the record, but if files are always listed ordered, it should not be a problem.
That's not the only problem: if you install A, then B, and both depend on C, then if you uninstall A you will break B. You would have to also track shared use of dependencies.
This situation is a big mess, and I haven't needed it for myself or any client as yet, so I've avoided the whole subject.
The simplest solutions are: * use py2exe or pyinstaller The problem is that there are some issues using them: - my scripts are handled by setuptools, and I use pkg_resource plugins and project data; this seems to not be supported - python >= 2.6 support have several issues * use virtualenv, and let the unistaller simply wipe out the virtual enviromnent The problem is that this requires a custom installer.
Some other related questions:
* Do you think you can change ez_setup.py so that the code used to check if setuptools must be installed is available as a separate function?
Grab the bit that looks like this:
[...]
Thanks.
For normal package dependencies (and assuming setuptools is available) I can just use working_set.__contains__(dist)
No, you need to use require() or resolve() or some other API that:
1. Checks versions, and 2. Handles recursive dependencies
Right, thanks.
* Is it safe to call `ez_setup.use_setuptools` setting `to_dir` parameter to `sysconfig.get_python_lib()`?
It's safe for setuptools, but whether it's safe in the context of a bdist_wininst I'm not sure.
By default the setuptools egg is copied to the same directory where the bdist_wininst file is, and this is not safe since the user may delete these files.
You could also fix that by calling easy_install(['--always-copy', 'setuptools']), in the case where you've had to install setuptools.
This is what I'm doing now.
[...]
Thanks Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyOEvgACgkQscQJ24LbaUTNQwCfcePklerVIA9+cZwqCXCsVJkL wioAn2m2Yv+nMJZsBLGcX2jrTX5s6w+e =u0aO -----END PGP SIGNATURE-----
At 02:03 PM 9/13/2010 +0200, Manlio Perillo wrote:
The simplest solutions are: * use py2exe or pyinstaller
The problem is that there are some issues using them: - my scripts are handled by setuptools, and I use pkg_resource plugins and project data; this seems to not be supported
I'm not sure what you mean. If your scripts import pkg_resources, it should be included in the resulting .exe or distribution.
- python >= 2.6 support have several issues
Is that just a general issue with py2exe/pyinstaller, or is there something specific to setuptools?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Il 13/09/2010 17:43, P.J. Eby ha scritto:
At 02:03 PM 9/13/2010 +0200, Manlio Perillo wrote:
The simplest solutions are: * use py2exe or pyinstaller
The problem is that there are some issues using them: - my scripts are handled by setuptools, and I use pkg_resource plugins and project data; this seems to not be supported
I'm not sure what you mean.
That my scripts are declared using gui_scripts entry points. Moreover my application make use of entry point based plugins.
If your scripts import pkg_resources, it should be included in the resulting .exe or distribution.
- python >= 2.6 support have several issues
Is that just a general issue with py2exe/pyinstaller, or is there something specific to setuptools?
pyinstaller specific: http://www.pyinstaller.org/wiki/Python26Win and (about setuptools support) http://www.pyinstaller.org/wiki/FAQ#Misc Regards Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyOkAUACgkQscQJ24LbaUQCPACfV5HiHlf9ruGVLiTciKG0BJvu WdMAn3xcbiIaQ9GLZYUGU/d048SkBTXa =Rb+8 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Il 11/09/2010 20:10, P.J. Eby ha scritto:
At 04:42 PM 9/11/2010 +0200, Manlio Perillo wrote:
This is a problem with the installer, since stdout and stderr are fully buffered.
You probably need to pop up some sort of dialog and direct stdout/stderr there. Alternately, you could run easy_install in a separate process as a console script.
By the way, I'm not sure to understand how running easy_install in a separate process would solve the installer GUI freeze problem. It should probably be the installer to execute the post install script in non blocking mode.
[...]
Thanks Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyMwgIACgkQscQJ24LbaUScqQCeK3I7cXfDfuhrdytBmbSuA2bI /QsAoJW1pBgchIjOPvi/S5+TMQHd+I+d =KuaC -----END PGP SIGNATURE-----
participants (2)
-
Manlio Perillo -
P.J. Eby