bdist_wininst and Vista User Account Control (UAC)
The pywin32 extensions require (well, prefer) administrative access during installation - certain files are copied to the System32 directory and the registry at HKEY_LOCAL_MACHINE is written to. Also, if I understand correctly, if Python happened to be installed into "\Program Files", admin access would be required to create any files in that directory tree - I'm not sure what permissions the \PythonXX directory are created with, but its not unreasable to assume that some shops might choose to secure that directory similarly to "\Program Files".
The simplest way to achieve this for bdist_wininst installations is to include some magic in a "manifest". I've confirmed that once this magic is added, programs created by bdist_wininst get the little "shield" icon overlay and prompt for elevation before starting the executable. A problem here is that not all installations will require admin access - eg, a user who installed Python just for themselves will not need elevation to install an extension. A solution here would be for the installer to *not* be marked as requiring elevation, then sniffing the registry to make an educated guess (eg, HKLM\Software\Python\PythonCore\2.5 existing could indicate admin access is required). If it finds elevation is required, it would spawn another copy of itself with elevation requested, and terminate. This will have a side-effect of meaning the installer never gets the "shield" overlay, meaning the user will not be expecting to be prompted - but that is something we can live with.
However, there is a complication here. Any "pure python" packages are not tied to a particular Python version, so the user can choose what installed Python version to use. Hence, in the general case, we can only determine if admin is required after the UI has opened and the user has selected the Python version. Arranging for the new "elevated" child process at this point will be (a) ugly, as the UI will vanish before the child process displays its GUI and (b) would require additional command-line processing logic - ie, passing the target directory to the child process. If we could make the determination *before* the GUI opens, it would appear seamless and would not require special command-line handling (the child process just does everything)
So I see a few alternatives, but none are very desirable:
* Only make this "admin required" check if a specific version of Python is necessary (ie, the package contains extension modules). This would leave pure-python packages out in the cold.
* Live with the ugly UI that would result in performing that check after the Python version has been selected, and add the command-line processing necessary to make this work.
* Ignore the issue and try to educate people that they must explicitly use "Run as Administrator" for such packages on Vista.
I'm wondering if anyone has any opinions or thoughts on how we should handle this?
Cheers,
Mark
Mark Hammond schrieb:
The pywin32 extensions require (well, prefer) administrative access during installation - certain files are copied to the System32 directory and the registry at HKEY_LOCAL_MACHINE is written to. Also, if I understand correctly, if Python happened to be installed into "\Program Files", admin access would be required to create any files in that directory tree - I'm not sure what permissions the \PythonXX directory are created with, but its not unreasable to assume that some shops might choose to secure that directory similarly to "\Program Files".
[...]
So I see a few alternatives, but none are very desirable:
I have not yet even used vista, so I fear I cannot answer your questions, or even offer an opinion...
- Only make this "admin required" check if a specific version of Python is
necessary (ie, the package contains extension modules). This would leave pure-python packages out in the cold.
- Live with the ugly UI that would result in performing that check after the
Python version has been selected, and add the command-line processing necessary to make this work.
This sound like the most universal option to me.
- Ignore the issue and try to educate people that they must explicitly use
"Run as Administrator" for such packages on Vista.
What if the user does not have or cannot get admin rights? Can he not install the package then?
I'm wondering if anyone has any opinions or thoughts on how we should handle this?
bdist_wininst has its own problems anyway - most severe the MSVC runtime dll issue. Another one is that it won't work for 64-bit installations.
bdist_msi solves the 64-bit and MSVCRT issue. I would expect that it also solves the UAC problems. OTOH it is not possible (AFAIK) to build bdist_msi installers on Linux systems.
Thomas
Hi Thomas,
So I see a few alternatives, but none are very desirable:
I have not yet even used vista, so I fear I cannot answer your questions, or even offer an opinion...
- Only make this "admin required" check if a specific
version of Python is
necessary (ie, the package contains extension modules).
This would leave
pure-python packages out in the cold.
- Live with the ugly UI that would result in performing
that check after the
Python version has been selected, and add the command-line
processing
necessary to make this work.
This sound like the most universal option to me.
- Ignore the issue and try to educate people that they must
explicitly use
"Run as Administrator" for such packages on Vista.
What if the user does not have or cannot get admin rights? Can he not install the package then?
That is correct - but that would be true regardless of how distutils tries to support it. Assuming we correctly detect that admin is required, my proposed changes would cause failure at the start of the install, rather than *trying* to install stuff and some stuff (but not necessarily all) failing, as would happen now.
I'm wondering if anyone has any opinions or thoughts on how
we should handle
this?
bdist_wininst has its own problems anyway - most severe the MSVC runtime dll issue.
What issue is this? Could it be solved by adding a manifest to the executable? It seems to me that Python itself built with VC2005 has this same issue. There is the problem of *installing* this library though, but if we assume Python itself has already done this, I don't see the problem for bdist_wininst.
Another one is that it won't work for 64-bit installations.
My experiments show that very nearly works! I haven't actually got such a binary to install, but a 64bit bdist_wininst based installer correctly runs all the way to asking me to select an installed Python (where it stops as I don't have such an install yet). IIUC, to get this far, bdist_wininst has already sniffed out most data it needs. I could be wrong here, but I'm wondering what problems you had in mind?
bdist_msi solves the 64-bit and MSVCRT issue. I would expect that it also solves the UAC problems. OTOH it is not possible (AFAIK) to build bdist_msi installers on Linux systems.
Yeah, that may be what I need to use going forward - which basically would leave bdist_wininst to slowly die - but I guess that is good to know now, rather than after I spend more time trying to apply life-support to it...
Cheers,
Mark
[sorry for the late reply] Mark Hammond schrieb:
Hi Thomas,
I'm wondering if anyone has any opinions or thoughts on how
we should handle
this?
bdist_wininst has its own problems anyway - most severe the MSVC runtime dll issue.
What issue is this? Could it be solved by adding a manifest to the executable? It seems to me that Python itself built with VC2005 has this same issue. There is the problem of *installing* this library though, but if we assume Python itself has already done this, I don't see the problem for bdist_wininst.
The usual C runtime lib problems as it embeds Python, and also calls problematic functions in the rumtime lib, so it should use the same lib as Python itself. If you build a pure Python bdist_wininst installer, which can be used to install into Python 2.3, 2.4, 2.5, and more versions, the postinstall script (if any) does not run correctly if Python uses the wrong runtime lib.
Another one is that it won't work for 64-bit installations.
My experiments show that very nearly works! I haven't actually got such a binary to install, but a 64bit bdist_wininst based installer correctly runs all the way to asking me to select an installed Python (where it stops as I don't have such an install yet). IIUC, to get this far, bdist_wininst has already sniffed out most data it needs. I could be wrong here, but I'm wondering what problems you had in mind?
I have not tried to build a 64bit runtime stub for bdist_wininst, that's the only reason so far. It could probably be solved.
bdist_msi solves the 64-bit and MSVCRT issue. I would expect that it also solves the UAC problems. OTOH it is not possible (AFAIK) to build bdist_msi installers on Linux systems.
Yeah, that may be what I need to use going forward - which basically would leave bdist_wininst to slowly die - but I guess that is good to know now, rather than after I spend more time trying to apply life-support to it...
Well, bdist_wininst had a sucessful live, I see no problems if it is replaced with something else - be it eggs or bdist_msi.
Thomas
At 07:43 PM 7/10/2007 +0200, Thomas Heller wrote:
Well, bdist_wininst had a sucessful live, I see no problems if it is replaced with something else - be it eggs or bdist_msi.
Perhaps someone could make an egg-to-msi translator, so that non-Windows users could still build and distribute eggs, but Windows users could still install them with built-in uninstallability.
Eggs certainly don't have a complex internal format -- indeed, they don't offer as much installation flexibility as bdist_wininst does. easy_install can convert most (but not all) bdist_wininst .exe's to eggs, so a similar conversion from eggs to MSI should be possible.
Since you happen to be paying attention to distributions atm ... :)
I would love to see a pywin32 egg, which by definition, didn't need to modify anything when it was installed. I guess there are some features in pywin32 that wouldn't work with such a setup, but I'm guessing that most features would. Even if it wasn't called pywin32, I'd really like to see an egg that contained the parts of pywin32 that can be used without modifying anything on installation.
Please. Pretty please. :) It would make pywin32 much mroe usable for people using setuptools.
Jim
-- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Hi Jim,
Since you happen to be paying attention to distributions atm ... :)
I would love to see a pywin32 egg, which by definition, didn't need to modify anything when it was installed. I guess there are some features in pywin32 that wouldn't work with such a setup, but I'm guessing that most features would. Even if it wasn't called pywin32, I'd really like to see an egg that contained the parts of pywin32 that can be used without modifying anything on installation.
Please. Pretty please. :) It would make pywin32 much mroe usable for people using setuptools.
I'm sure you know the drill - patches are always welcome :) I've never modified a setup script to support eggs, and at this stage I'm more interested in getting existing functionality ported to x64 and Vista than creating a new distribution with reduced functionality.
Also, if part of the functionality we had to drop was implementing COM objects (and we would if we couldn't install stuff into system32 or modify the HKLM part of the registry), then I'd be reluctant to call this a suitable distribution for anyone other than people wishing to package pywin32 with their own custom application, and I'd struggle to justify spending much time creating and supporting a distribution just for them.
IOW, I'm certainly not opposed to the concept, but struggling to justify squeezing it into the time I have available for this stuff...
Cheers,
Mark
At 03:20 PM 7/7/2007 +1000, Mark Hammond wrote:
Hi Jim,
Since you happen to be paying attention to distributions atm ... :)
I would love to see a pywin32 egg, which by definition, didn't need to modify anything when it was installed. I guess there are some features in pywin32 that wouldn't work with such a setup, but I'm guessing that most features would. Even if it wasn't called pywin32, I'd really like to see an egg that contained the parts of pywin32 that can be used without modifying anything on installation.
Please. Pretty please. :) It would make pywin32 much mroe usable for people using setuptools.
I'm sure you know the drill - patches are always welcome :) I've never modified a setup script to support eggs, and at this stage I'm more interested in getting existing functionality ported to x64 and Vista than creating a new distribution with reduced functionality.
Also, if part of the functionality we had to drop was implementing COM objects (and we would if we couldn't install stuff into system32 or modify the HKLM part of the registry), then I'd be reluctant to call this a suitable distribution for anyone other than people wishing to package pywin32 with their own custom application, and I'd struggle to justify spending much time creating and supporting a distribution just for them.
I'm trying to remember for sure, but ISTR that I've successfully installed pywin32 in egg form. Ah yes, I did, and it's working for me in one of my Python installations. Don't remember how I did it though!
[pause to try and install it with easy_install in another Python installation]
Ah. Okay, first, pywin32 isn't registered on PyPI, so you need a -f link to find the files:
easy_install -f http://sourceforge.net/project/showfiles.php?group_id=78018 pywin32
This is easily fixed by adding a PyPI listing with the above URL registered as the "Download URL".
easy_install successfully installs the package, but in order to do the registry and copying stuff, the postinstall script has to be run:
$ Scripts/pywin32_postinstall.py -install Traceback (most recent call last): File "Scripts\pywin32_postinstall.py", line 5, in ? pkg_resources.run_script('pywin32==210', 'pywin32_postinstall.py') File "c:\cygwin\home\pje\projects\setuptools-0.6\pkg_resources.py", line 448, in run_script self.require(requires)[0].run_script(script_name, ns) File "c:\cygwin\home\pje\projects\setuptools-0.6\pkg_resources.py", line 1166, in run_script execfile(script_filename, namespace, namespace) File "c:\cygwin\home\pje\chandlerstuff\chandler\release\bin\lib\site-packages\pywin32-210-py2.4-win32.egg\EGG-INFO\scripts\pywin32_postinstall.py", line 365, in ? install() File "c:\cygwin\home\pje\chandlerstuff\chandler\release\bin\lib\site-packages\pywin32-210-py2.4-win32.egg\EGG-INFO\scripts\pywin32_postinstall.py", line 155, in install LoadSystemModule(lib_dir, "pywintypes") File "c:\cygwin\home\pje\chandlerstuff\chandler\release\bin\lib\site-packages\pywin32-210-py2.4-win32.egg\EGG-INFO\scripts\pywin32_postinstall.py", line 94, in LoadSystemModule ('.dll', 'rb', imp.C_EXTENSION)) ImportError: DLL load failed: The specified module could not be found.
The problem here is that lib_dir is calculated using the system configuration, instead of by locating where the package is actually installed. I changed the install() function like this:
import timer lib_dir = os.path.dirname(timer.__file__)
Importing some other pywin32 module than 'timer' would be okay; it just looked like something small and unlikely to depend on anything else. With this change, the install script produced this output::
$ Scripts/pywin32_postinstall.py` -install Copied pythoncom24.dll to C:\WINDOWS\system32\pythoncom24.dll Copied pywintypes24.dll to C:\WINDOWS\system32\pywintypes24.dll Registered: Python.Interpreter Registered: Python.Dictionary Registered: Python -> Software\Python\PythonCore\2.4\Help[None]=None -> Software\Python\PythonCore\2.4\Help\Pythonwin Reference[None]='C:\...\lib\site-packages\pywin32-210-py2.4-win32.egg\PyWin32.chm' Creating directory C:...\bin\lib\site-packages\pywin32-210-py2.4-win32.egg\win32com\gen_py The pywin32 extensions were successfully installed.
Is there anything I can do to verify that this did in fact work correctly?
It appears that all would be needed for pywin32 to be fully easy_install-able would be this change to the postinstall script (which shouldn't affect its functionality for direct installation), and an entry on PyPI. No setup script changed required -- or so it appears. Anyone wishing to use the features enabled by the postinstall script would just need to run it with -install.
One question of course is, what features do you lose if you don't run the postinstall? If I hadn't run it, would I still have been able to use Python to access COM objects exposed by other applications, or would I have been unable to do COM at all?
Ah. Okay, first, pywin32 isn't registered on PyPI, so you need a -f link to find the files:
easy_install -fhttp://sourceforge.net/project/showfiles.php?group_id=78018 pywin32
This is easily fixed by adding a PyPI listing with the above URL registered as the "Download URL".
Cool - done!
easy_install successfully installs the package, but in order to do the registry and copying stuff, the postinstall script has to be run:
$ Scripts/pywin32_postinstall.py -install Traceback (most recent call last): File "Scripts\pywin32_postinstall.py", line 5, in ? pkg_resources.run_script('pywin32==210', 'pywin32_postinstall.py') File "c:\cygwin\home\pje\projects\setuptools-0.6\pkg_resources.py", line 448, in run_script self.require(requires)[0].run_script(script_name, ns) File "c:\cygwin\home\pje\projects\setuptools-0.6\pkg_resources.py", line 1166, in run_script execfile(script_filename, namespace, namespace) File "c:\cygwin\home\pje\chandlerstuff\chandler\release\bin\lib\sit e-packages\pywin32-210-py2.4-win32.egg\EGG-INFO\scripts\pywin3 2_postinstall.py", line 365, in ? install() File "c:\cygwin\home\pje\chandlerstuff\chandler\release\bin\lib\sit e-packages\pywin32-210-py2.4-win32.egg\EGG-INFO\scripts\pywin3 2_postinstall.py", line 155, in install LoadSystemModule(lib_dir, "pywintypes") File "c:\cygwin\home\pje\chandlerstuff\chandler\release\bin\lib\sit e-packages\pywin32-210-py2.4-win32.egg\EGG-INFO\scripts\pywin3 2_postinstall.py", line 94, in LoadSystemModule ('.dll', 'rb', imp.C_EXTENSION)) ImportError: DLL load failed: The specified module could not be found.
The problem here is that lib_dir is calculated using the system configuration, instead of by locating where the package is actually installed. I changed the install() function like this:
import timer lib_dir = os.path.dirname(timer.__file__)
Importing some other pywin32 module than 'timer' would be okay; it just looked like something small and unlikely to depend on anything else. With this change, the install script produced this output::
hrm - normally lib_dir is pointing at site-packages - ie, the parent of 'win32', 'win32com' etc - whereas the code above would give site-packages\win32. That function should be fairly easy to make egg-friendly though.
Is there anything I can do to verify that this did in fact work correctly?
Simplest would be to paste the following 2 lines into a file:
set py = CreateObject("Python.Interpreter") WScript.Echo("1+1=" & py.Eval("1+1"))
then execute: cscript.exe //E:VBScript filename
I suspect it will work though given the output you demonstrated. post_install.py creates a .pth file, which I suspect is redundant, or just plain wrong, when using eggs - it might be work looking for that pywin32.pth file and seeing what is in it.
It appears that all would be needed for pywin32 to be fully easy_install-able would be this change to the postinstall script (which shouldn't affect its functionality for direct installation), and an entry on PyPI. No setup script changed required -- or so it appears. Anyone wishing to use the features enabled by the postinstall script would just need to run it with -install.
Excellent!
One question of course is, what features do you lose if you don't run the postinstall? If I hadn't run it, would I still have been able to use Python to access COM objects exposed by other applications, or would I have been unable to do COM at all?
You probably would have only lost the ability to implement COM objects - consuming them should be fine. You don't get the help file registered where pythonwin can find it and don't get a bogus .pth file <wink>, but that is about it.
Cheers,
Mark
At 08:49 AM 7/10/2007 +1000, Mark Hammond wrote:
hrm - normally lib_dir is pointing at site-packages - ie, the parent of 'win32', 'win32com' etc - whereas the code above would give site-packages\win32.
Ah. Well, in the egg case, there is no win32 directory, nor win32/lib subdirectory thereof; everything that's not a package just goes right into the site-packages\pywin32-210-py2.4-win32.egg directory.
If you'd like to see what's happening, download http://peak.telecommunity.com/dist/ez_setup.py and run:
ez_setup.py pywin32
or alternately, install the Windows installer for setuptools, then run "python -m easy_install pywin32".
At that point, you'll be able to see the site-packages layout the way it ends up under setuptools. If setuptools' directory layout is wrong in some way, let me know and I'll see what I can do.
Is there anything I can do to verify that this did in fact work correctly?
Simplest would be to paste the following 2 lines into a file:
set py = CreateObject("Python.Interpreter") WScript.Echo("1+1=" & py.Eval("1+1"))
then execute: cscript.exe //E:VBScript filename
Microsoft (R) Windows Script Host Version 5.6 Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.
C:...\testfile(1, 1) (null): The specified module could not be found.
You probably would have only lost the ability to implement COM objects - consuming them should be fine. You don't get the help file registered where pythonwin can find it and don't get a bogus .pth file <wink>, but that is about it.
As it happens, the bogus .pth file gets installed inside of the pywin32*.egg directory, so it doesn't actually get read by Python. By design, easy_install wants everything for a package to be contained in its .egg, and refuses to put anything outside of it except wrapper scripts and a .pth file to put the .egg on sys.path.
I recently discovered that one reason why pywin32 didn't work so well with easy_install was that easy_install wasn't processing pywin32's .pth file due to a case-sensitivity bug!
I've checked in a fix that appears to handle everything correctly, except for the previously-discussed issues with the post-install script, and a minor issue in pywintypes.py. Adding the following at line 91 in pywintypes.py fixed it for me:
if found is None: # Not in the Python directory? Maybe we were installed via # easy_install... if os.path.isfile(os.path.join(os.path.dirname(__file__), filename)): found = os.path.join(os.path.dirname(__file__), filename)
With this addition and my fix to easy_install, it's possible to "easy_install pywin32" and get a working pywin32 install, minus the various registrations and shortcuts handled by the post-install script.
Fixing the post-install script to work with an easy_install-supplied payload is a bit more difficult, since it can't find its library directory and various other things. However, my personal use case for having a pywin32 egg is simply to be able to install it into a Chandler Python installation, which is not registered in the registry. (The .exe installer won't work with unregistered Pythons, so I'm win32-less otherwise.)
On Jul 7, 2007, at 1:20 AM, Mark Hammond wrote:
Hi Jim,
Since you happen to be paying attention to distributions atm ... :)
I would love to see a pywin32 egg, which by definition, didn't need to modify anything when it was installed. I guess there are some features in pywin32 that wouldn't work with such a setup, but I'm guessing that most features would. Even if it wasn't called pywin32, I'd really like to see an egg that contained the parts of pywin32 that can be used without modifying anything on installation.
Please. Pretty please. :) It would make pywin32 much mroe usable for people using setuptools.
I'm sure you know the drill - patches are always welcome :)
Unfortunately, I don't have sufficient Windows knowledge.
I've never modified a setup script to support eggs, and at this stage
You generally don't need to unless you want to be able to specify extra meta data. A standard distutils setup file should do.
The main difference is that eggs are "zero installation", so someone installing an egg won't get any post-installation done automatically.
...
Also, if part of the functionality we had to drop was implementing COM objects (and we would if we couldn't install stuff into system32 or modify the HKLM part of the registry), then I'd be reluctant to call this a suitable distribution for anyone other than people wishing to package pywin32 with their own custom application, and I'd struggle to justify spending much time creating and supporting a distribution just for them.
I'm sorry to hear that.
IOW, I'm certainly not opposed to the concept, but struggling to justify squeezing it into the time I have available for this stuff...
I might be willing to create such distributions, at least for our own use, and make them publicly available, much as I've done for Twisted, but I probably wouldn't do a great job.
Jim
-- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
participants (4)
-
Jim Fulton -
Mark Hammond -
Phillip J. Eby -
Thomas Heller