Corporate installations
Python is getting used widely enough in my place of employment that people are agitating for a common installation (personal variants are screwing people too often). Various machines are running Win95, Win98, NT, Windows2000, assorted flavors of Linux, Solaris, Irix and MacOS. I'll choke on that whole banana when it's shoved down my throat. For starters, I just need to get it running across Windows platforms. Never looked into this before, and it seems to be exceedingly complicated right out of the box <0.5 wink>: D:\Python>python Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
import sys ['', 'D:\\Python\\win32', 'D:\\Python\\win32\\lib', 'D:\\Python', 'D:\\Python\\Pythonwin', 'D:\\Python\\Lib\\plat-win', 'D:\\Python\\Lib', 'D:\\Python\\DLLs', 'D:\\Python\\Lib\\lib-tk', 'D:\\PYTHON\\DLLs', 'D:\\PYTHON\\lib', 'D:\\PYTHON\\lib\\plat-win', 'D:\\PYTHON\\lib\\lib-tk', 'D:\\PYTHON']
Where does all that stuff come from? The first is apparently the current directory; cool. The next 4 appear to come from MarkH's stuff, via the registry. The next 4 appear to come from Python's own registry PythonPath key. The 4 after that are a mystery, and duplicate the preceding 4 (except for meaningless-- it's Windows <wink> -- case differences). The last is another mystery. site.py isn't implicated in any of them (same thing when running with -S). I suppose this is a clue: D:\Python>set PYTHONHOME=ick;fooey D:\Python>python Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
import sys sys.path ['', 'D:\\Python\\win32', 'D:\\Python\\win32\\lib', 'D:\\Python', 'D:\\Python\\Pythonwin', 'D:\\Python\\Lib\\plat-win', 'D:\\Python\\Lib', 'D:\\Python\\DLLs', 'D:\\Python\\Lib\\lib-tk', 'ick', 'fooey\\DLLs', 'ick', 'fooey\\lib', 'ick', 'fooey\\lib\\plat-win', 'ick', 'fooey\\lib\\lib-tk', 'D:\\PYTHON']
Doesn't appear to be sensible, but at least it's predictable <wink>. I was surprised to find that setting PYTHONPATH can't be used to override any of this -- it just inserts even more stuff, into sys.path[1:1]. Do I really care? Not much -- the complexity just makes it hard to get a mental model of what Python *thinks* it's trying to accomplish here, and so harder to figure out what needs to be done. Which is pretty severe: "Dragon std" apps must run the same version of Python (which can be local) with the same libraries (which must be network-mounted), regardless of whatever version of Python each user may have built on their own, and regardless of their registry settings or envar values. I think this means I need to distribute a python15.dll and python.exe (via our source control system) into a directory very early on the PATH (other common .exe's are distributed this way, so no big deal), and add a sitecustomize.py that replaces all of sys.path with the Big Four (Lib\plat-win, Lib, DLLs, and Lib\lib-tk) expressed as UNC paths, + the common Dragon dir. Network name mangling being what it is, I suppose I'll also need to force PYTHONCASEOK. There's no way to do that from within Python, is there? If not, everyone going through "unfortunate" paths in the network will have to set that themselves. Anyone have better ideas? That shouldn't be hard <wink>. the-difference-between-one-user-and-two-is-infinite-ly y'rs - tim
The 4 after that are a mystery, and duplicate the preceding 4 (except for meaningless-- it's Windows <wink> -- case differences).
Python makes a brave attempt to deduce the PYTHONHOME itself by looking for "well-known" files in the lib directory. Python15.dll uses argv[0] or the result of GetModuleFileName(NULL) (whichever is deemed "better" by the existance of a slash) as the basis for a search for the landmark file. If found, those 4 entries are added and if not they are added relatively (ie ".\lib" in the vain hope they will suddenly become meaningful (which they almost certainly wont) So, when running a .EXE that lives in the main Python directory (such as Python.exe), we get duplicated entries - the ones we deduced, and the ones explicitely in the registry. The problem arises when Python15.dll is used by a .exe that doesnt live in this nice directory - eg Pythonwin, or any other embedding, such as COM objects or peoples custom apps. As argv[0] and GetModuleFileName(NULL) both return a directory where "\lib" appended will not find the landmark file (string.py for windows), you get the useless relative paths added. So, what this means is that Python's strategy for deducing the PythonPath only works for Python.exe and Pythonw.exe - typically the only .exes in the Python directory. So we have the situation where the paths must also be registered in the registry if any other .exe wants to work.
I was surprised to find that setting PYTHONPATH can't be used to override any of this -- it just inserts even more stuff, into sys.path[1:1].
Once upon a time Guido stated that PYTHONPATH should override the registry completely. I cant recall the history of this tho, and wouldnt object is that was the case. You are then faced with how to set this env variable (and if you are doing that, why not just set the registry?)
I think this means I need to distribute a python15.dll and python.exe (via our source control system) into a directory very early on the PATH (other common .exe's are distributed this way, so no big deal), and add a sitecustomize.py that replaces all of sys.path with the Big Four (Lib\plat-win, Lib, DLLs, and Lib\lib-tk) expressed as UNC paths, + the common Dragon dir.
This should be fine - as long as no one needs to run any other .EXE that needs Python15.dll (or unless that other .exe is also in that same directory).
Network name mangling being what it is, I suppose I'll also need to force PYTHONCASEOK. There's no way to do that from within Python, is there? If
While my views on this have temepered from illogically ranting to Guido how much I hate it, I still believe this is a mis-feature for Windows. The other alternative is to put Python.exe on the network path, and have your library in a ".\lib" directory under that. Doesnt really solve the problem for your "DragonLib" stuff - Im sure you dont want to stuff _everything_ into a single .lib directory. Also, IMO Python shouldnt bother going sniffing for its library if the registry exists. It just adds time to startup, and if it works is almost guaranteed to be redundant, and if it doesnt work will add useless entries that slow down module searches. However, that still doesnt solve your problem. The final suggestion is to put some of the win32 extensions (specifically, win32api) on a network share along with Python.exe and the standard lib. Then provide a .bat file that actually sets up the registry, and off you go. Sure, people hate the registry, but setting an environment variable is worse. Or an alternative - provide a .reg file with the necessary registry keys - users can "execute" it (double-clicking from explorer, typing the name in start->run, execute "start whatever.reg in their logon script, etc) to have their registry setup. I wish I had a better answer, but can't for the life of me work out what it should be even if MS supported it! Mark.
Tim Peters wrote:
Python is getting used widely enough in my place of employment that people are agitating for a common installation (personal variants are screwing people too often). Various machines are running Win95, Win98, NT, Windows2000, assorted flavors of Linux, Solaris, Irix and MacOS. I'll choke on that whole banana when it's shoved down my throat.
For starters, I just need to get it running across Windows platforms. Never looked into this before, and it seems to be exceedingly complicated right out of the box <0.5 wink>:
D:\Python>python Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
import sys ['', 'D:\\Python\\win32', 'D:\\Python\\win32\\lib', 'D:\\Python', 'D:\\Python\\Pythonwin', 'D:\\Python\\Lib\\plat-win', 'D:\\Python\\Lib', 'D:\\Python\\DLLs', 'D:\\Python\\Lib\\lib-tk', 'D:\\PYTHON\\DLLs', 'D:\\PYTHON\\lib', 'D:\\PYTHON\\lib\\plat-win', 'D:\\PYTHON\\lib\\lib-tk', 'D:\\PYTHON']
Where does all that stuff come from? The first is apparently the current directory; cool. The next 4 appear to come from MarkH's stuff, via the registry. The next 4 appear to come from Python's own registry PythonPath key. The 4 after that are a mystery, and duplicate the preceding 4 (except for meaningless-- it's Windows <wink> -- case differences). The last is another mystery. site.py isn't implicated in any of them (same thing when running with -S).
On Linux I get: ['', '/home/lemburg/bin', '/home/lemburg/lib', '/home/lemburg/projects', '/home/lemburg/python/Lib/', '/home/lemburg/python/Lib/test', '/home/lemburg/python/Lib/plat-linux2', '/home/lemburg/python/Lib/lib-tk', '/home/lemburg/python/Modules', '/usr/local/lib/python1.5/site-packages'] It seems that the last entry refers to what Python thinks is the standard location for system wide extensions. The first 3 come from my PYTHONPATH setting, the next few are inserted by the Python startup code based on what Python found as landmark to determine PYTHONHOME.
I think this means I need to distribute a python15.dll and python.exe (via our source control system) into a directory very early on the PATH (other common .exe's are distributed this way, so no big deal), and add a sitecustomize.py that replaces all of sys.path with the Big Four (Lib\plat-win, Lib, DLLs, and Lib\lib-tk) expressed as UNC paths, + the common Dragon dir.
Sounds reasonable ;-)
Network name mangling being what it is, I suppose I'll also need to force PYTHONCASEOK. There's no way to do that from within Python, is there? If not, everyone going through "unfortunate" paths in the network will have to set that themselves.
If Windows allows writing the process' environment you can set PYTHONCASEOK dynamically: the check queries the environment for the flag prior to every check (instead of doing going the usual internal Python flag way as many of the others do). See Python/import.c:check_case() for details. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 77 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
Tim Peters wrote:
['', 'D:\\Python\\win32', 'D:\\Python\\win32\\lib', 'D:\\Python', 'D:\\Python\\Pythonwin', 'D:\\Python\\Lib\\plat-win', 'D:\\Python\\Lib', 'D:\\Python\\DLLs', 'D:\\Python\\Lib\\lib-tk', 'D:\\PYTHON\\DLLs', 'D:\\PYTHON\\lib', 'D:\\PYTHON\\lib\\plat-win', 'D:\\PYTHON\\lib\\lib-tk', 'D:\\PYTHON']
Where does all that stuff come from?
Good question. I don't know, and every time I start to think I do, I am wrong. Certainly PYTHONPATH is full of junk on Windows. The point is that PYTHONPATH just doesn't work for commercial installations. Please see these for some discussion: ftp://ftp.interet.com/bootmodule.html ftp://ftp.interet.com/pylib.html I think the solution is to put the library files and packages into archive files (in some format yet to be determined) and find them by looking in the directory of python15.dll and/or python.exe. On Unix there is $prefix, which actually seems to work.
I think this means I need to distribute a python15.dll and python.exe (via our source control system) into a directory very early on the PATH
Yes, that is a good start. Just understand that *.pyd and and DLL's which are imported (instead of linked against) are found from PYTHONPATH, not PATH. Also you will need a valid PATH on all PC's which is quite a bother. An alternative is to use the Wise installer to set up icons for your apps. Real Windows apps don't use PATH. We use Wise for internal as well as external apps. Note that if you import a module "import foo", and foo lives in "foo.dll", then on Windows, PYTHONPATH is searched, and the standard Windows DLL path is not searched. That means you can't use standard Windows locations for DLL's which are dynamically imported. This is an error and should be fixed.
and add a sitecustomize.py that replaces all of sys.path with the Big Four (Lib\plat-win, Lib, DLLs, and Lib\lib-tk)
The problem is that you already need a valid PYTHONPATH to find sitecustomize.py. Better is to create a custom python15.dll with sitecustomize.py, exceptions.py and site.py (all needed during boot) frozen in as frozen modules. In our office I freeze in other library files too.
Network name mangling being what it is, I suppose I'll also need to force PYTHONCASEOK.
I don't seem to need that on 95/98/NT. Network names shouldn't get mangled on these systems. I guess you mean lack of case sensitivity, but just make sure the case is correct in the Lib directory. This is another reason to prefer archive files anyway. But no, you shouldn't need PYTHONCASEOK.
Anyone have better ideas? That shouldn't be hard <wink>.
Yes, this is a major problem with Python's acceptance for commercial apps (and I include internal production code as an app) so lets figure out what to do and fix it. Jim Ahlstrom
"James C. Ahlstrom" wrote: Sorry, correct URL is: ftp://ftp.interet.com/pub/bootmodule.html ftp://ftp.interet.com/pub/pylib.htm Jim Ahlstrom
while doing the final checks for my upcoming standard library guide, I thought I'd check what had changed in the CVS version. some old (1.5.1) junk was still to be found in the lib-old subdirectory, but certain 1.5.2 things seem to have vanished into thin air. like find.py, for example. does anyone know where it is? after all, it's kinda useful, and I just pointed someone to it, only to find that he didn't have it on his machine... is there an official list of totally deprecated modules somewhere? </F>
Fredrik Lundh writes:
some old (1.5.1) junk was still to be found in the lib-old subdirectory, but certain 1.5.2 things seem to have vanished into thin air.
like find.py, for example. does anyone know where it is? after all, it's kinda useful, and I
It should be in lib-old. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives
Fred L. Drake, Jr. <fdrake@acm.org> wrote:
like find.py, for example. does anyone know where it is? after all, it's kinda useful, and I
It should be in lib-old.
"should" as in "is already there" or "oops, I'll fix it" ? :-) </F> [fredrik@brain BleedingEdge]$ ls python/dist/src/Lib/lib-old/ CVS codehack.py newdir.py rand.py whatsound.py Para.py fmt.py ni.py tb.py zmod.py addpack.py lockfile.py poly.py util.py (sure looks like "oops" to me...)
"Fred L. Drake, Jr." wrote:
Fredrik Lundh writes:
"should" as in "is already there" or "oops, I'll fix it" ? :-)
It's certainly in lib-old/ in the CVS repository.
Sorry. Definately certainly not. But a find.py is in Demo/threads. Fred (any of both :) did you mean that? ciao - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net 10553 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home
"should" as in "is already there" or "oops, I'll fix it" ? :-)
It's certainly in lib-old/ in the CVS repository.
Sorry. Definately certainly not.
But a find.py is in Demo/threads. Fred (any of both :) did you mean that?
I cannot see any "find.py" in either Lib (where it is in all my copies of 1.5.2) or Lib/lib-old. fwiw, I can think of a few dozen other standard library modules that I'd remove before this one. cannot we put it back? please? :-) (after all, it's been a reoccuring theme on comp.lang.python lately. and 1.5.2+ breaks my new book...). </F>
Fredrik Lundh writes:
I cannot see any "find.py" in either Lib (where it is in all my copies of 1.5.2) or Lib/lib-old.
It most certainly *should* be appearing in lib-old; I just checked the internal CVS repository and it's really there. Perhaps the anonymous mirror isn't right? (Barry, can you check this? I don't even know where it is, much less whether I can see those directories.)
fwiw, I can think of a few dozen other standard library modules that I'd remove before this one. cannot we put it back? please? :-)
That's up to Guido, not me. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives
I cannot see any "find.py" in either Lib (where it is in all my copies of 1.5.2) or Lib/lib-old.
It *is* in Lib/lib-old. Maybe the CVS mirror is messed up, we'll ask Barry to investigate. Maybe the branching messed things up?
fwiw, I can think of a few dozen other standard library modules that I'd remove before this one. cannot we put it back? please? :-)
It's better (and more illustrative) to use os.walk() to do this thing.
(after all, it's been a reoccuring theme on comp.lang.python lately. and 1.5.2+ breaks my new book...).
Good thing it isn't printed yet. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
"Guido" == Guido van Rossum <guido@cnri.reston.va.us> writes:
Guido> It *is* in Lib/lib-old. Maybe the CVS mirror is messed up, Guido> we'll ask Barry to investigate. Maybe the branching messed Guido> things up? Somehow the mirror was out of date. Perhaps the branch merge got it out of whack, but in any case, I've done a top-level resync. find.py is definitely in the repository under lib-old now! Hope that helps, -Barry
participants (9)
-
Barry A. Warsaw -
Christian Tismer -
Fred L. Drake, Jr. -
Fredrik Lundh -
Guido van Rossum -
James C. Ahlstrom -
M.-A. Lemburg -
Mark Hammond -
Tim Peters