I've updated imputil... The main changes is that I added SysPathImporter and BuiltinImporter. I also did some restructing to help with bootstrapping the module (remove dependence on os.py). For testing a revamped Python import system, you can importing the thing and call imputil._test_revamp() to set it up. This will load normal, builtin, and frozen modules via imputil. Dynamic modules are still handled by Python, however. I ran a timing comparisons of importing all modules in /usr/lib/python1.5 (using standard and imputil-based importing). The standard mechanism can do it in about 8.8 seconds. Through imputil, it does it in about 13.0 seconds. Note that I haven't profiled/optimized any of the Importer stuff (yet). The point about dynamic modules actually discovered a basic problem that I need to resolve now. The current imputil assumes that if a particular Importer loaded the top-level module in a package, then that Importer is responsible for loading all other modules within that package. In my particular test, I tried to import "xml.parsers.pyexpat". The two package modules were handled by SysPathImporter. The pyexpat module is a dynamic load module, so it is *not* handled by the Importer -- bam. Failure. Basically, each part of "xml.parsers.pyexpat" may need to use a different Importer... Off to ponder, -g -- Greg Stein, http://www.lyra.org/
oops... forgot: http://www.lyra.org/greg/python/imputil.py -g On Sat, 20 Nov 1999, Greg Stein wrote:
I've updated imputil... The main changes is that I added SysPathImporter and BuiltinImporter. I also did some restructing to help with bootstrapping the module (remove dependence on os.py).
For testing a revamped Python import system, you can importing the thing and call imputil._test_revamp() to set it up. This will load normal, builtin, and frozen modules via imputil. Dynamic modules are still handled by Python, however.
I ran a timing comparisons of importing all modules in /usr/lib/python1.5 (using standard and imputil-based importing). The standard mechanism can do it in about 8.8 seconds. Through imputil, it does it in about 13.0 seconds. Note that I haven't profiled/optimized any of the Importer stuff (yet).
The point about dynamic modules actually discovered a basic problem that I need to resolve now. The current imputil assumes that if a particular Importer loaded the top-level module in a package, then that Importer is responsible for loading all other modules within that package. In my particular test, I tried to import "xml.parsers.pyexpat". The two package modules were handled by SysPathImporter. The pyexpat module is a dynamic load module, so it is *not* handled by the Importer -- bam. Failure.
Basically, each part of "xml.parsers.pyexpat" may need to use a different Importer...
Off to ponder, -g
-- Greg Stein, http://www.lyra.org/
_______________________________________________ Python-Dev maillist - Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev
-- Greg Stein, http://www.lyra.org/
<enable-ramble-mode> :-) On Sat, 20 Nov 1999, Greg Stein wrote:
... The point about dynamic modules actually discovered a basic problem that I need to resolve now. The current imputil assumes that if a particular Importer loaded the top-level module in a package, then that Importer is responsible for loading all other modules within that package. In my particular test, I tried to import "xml.parsers.pyexpat". The two package modules were handled by SysPathImporter. The pyexpat module is a dynamic load module, so it is *not* handled by the Importer -- bam. Failure.
Basically, each part of "xml.parsers.pyexpat" may need to use a different Importer...
I've thought about this and decided the issue is with my particular Importer, rather than the imputil design. The PathImporter traverses a set of paths and establishes a package hierarchy based on a filesystem layout. It should be able to load dynamic modules from within that filesystem area. A couple alternatives, and why I don't believe they work as well: * A separate importer to just load dynamic libraries: this would need to replicate PathImporter's mapping of Python module/package hierarchy onto the filesystem. There would also be a sequencing issue because one Importer's paths would be searched before the other's paths. Current Python import rules establishes that a module earlier in sys.path (whether a dyn-lib or not) is loaded before one later in the path. This behavior could be broken if two Importers were used. * A design whereby other types of modules can be placed into the filesystem and multiple Importers are used to load parts of the path (e.g. PathImporter for xml.parsers and DynLibImporter for pyexpat). This design doesn't work well because the mapping of Python module/package to the filesystem is established by PathImporter -- try to mix a "private" mapping design among Importers creates too much coupling. There is also an argument that the design is fundamentally incorrect :-). I would argue against that, however. I'm not sure what form an argument *against* imputil would be, so I'm not sure how to preempty it :-). But we can get an idea of various arguments by hypothesizing different scenarios and requireing that the imputil design satisifies them. In the above two alternatives, they were examing the use of a secondary Importer to load things out of the filesystem (and it explained why two Importers in whatever configuration is not a good thing). Let's state for argument's sake that files of some type T must be placable within the filesystem (i.e. according to the layout defined by PathImporter). We'll also say that PathImporter doesn't understand T, since the latter was designed later or is private to some app. The way to solve this is to allow PathImporter to recognize it through some configuration of the instance (e.g. self.recognized_types). A set of hooks in the PathImporter would then understand how to map files of type T to a code or module object. (alternatively, a generalized set of hooks at the Importer class level) Note that you could easily have a utility function that scans sys.importers for a PathImporter instance and adds the data to recognize a new type -- this would allow for simple installation of new types. Note that PathImporter inherently defines a 1:1 mapping from a module to a file. Archives (zip or jar files) cannot be recognized and handled by PathImporter. An archive defines an entirely different style of mapping between a module/package and a file in the filesystem. Of course, an Importer that uses archives can certainly look for them in sys.path. The imputil design is derived directly from the "import" statement. "Here is a module/package name, give me a module." (this is embodied in the get_code() method in Importer) The find/load design established by ihooks is very filesystem-based. In many situations, a find/load is very intertwined. If you want to take the URL case, then just examine the actual network activity -- preferably, you want a single transaction (e.g. one HTTP GET). Find/load implies two transactions. With nifty context handling between the two steps, you can get away with a single transaction. But the point is that the design requires you to get work around its inherent two-step mechanism and establish a single step. This is weird, of course, because importing is never *just* a find or a load, but always both. Well... since I've satisfied to myself that PathImporter needs to load dynamic lib modules, I'm off to code it... Cheers, -g -- Greg Stein, http://www.lyra.org/
On Tue, 23 Nov 1999, Greg Stein wrote:
... Well... since I've satisfied to myself that PathImporter needs to load dynamic lib modules, I'm off to code it...
All right. imputil.py now comes with code to emulate the builtin Python import mechanism. It loads all the same types of files, uses sys.path, and (pointed out by JimA) loads builtins before looking on the path. The only "feature" it doesn't support is using package.__path__ to look for submodules. I never liked that thing, so it isn't in there. (imputil *does* set the __path__ attribute, tho) Code is available at: http://www.lyra.org/greg/python/imputil.py Next step is to add a "standard" library/archive format. JimA and I have been tossing some stuff back and forth on this. Cheers, -g -- Greg Stein, http://www.lyra.org/
Greg Stein wrote:
* A separate importer to just load dynamic libraries: this would need to replicate PathImporter's mapping of Python module/package hierarchy onto the filesystem. There would also be a sequencing issue because one Importer's paths would be searched before the other's paths. Current Python import rules establishes that a module earlier in sys.path (whether a dyn-lib or not) is loaded before one later in the path. This behavior could be broken if two Importers were used.
I would like to argue that on Windows, import of dynamic libraries is broken. If a file something.pyd is imported, then sys.path is searched to find the module. If a file something.dll is imported, the same thing happens. But Windows defines its own search order for *.dll files which Python ignores. I would suggest that this is wrong for files named *.dll, but OK for files named *.pyd. A SysAdmin should be able to install and maintain *.dll as she has been trained to do. This makes maintaining Python installations simpler and more un-surprising. I have no solution to the backward compatibilty problem. But the code is only a couple lines. A LoadLibrary() call does its own path searching. Jim Ahlstrom
I would like to argue that on Windows, import of dynamic libraries is broken. If a file something.pyd is imported, then sys.path is searched to find the module. If a file something.dll is imported, the same thing happens. But Windows defines its own search order for *.dll files which Python ignores. I would suggest that this is wrong for files named *.dll, but OK for files named *.pyd.
I think you misunderstand some of the issues. Python cannot import every .dll file. Only .dll files that conform to the convention for Python extension modules can be imported. (The convention is that it must export an init<module> function.) On most other platforms, shared libraries must have a specific extension (e.g. .so on most Unix). Python allows you to drop such a file into any directory where is looks for modules, and it will then direct the dynamic load support to load that specific file. This seems logical -- Python extensions must live in directories that Python searches (Python must do its own search because the search order is significant). On Windows, Python uses the same strategy. The only modification is that it is allowed to give the file a different extension, namely .pyd, to indicate that this really is a Python extension and not a regular DLL. This was mostly introduced because it is apparently common to have an existing DLL "foo.dll" and write a Python wrapper for it that is also called "foo". Clearly, two files foo.dll are too confusing, so we let you name the wrapper foo.pyd. But because the file format is essentially that of a DLL, we don't *require* this renaming; some ways of creating DLLs in the first place may make it difficult to do.
A SysAdmin should be able to install and maintain *.dll as she has been trained to do. This makes maintaining Python installations simpler and more un-surprising.
I don't see that a SysAdmin needs to do much DLL management. This is up to installer scripts. Anyway how hard can it be for a SysAdmin to leave DLLs in specific directories alone?
I have no solution to the backward compatibilty problem. But the code is only a couple lines. A LoadLibrary() call does its own path searching.
But at what point should this LoadLibrary() call be called? The import statement contains no clue that a DLL is requested -- the sys.path search reveals that. I claim that there is nothing with the current strategy. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
I would like to argue that on Windows, import of dynamic libraries is broken. If a file something.pyd is imported, then sys.path is searched to find the module. If a file something.dll is imported, the same thing happens. But Windows defines its own search order for *.dll files which Python ignores. I would suggest that this is wrong for files named *.dll, but OK for files named *.pyd.
I think you misunderstand some of the issues.
Python cannot import every .dll file. Only .dll files that conform to the convention for Python extension modules can be imported. (The convention is that it must export an init<module> function.)
Of course I meant that the test is LoadLibrary(module) followed by GetProcAddress(h, "init" + module). Both must succeed.
This seems logical -- Python extensions must live in directories that Python searches (Python must do its own search because the search order is significant).
The PYTHONPATH search path is what I am trying to get away from. If I eliminate PYTHONPATH I still can not use the Windows DLL search path (which is superior) because DLLs are searched on PYTHONPATH too; thus my post. I don't believe it is important for Python module.dll to be located on PYTHONPATH.
A SysAdmin should be able to install and maintain *.dll as she has been trained to do. This makes maintaining Python installations simpler and more un-surprising.
I don't see that a SysAdmin needs to do much DLL management. This is up to installer scripts. Anyway how hard can it be for a SysAdmin to leave DLLs in specific directories alone?
The problem is maintaining PYTHONPATH plus having DLL's on a non-standard search path. Yes, PythonDev[:] and professional SysAdmins can do it. But it is not as simple as it could be. Someone has to write the install scripts. And what if something doesn't work? Think of Python being used as a teaching language for the 8th grade. Think of the 8th grade teacher trying to get all this right. The only thing that works is simplicity.
But at what point should this LoadLibrary() call be called? The import statement contains no clue that a DLL is requested -- the sys.path search reveals that.
Just after built-in and frozen modules.
I claim that there is nothing with the current strategy.
Thank you for thoughtfully considering and commenting at length on this issue. Lets ignore it for the moment. The other problems with PYTHONPATH are more pressing. But if those issues are solved, this one will stick out. JimA
On Wed, 1 Dec 1999, James C. Ahlstrom wrote:
This seems logical -- Python extensions must live in directories that Python searches (Python must do its own search because the search order is significant).
The PYTHONPATH search path is what I am trying to get away from. If I eliminate PYTHONPATH I still can not use the Windows DLL search path (which is superior) because DLLs are searched on PYTHONPATH too; thus my post. I don't believe it is important for Python module.dll to be located on PYTHONPATH.
Why is the DLL search path superior? In my experience, the DLL search path (PATH for short) is problematic because it requires either using the System control panel or modifying autoexec.bat, both of which can have massive systemic effects completely unrelated to Python if a mistake is made during the modification. On UNIX, the equivalent to Windows' PATH is typically LD_LIBRARY_PATH, although I think there are significant variations in how that works across platforms. Most beginning unix users have no idea how to modify their LD_LIBRARY_PATH, as they typically don't understand the configuration mechanisms on Unix (system vs. user-specific, login vs. shell-specific, different shell configuration languages, etc.). I know it's not what you had in mind, but have you tried doing something like: import sys, os, string sys.path.extend(string.split(os.environ['PATH'], ';')) --david
David Ascher wrote:
On Wed, 1 Dec 1999, James C. Ahlstrom wrote:
This seems logical -- Python extensions must live in directories that Python searches (Python must do its own search because the search order is significant).
The PYTHONPATH search path is what I am trying to get away from. If I eliminate PYTHONPATH I still can not use the Windows DLL search path (which is superior) because DLLs are searched on PYTHONPATH too; thus my post. I don't believe it is important for Python module.dll to be located on PYTHONPATH.
Why is the DLL search path superior?
In my experience, the DLL search path (PATH for short)
Make that: [ os.path.dirname(sys.executable), os.getcwd(), win32api.GetSystemDirectory(), os.path.join(win32api.GetSystemDirectory(), '../SYSTEM'), win32api.GetWindowsDirectory() ] + string.split(os.environ['PATH'], ';')
is problematic because it requires either using the System control panel or modifying autoexec.bat, both of which can have massive systemic effects completely unrelated to Python if a mistake is made during the modification.
Hear, hear! [snip] - Gordon
David Ascher wrote:
Why is the DLL search path superior?
In my experience, the DLL search path (PATH for short) is problematic because it requires either using the System control panel or modifying autoexec.bat, both of which can have massive systemic effects completely unrelated to Python if a mistake is made during the modification.
I agree that altering PATH is problematic. So is altering PYTHONPATH and for exactly the same reason. That is why I think PYTHONPATH is a bad idea. The reason the DLL search path is superior is that it is not just PATH. It defines a path which includes the install directory of the application plus the system directories, and this path is discovered at runtime. So it is not necessary to set a global PYTHONPATH, nor make registry entries, nor do anything at all. It Just Works. The Windows DLL search path is: 1) The directory of the executable program. That means you can just throw all your DLL's in with the *.exe's, and it all Just Works. 2) The current directory. Also useful. 3) The Windows system directory (call GetSystemDirectory() to get this). 4) The Windows directory (call GetWindowsDirectory() to get this). These two directories are used for system files. Think of /sbin, /bin. Windows apps usually throw some of their DLL's here, especially if they are of general interest. 5) The directories in PATH. This is relatively useless, and AFAIK it is seldom used in a real installation. It is a left-over from DOS. That is also why it appears last.
On UNIX, the equivalent to Windows' PATH is typically LD_LIBRARY_PATH, although I think there are significant variations in how that works across platforms. Most beginning unix users have no idea how to modify their LD_LIBRARY_PATH, as they typically don't understand the configuration mechanisms on Unix (system vs. user-specific, login vs. shell-specific, different shell configuration languages, etc.).
I agree.
I know it's not what you had in mind, but have you tried doing something like:
import sys, os, string sys.path.extend(string.split(os.environ['PATH'], ';'))
Adding PATH (or anything else) to PYTHONPATH is making it worse. Have you tried "import sys; print sys.path" on Windows? It is junk. JimA
Gordon McMillan wrote:
Make that: [ os.path.dirname(sys.executable), os.getcwd(), win32api.GetSystemDirectory(), os.path.join(win32api.GetSystemDirectory(), '../SYSTEM'), win32api.GetWindowsDirectory() ] + string.split(os.environ['PATH'], ';')
Very nice! "../SYSTEM" needed on NT I guess. JimA
James C. Ahlstrom
Adding PATH (or anything else) to PYTHONPATH is making it worse. Have you tried "import sys; print sys.path" on Windows? It is junk.
not on my machine. it would help if you stopped assuming that every- one have the same problems as you have. we've distributed several python apps on windows, and frankly, I don't understand what you're talking about. </F>
Fredrik Lundh wrote:
you tried "import sys; print sys.path" on Windows? It is junk.
not on my machine.
On my Windows machine I get: ['', '.', 'N:/prd/winlease/vest', '.\\DLLs', '.\\lib', '.\\lib\\plat-win', '.\\lib\\lib-tk', 'f:\\bin'] PYTHONPATH is N:/prd/winlease/vest. os.path.dirname(sys.executable) is F:/bin. The others are junk. What do you get? Did you change sys.path from the default?
it would help if you stopped assuming that every- one have the same problems as you have. we've distributed several python apps on windows, and frankly, I don't understand what you're talking about.
We distribute our app by freezing all *.py files into a DLL, and we don't set PYTHONPATH on the target machine. The files are located with the executable file and are found there. This works fine and we don't have a problem with it. It would help me a lot if you could describe how you distribute your app. Do you set PYTHONPATH on the target machine? JimA
On Wed, 1 Dec 1999, James C. Ahlstrom wrote:
In my experience, the DLL search path (PATH for short) is problematic because it requires either using the System control panel or modifying autoexec.bat, both of which can have massive systemic effects completely unrelated to Python if a mistake is made during the modification.
I agree that altering PATH is problematic. So is altering PYTHONPATH and for exactly the same reason. That is why I think PYTHONPATH is a bad idea.
I see. Thanks for the explanation. I didn't know the complete story of the "Windows DLL search path". BTW, I think a huge difference b/w PYTHONPATH and PATH is the system-wide nature of PATH, vs. the Python-restriced nature of PYTHONPATH. --david
I see. Thanks for the explanation. I didn't know the complete story of the "Windows DLL search path". BTW, I think a huge difference b/w PYTHONPATH and PATH is the system-wide nature of PATH, vs. the Python-restriced nature of PYTHONPATH.
And more to the point - and the critical distinction - is that PYTHONPATH is actually specific to the Python _app_, not just Python on the machine. Sure - the standard Python installation puts a "default" PYTHONPATH suitable for general purpose development - but any distributed application _can_ define their own PYTHONPATH that is independant of any other Python systems or applications. People have been doing this for years, including MS :-) Sorry Jim, but count this as another vote against it - which isnt to argue that the current system is perfect, simply (IMO) better than the Windows path and DLL search order. Mark.
Fredrik Lundh wrote:
you tried "import sys; print sys.path" on Windows? It is junk.
not on my machine.
On my Windows machine I get:
['', '.', 'N:/prd/winlease/vest', '.\\DLLs', '.\\lib', '.\\lib\\plat-win', '.\\lib\\lib-tk', 'f:\\bin']
PYTHONPATH is N:/prd/winlease/vest. os.path.dirname(sys.executable) is F:/bin. The others are junk. What do you get? Did you change sys.path from the default?
You must not have used the standard Python installer; if you had used it you wouldn't have had this problem (and perhaps we wouldn't have had this discussion). The problem is that you apparently have installed python.exe in f:\bin. "Modern" Python versions execute some code at startup that comes up with a suitable value for sys.path; the Windows version of this code is in PC/getpathp.c -- I recommend that you study it. This code tries to find the Python install directory by looking for a "landmark" file relative to the executable path, and then adds a bunch of directory entries to the path relative to the install directory. If it fails, it defaults to "." for the install directory. The entries '.\\DLLs', '.\\lib', '.\\lib\\plat-win', '.\\lib\\lib-tk' are all a result of this failing. As long as this works, there is no need for the user (or anyone) to ever set the PYTHONPATH variable -- that variable is only needed to add directories in front of sys.path for stuff that getpathp.c doesn't know about (e.g. PIL, Numeric, etc.). With packagized versions of those modules, even that won't be necessary, because the packages will be dropped in the Python install directory (typically C:\Program Files\Python). I believe that most of your desire to get rid of PYTHONPATH comes from your insistence to bypass the default installer. There's probably a way to install your app in such a way that the getpathp.c algorithm actually succeeds? There's also a separate env variable, PYTHONHOME, which overrides the Python install directory; if getpathp.c sees that it is set, it will bypass the search relative to the executable's path. I take blame for not documenting all this well enough. However I wish you stopped criticizing the design -- I think the design is quite solid. --Guido van Rossum (home page: http://www.python.org/~guido/)
This seems logical -- Python extensions must live in directories that Python searches (Python must do its own search because the search order is significant).
The PYTHONPATH search path is what I am trying to get away from. If I eliminate PYTHONPATH I still can not use the Windows DLL search path (which is superior) because DLLs are searched on PYTHONPATH too; thus my post. I don't believe it is important for Python module.dll to be located on PYTHONPATH.
But I do. First of all, I'm not sure whether you're talking here about sys.path or PYTHONPATH. As I explained in a previous post, you should normally not have to set PYTHONPATH at all. Let's assume you really meant sys.path. Let's assume sys.path is [A, B]. Let's assume there's a foo.py and a foo.dll. If foo.py lives in A and foo.dll lives in B, then import foo should load foo.py. If it's the other way around, it should load foo.dll. If we were to use the default DLL search path, there's no way that we can get this behavior: either you have to look for a DLL first, which means there's no way for foo.py to override foo.dll, or you have to look for a DLL last, and then there's no way for a foo.dll to override foo.py. It is desirable that both overrides are possible: we want to be able to have foo.dll override foo.py, because perhaps foo.py should only be used when for some reason foo.dll can't be loaded (say foo.py does the same thing only slower); but we also want to be able to have foo.py override foo.dll (by simply placing it in a directory that's earlier on the path) e.g. in a situation where the dll version does something undesirable and we want to create a safe substitute. (Deleting files is not always an option.)
The problem is maintaining PYTHONPATH plus having DLL's on a non-standard search path.
I've commented already that PYTHONPATH maintenance is probably a red herring due to your non-standard install. I'm not sure what the problem is with having a DLL on a non-std path?
Yes, PythonDev[:] and professional SysAdmins can do it. But it is not as simple as it could be. Someone has to write the install scripts.
The distutil-sig (a.k.a. Greg Ward :-) is taking care of this as we speak.
And what if something doesn't work? Think of Python being used as a teaching language for the 8th grade. Think of the 8th grade teacher trying to get all this right. The only thing that works is simplicity.
We will provide an installer that Just Works [tm].
But at what point should this LoadLibrary() call be called? The import statement contains no clue that a DLL is requested -- the sys.path search reveals that.
Just after built-in and frozen modules.
See my long comment above.
I claim that there is nothing with the current strategy.
Thank you for thoughtfully considering and commenting at length on this issue. Lets ignore it for the moment. The other problems with PYTHONPATH are more pressing. But if those issues are solved, this one will stick out.
And those other issues should be resolved in a different way than what you have been proposing. See other post. --Guido van Rossum (home page: http://www.python.org/~guido/)
Mark Hammond wrote:
Sure - the standard Python installation puts a "default" PYTHONPATH suitable for general purpose development - but any distributed application _can_ define their own PYTHONPATH that is independant of any other Python systems or applications. People have been doing this for years, including MS :-)
How is this done?
Sorry Jim, but count this as another vote against it - which isnt to argue that the current system is perfect, simply (IMO) better than the Windows path and DLL search order.
Sigh..... JimA
Guido van Rossum wrote:
You must not have used the standard Python installer; if you had used it you wouldn't have had this problem (and perhaps we wouldn't have had this discussion).
Correct, I did not use the standard Python installer. I compiled Python from the source distribution. There are good reasons for this in my case. First, my real issue is how to DISTRIBUTE Python programs, not to get Python working on my own machine. We have 12 machines on a network. It is not acceptable to run a Python installation script on every one of them just to run a simple Python program. OK, I guess I could do 12, but what about a larger company? And we ship to hundreds of customers. I can distribute simple C or C++ programs without a hassle, why not Python? It is not acceptable to ask our customers to run a separate Python installer. We have our own Wise installer to install our software. Every commercial vendor has Wise, Install Shield or other installer in place. No commercial vendor is going to abandon Wise et al. and move to The Official Python Installer because it will not have the features of Wise (such as binary patches across the network), and because what it does won't be documented, and because it is Just Different. Second, I can not run ANY installer on my development machine, Python or otherwise. This is a general Windows problem not specific to Python. Right now our help system is broken on every office machine except the one where the help system installer was run (where we develop help). If I run a Python installer, it may Just Work here. So testing is fine, but when I distribute the program to customers where the install program has not been run it fails. The installer made registry entries, installed files, etc. And what did it do?? No one knows. And how do I install at a customer site if I don't have documentation on what the Help installer or Python installer did?? No one knows. Who fixes it if something goes wrong?? Hours on the phone to Help System customer support. Does it work on Windows 2000?? No one knows.
f:\bin. "Modern" Python versions execute some code at startup that comes up with a suitable value for sys.path; the Windows version of this code is in PC/getpathp.c -- I recommend that you study it. This
[ Highly useful discussion of startup...]
Thank you, I will study this.
know about (e.g. PIL, Numeric, etc.). With packagized versions of those modules, even that won't be necessary, because the packages will be dropped in the Python install directory (typically C:\Program Files\Python).
Yes, this is essential. Packages must be easily installed. I was hoping for single file package archive files.
I believe that most of your desire to get rid of PYTHONPATH comes from your insistence to bypass the default installer.
Correct, I refuse to execute the default installer. And I am a patient person who loves Python, so I will read getpathp.c to see what is happening. But other commercial developers, students, teachers, SysAdmins etc. are not so patient. In the interest of promoting Python, there should be documentation on the official way to easily install Python programs.
There's probably a way to install your app in such a way that the getpathp.c algorithm actually succeeds? There's also a separate env variable, PYTHONHOME,
Perhaps, and if there is it should be prominently documented in the How to Distribute Your App section of the manual. I am worried about supporting versioning, but I will think about it.
I take blame for not documenting all this well enough. However I wish you stopped criticizing the design -- I think the design is quite solid.
Thank you for the explanation. I will study the design again. I always wondered what PYTHONHOME did. JimA
Perhaps, and if there is it should be prominently documented in the How to Distribute Your App section of the manual. I am worried about supporting versioning, but I will think about it.
Join the distutil-SIG, they are discussing just this. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Let's assume sys.path is [A, B]. Let's assume there's a foo.py and a foo.dll. If foo.py lives in A and foo.dll lives in B, then import foo ...
Thank you for the detailed discussion showing that sys.path is needed so a choice can be made whether to load foo.dll or foo.py. As you correctly point out, a separate search path defeats this behavior. But I don't think the usefulness of the feature compensates for its resultant complexity. Specifically, it will be hard to create this behavior in archive files. As I envision archive files (which of course is subject to change) they contain *.pyc files and not DLL's. The DLL's must be in a ./DLL directory since the OS can not load them from strings. So if every *.pyc is in an archive file, your only choice is whether to load all DLL's first or last. That is, archive.pyl is either before or after ./DLL. If a package (probably with lots of subdirectories) author depends on having a search path within a package which discriminates between pyc and DLL files with equal names, then that search path plus the existence of the DLL's must be recorded in the archive. This is much more complicated than just an archive with all *.pyc files entered in a dotted name space: foo foo.sub1 foo.sub2 foo.sub2.pkx I would question whether equally named foo.dll and foo.py is worth it. The alternative (which is IMHO more common) is to code the choice in Python in the module that cares about it.
And what if something doesn't work? Think of Python being used as a teaching language for the 8th grade. Think of the 8th grade teacher trying to get all this right. The only thing that works is simplicity.
We will provide an installer that Just Works [tm].
OK for this case. Not enough for Python program distribution. JimA
Guido van Rossum wrote:
Perhaps, and if there is it should be prominently documented in the How to Distribute Your App section of the manual. I am worried about supporting versioning, but I will think about it.
Join the distutil-SIG, they are discussing just this.
I already belong to the distutil-SIG and have seen no such discussion. Jim
[Jim]
Perhaps, and if there is it should be prominently documented in the How to Distribute Your App section of the manual. I am worried about supporting versioning, but I will think about it.
[me]
Join the distutil-SIG, they are discussing just this.
[Jim again]
I already belong to the distutil-SIG and have seen no such discussion.
Sorry, you're right (except for a brief exchange between you and Paul Dubois :-). But I think they should, it falls under their charter. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum writes:
Sorry, you're right (except for a brief exchange between you and Paul Dubois :-). But I think they should, it falls under their charter.
This was deliberatly postponed until after extension packages are
supported and in place. I know Greg is interested in application
installation as well as package installation.
-Fred
--
Fred L. Drake, Jr.
participants (8)
-
David Ascher -
Fred L. Drake, Jr. -
Fredrik Lundh -
Gordon McMillan -
Greg Stein -
Guido van Rossum -
James C. Ahlstrom -
Mark Hammond