[Python-Dev] Re: .DLL vs .PYD search order

[Guido van Rossum]

> > This seems logical -- Python extensions must live in directories that
> > Python searches (Python must do its own search because the search
> > order is significant).
> 
> The PYTHONPATH search path is what I am trying to get away
> from.  If I eliminate PYTHONPATH I still can not use the
> Windows DLL search path (which is superior) because DLLs
> are searched on PYTHONPATH too; thus my post.  I don't believe
> it is important for Python module.dll to be located on PYTHONPATH.

But I do.

First of all, I'm not sure whether you're talking here about sys.path
or PYTHONPATH.  As I explained in a previous post, you should normally
not have to set PYTHONPATH at all.  Let's assume you really meant
sys.path.

Let's assume sys.path is [A, B].  Let's assume there's a foo.py and a
foo.dll.  If foo.py lives in A and foo.dll lives in B, then import foo
should load foo.py.  If it's the other way around, it should load
foo.dll.  If we were to use the default DLL search path, there's no
way that we can get this behavior: either you have to look for a DLL
first, which means there's no way for foo.py to override foo.dll, or
you have to look for a DLL last, and then there's no way for a foo.dll
to override foo.py.  It is desirable that both overrides are possible:
we want to be able to have foo.dll override foo.py, because perhaps
foo.py should only be used when for some reason foo.dll can't be
loaded (say foo.py does the same thing only slower); but we also want
to be able to have foo.py override foo.dll (by simply placing it in a
directory that's earlier on the path) e.g. in a situation where the
dll version does something undesirable and we want to create a safe
substitute.  (Deleting files is not always an option.)

> The problem is maintaining PYTHONPATH plus having DLL's on a
> non-standard search path.

I've commented already that PYTHONPATH maintenance is probably a red
herring due to your non-standard install.  I'm not sure what the
problem is with having a DLL on a non-std path?

> Yes, PythonDev[:] and professional
> SysAdmins can do it.  But it is not as simple as it could be.
> Someone has to write the install scripts.

The distutil-sig (a.k.a. Greg Ward :-) is taking care of this as we
speak.

> And what if something
> doesn't work?  Think of Python being used as a teaching language
> for the 8th grade.  Think of the 8th grade teacher trying to get
> all this right.  The only thing that works is simplicity.

We will provide an installer that Just Works [tm].

> > But at what point should this LoadLibrary() call be called?  The
> > import statement contains no clue that a DLL is requested -- the
> > sys.path search reveals that.
> 
> Just after built-in and frozen modules.

See my long comment above.

> > I claim that there is nothing with the current strategy.
> 
> Thank you for thoughtfully considering and commenting at length
> on this issue.  Lets ignore it for the moment.  The other
> problems with PYTHONPATH are more pressing.  But if those
> issues are solved, this one will stick out.

And those other issues should be resolved in a different way than what
you have been proposing.  See other post.

--Guido van Rossum (home page: http://www.python.org/~guido/)