
Rebooting the thread... While testing mxNumber, I discovered what looks like a bug in Win95: both Thomas Heller and I are seeing a problem with Python 2.1 when importing extension modules which rely on other DLLs as library. Interestingly, the problem only shows up when starting Python from the installation directory. Looking at the imports using python -vv shows that in this situation, Python tries to import modules, packages, extensions etc. using *relative* paths. Now, under Win98 there is no problem, but Win95 doesn't seem to like these relative imports when it comes to .pyd files which reference DLLs in the same directory. It doesn't have any problems when Python is started outside the installation dir, since in that case Python uses absolute paths for importing modules and extensions. Would it be hard to tweak Python into always using absolute search paths during module import ? -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

I'm not quite with you here. Are you saying both Win98 and 95 use relative paths, but only Win95 has the problem, or that only Win95 sees relative paths? My Win98 box uses absolute paths for all imports when using -vv
Would it be hard to tweak Python into always using absolute search paths during module import ?
Where are the relative paths coming from? If we can determine that, we can determine how hard it would be to fix ;-) Mark.

Mark Hammond wrote:
Both are using relative paths, but only Win95 has a problem with not finding the DLL needed by a PYD file in a subpackage: abc/ __init__.py mxABC.pyd mamba.dll mxABC.pyd needs mamba.dll.
My Win98 box uses absolute paths for all imports when using -vv
Are you sure ? Please CD to the C:\Python21 dir and then try to import and installed package (say mx.Tools from egenix-mx-base). You should be seeing relative paths with -vv.
The relative paths come from the import logic: the current dir is scanned first and if the package is found in that directory, all subsequent lookups are done using relative paths. While this is prefectly OK, Win95 seems to have a problem with importing extensions using these relative paths. I think we could solve the problem by making the pathname which is passed to LoadLibraryEx() in dynload_win.c absolute. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

Mark Hammond wrote:
Here's a stab at a patch. Could you review it and test it ? I don't have enough knowledge of win32 for this... dynload_win.c: ... #ifdef MS_WIN32 { HINSTANCE hDLL; char pathbuf[260]; LPTSTR *dummy; if (strchr(pathname, '\\') == NULL && strchr(pathname, '/') == NULL) { /* Prefix bare filename with ".\" */ char *p = pathbuf; *p = '\0'; _getcwd(pathbuf, sizeof pathbuf); if (*p != '\0' && p[1] == ':') p += 2; sprintf(p, ".\\%-.255s", pathname); pathname = pathbuf; } /* Convert to full pathname; we ignore errors to maintain b/w compatibility here. */ if (GetFullPathName(pathname, sizeof(pathbuf), pathbuf, &dummy)) pathname = pathbuf; /* Look for dependent DLLs in directory of pathname first */ /* XXX This call doesn't exist in Windows CE */ if (pathname) hDLL = LoadLibraryEx(pathname, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); if (hDLL == NULL) { char errBuf[256]; unsigned int errorCode; ... Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

Here's a stab at a patch. Could you review it and test it ? I don't have enough knowledge of win32 for this...
I think we can drop the getcwd call here completely. I prefer the patch below. Mark. Index: dynload_win.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/dynload_win.c,v retrieving revision 2.7 diff -u -r2.7 dynload_win.c --- dynload_win.c 2000/10/05 10:54:45 2.7 +++ dynload_win.c 2001/05/01 00:36:40 @@ -163,24 +163,21 @@ #ifdef MS_WIN32 { - HINSTANCE hDLL; + HINSTANCE hDLL = NULL; char pathbuf[260]; - if (strchr(pathname, '\\') == NULL && - strchr(pathname, '/') == NULL) - { - /* Prefix bare filename with ".\" */ - char *p = pathbuf; - *p = '\0'; - _getcwd(pathbuf, sizeof pathbuf); - if (*p != '\0' && p[1] == ':') - p += 2; - sprintf(p, ".\\%-.255s", pathname); - pathname = pathbuf; - } - /* Look for dependent DLLs in directory of pathname first */ - /* XXX This call doesn't exist in Windows CE */ - hDLL = LoadLibraryEx(pathname, NULL, - LOAD_WITH_ALTERED_SEARCH_PATH); + LPTSTR dummy; + /* We use LoadLibraryEx so Windows looks for dependent DLLs + in directory of pathname first. However, Windows95 + can sometimes not work correctly unless the absolute + path is used. If GetFullPathName() fails, the LoadLibrary + will certainly fail too, so use its error code */ + if (GetFullPathName(pathname, + sizeof(pathbuf), + pathbuf, + &dummy)) + /* XXX This call doesn't exist in Windows CE */ + hDLL = LoadLibraryEx(pathname, NULL, + LOAD_WITH_ALTERED_SEARCH_PATH); if (hDLL==NULL){ char errBuf[256]; unsigned int errorCode;

Mark Hammond wrote:
If this works as expected, please check in the patch. (Note that I have not tested the patch I posted -- I've never used VC++ for anything else than compiling C extensions and GMP.)
-- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

Would it be hard to tweak Python into always using absolute search paths during module import ?
I resist doing this *in general* because absolute paths don't always mean the right thing on all platforms (and aren't always obtainable), but I agree that for Windows this can and should be done. The easiest way I can think of is for PC/getpathp.py to tweak the path entries to be absolute pathnames. A single getcwd() call should suffice. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I have only run into the problem with Win95, so yes, doing this for Windows only should be OK (and even there it's probably only needed for importing PYD files).
I'd rather keep the changes in dynload_win.c if that's possible. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

I'd rather keep the changes in dynload_win.c if that's possible.
Fine with me. --Guido van Rossum (home page: http://www.python.org/~guido/)

I'm not quite with you here. Are you saying both Win98 and 95 use relative paths, but only Win95 has the problem, or that only Win95 sees relative paths? My Win98 box uses absolute paths for all imports when using -vv
Would it be hard to tweak Python into always using absolute search paths during module import ?
Where are the relative paths coming from? If we can determine that, we can determine how hard it would be to fix ;-) Mark.

Mark Hammond wrote:
Both are using relative paths, but only Win95 has a problem with not finding the DLL needed by a PYD file in a subpackage: abc/ __init__.py mxABC.pyd mamba.dll mxABC.pyd needs mamba.dll.
My Win98 box uses absolute paths for all imports when using -vv
Are you sure ? Please CD to the C:\Python21 dir and then try to import and installed package (say mx.Tools from egenix-mx-base). You should be seeing relative paths with -vv.
The relative paths come from the import logic: the current dir is scanned first and if the package is found in that directory, all subsequent lookups are done using relative paths. While this is prefectly OK, Win95 seems to have a problem with importing extensions using these relative paths. I think we could solve the problem by making the pathname which is passed to LoadLibraryEx() in dynload_win.c absolute. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

Mark Hammond wrote:
Here's a stab at a patch. Could you review it and test it ? I don't have enough knowledge of win32 for this... dynload_win.c: ... #ifdef MS_WIN32 { HINSTANCE hDLL; char pathbuf[260]; LPTSTR *dummy; if (strchr(pathname, '\\') == NULL && strchr(pathname, '/') == NULL) { /* Prefix bare filename with ".\" */ char *p = pathbuf; *p = '\0'; _getcwd(pathbuf, sizeof pathbuf); if (*p != '\0' && p[1] == ':') p += 2; sprintf(p, ".\\%-.255s", pathname); pathname = pathbuf; } /* Convert to full pathname; we ignore errors to maintain b/w compatibility here. */ if (GetFullPathName(pathname, sizeof(pathbuf), pathbuf, &dummy)) pathname = pathbuf; /* Look for dependent DLLs in directory of pathname first */ /* XXX This call doesn't exist in Windows CE */ if (pathname) hDLL = LoadLibraryEx(pathname, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); if (hDLL == NULL) { char errBuf[256]; unsigned int errorCode; ... Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

Here's a stab at a patch. Could you review it and test it ? I don't have enough knowledge of win32 for this...
I think we can drop the getcwd call here completely. I prefer the patch below. Mark. Index: dynload_win.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/dynload_win.c,v retrieving revision 2.7 diff -u -r2.7 dynload_win.c --- dynload_win.c 2000/10/05 10:54:45 2.7 +++ dynload_win.c 2001/05/01 00:36:40 @@ -163,24 +163,21 @@ #ifdef MS_WIN32 { - HINSTANCE hDLL; + HINSTANCE hDLL = NULL; char pathbuf[260]; - if (strchr(pathname, '\\') == NULL && - strchr(pathname, '/') == NULL) - { - /* Prefix bare filename with ".\" */ - char *p = pathbuf; - *p = '\0'; - _getcwd(pathbuf, sizeof pathbuf); - if (*p != '\0' && p[1] == ':') - p += 2; - sprintf(p, ".\\%-.255s", pathname); - pathname = pathbuf; - } - /* Look for dependent DLLs in directory of pathname first */ - /* XXX This call doesn't exist in Windows CE */ - hDLL = LoadLibraryEx(pathname, NULL, - LOAD_WITH_ALTERED_SEARCH_PATH); + LPTSTR dummy; + /* We use LoadLibraryEx so Windows looks for dependent DLLs + in directory of pathname first. However, Windows95 + can sometimes not work correctly unless the absolute + path is used. If GetFullPathName() fails, the LoadLibrary + will certainly fail too, so use its error code */ + if (GetFullPathName(pathname, + sizeof(pathbuf), + pathbuf, + &dummy)) + /* XXX This call doesn't exist in Windows CE */ + hDLL = LoadLibraryEx(pathname, NULL, + LOAD_WITH_ALTERED_SEARCH_PATH); if (hDLL==NULL){ char errBuf[256]; unsigned int errorCode;

Mark Hammond wrote:
If this works as expected, please check in the patch. (Note that I have not tested the patch I posted -- I've never used VC++ for anything else than compiling C extensions and GMP.)
-- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

Would it be hard to tweak Python into always using absolute search paths during module import ?
I resist doing this *in general* because absolute paths don't always mean the right thing on all platforms (and aren't always obtainable), but I agree that for Windows this can and should be done. The easiest way I can think of is for PC/getpathp.py to tweak the path entries to be absolute pathnames. A single getcwd() call should suffice. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I have only run into the problem with Win95, so yes, doing this for Windows only should be OK (and even there it's probably only needed for importing PYD files).
I'd rather keep the changes in dynload_win.c if that's possible. -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

I'd rather keep the changes in dynload_win.c if that's possible.
Fine with me. --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (3)
-
Guido van Rossum
-
M.-A. Lemburg
-
Mark Hammond