Hello,
I tried to compile some extensions under Windows/Python 2.4 using distutils and mingw as compiler (not MS-VC++).
----- (You can just jump to conclusion at the bottom if you like, but be aware that I don't really master the subject, so I may have missed something. If you thing my conclusions doesn't make sense, please read what happens to me.) -----
(Official Python2.4, the one that use msvcr71.dll)
I got some strange results like "the softwares I wrote using this extension crashed randomly". I first thought that extension (that I didn't wrote) was crappy.
When the second extension crashed too, I tried to understand a bit more.
The crash is more or less random, (but when it doesn't crash, everything works as expected even the code in the extension) and it crash with an OS error 0xc0000008 (it look like it's an internal windows error which mean there is something really nasty with the memory.
So the .pyd was linked to msvcrt.dll and to msvcr71.dll. As it looks fishy, I tried to reinstall python 2.3 for windows that doesn't use msvcr71.dll but msvcrt.dll).
I wasn't able to reproduce the crash.
Then I realize the extension provided a way to build the same .pyd (but I wasn't able to generate an installer for the extension, that's why I didn't use it).
So I look at the .pyd generated for python 2.4 with the makefile, and... it was only linked to msvcrt.dll. So I trie to use it instead and... it works !!!
I tried to recompile everything from scratch using distutils because it may just be because a "bad neutrino who hit my motherboard at the worst moment", but the bug was still here.
So I tried to figure out what was the difference between the command lines used by the makefile and distutils. And... the obvious difference was that distutils used a -lmsvcr71.
Gooood
I looked at distutils code source, and quickly found the line that makes mingw use msvcr71. IV just changed it.
I rebuild all, and... It just worked well. (and of course, the .pyd was just linked to msvcrt and not msvcr71 anymore)
--------------
Sooooooooooooooooo here are my conclusions :
Something isn't right when compiling with mingw using msvcr71. (command line option ? problem with mingw ? problem with msvcr71 ? mingw just need some more configuration to works with msvcr71 ? I don't really know)
It *look* like compilling extension using msvcrt instad of msvcr71, even if python use msvcr71 is safer that letting mingw using msvcr71.
So I propose to remove the lines from 330 to 332 on the file cygwinccompiler.py (revision 1.29, the actual one, and the one used with python 2.4)
The code is specific for Windows/Mingw/(python compiled with msvc 7.1) so there shouldn't be any side effect.
Perahps it's wise to do the same for the line that add "msvcr70" (but I don't know).
Perhaps it's wise to do the same on lines 130-135 (The same code, but for cygwin's internal gcc, and not mingw's gcc), but I don't know if cygwin suffer for the same problems.
I just tried to compile a bunch of C extensions (coming from the biopython project) with mingw as compiler. Most of them link to msvcr71.dll and not to msvcrt.dll, except for one, which links to both. By uncommenting various parts of the code, I discovered that the culprit is a call to strdup. When such a call is made, the extension module will link to both msvcr71.dll and msvcrt.dll. Now, strdup is not part of ANSI-C, so for portability it is better to avoid this function altogether. I don't know if your extension module links to msvcrt.dll for the same reason, but you may be able to find out which part of the code is causing the .pyd to be linked to msvcrt. It may turn out to be strdup or another non-standard function.
--Michiel.
AGiss wrote:
Hello,
I tried to compile some extensions under Windows/Python 2.4 using distutils and mingw as compiler (not MS-VC++).
(You can just jump to conclusion at the bottom if you like, but be aware that I don't really master the subject, so I may have missed something. If you thing my conclusions doesn't make sense, please read what happens to me.)
(Official Python2.4, the one that use msvcr71.dll)
I got some strange results like "the softwares I wrote using this extension crashed randomly". I first thought that extension (that I didn't wrote) was crappy.
When the second extension crashed too, I tried to understand a bit more.
The crash is more or less random, (but when it doesn't crash, everything works as expected even the code in the extension) and it crash with an OS error 0xc0000008 (it look like it's an internal windows error which mean there is something really nasty with the memory.
So the .pyd was linked to msvcrt.dll and to msvcr71.dll. As it looks fishy, I tried to reinstall python 2.3 for windows that doesn't use msvcr71.dll but msvcrt.dll).
I wasn't able to reproduce the crash.
Then I realize the extension provided a way to build the same .pyd (but I wasn't able to generate an installer for the extension, that's why I didn't use it).
So I look at the .pyd generated for python 2.4 with the makefile, and... it was only linked to msvcrt.dll. So I trie to use it instead and... it works !!!
I tried to recompile everything from scratch using distutils because it may just be because a "bad neutrino who hit my motherboard at the worst moment", but the bug was still here.
So I tried to figure out what was the difference between the command lines used by the makefile and distutils. And... the obvious difference was that distutils used a -lmsvcr71.
Gooood
I looked at distutils code source, and quickly found the line that makes mingw use msvcr71. IV just changed it.
I rebuild all, and... It just worked well. (and of course, the .pyd was just linked to msvcrt and not msvcr71 anymore)
Sooooooooooooooooo here are my conclusions :
Something isn't right when compiling with mingw using msvcr71. (command line option ? problem with mingw ? problem with msvcr71 ? mingw just need some more configuration to works with msvcr71 ? I don't really know)
It *look* like compilling extension using msvcrt instad of msvcr71, even if python use msvcr71 is safer that letting mingw using msvcr71.
So I propose to remove the lines from 330 to 332 on the file cygwinccompiler.py (revision 1.29, the actual one, and the one used with python 2.4)
The code is specific for Windows/Mingw/(python compiled with msvc 7.1) so there shouldn't be any side effect.
Perahps it's wise to do the same for the line that add "msvcr70" (but I don't know).
Perhaps it's wise to do the same on lines 130-135 (The same code, but for cygwin's internal gcc, and not mingw's gcc), but I don't know if cygwin suffer for the same problems.
Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On Sun, Apr 03, 2005 at 06:08:34PM +0900, Michiel Jan Laurens de Hoon wrote:
I just tried to compile a bunch of C extensions (coming from the biopython project) with mingw as compiler. Most of them link to msvcr71.dll and not to msvcrt.dll, except for one, which links to both. By uncommenting various parts of the code, I discovered that the culprit is a call to strdup. When such a call is made, the extension module will link to both msvcr71.dll and msvcrt.dll. Now, strdup is not part of ANSI-C, so for portability it is better to avoid this function altogether. I don't know if your extension module links to msvcrt.dll for the same reason, but you may be able to find out which part of the code is causing the .pyd to be linked to msvcrt. It may turn out to be strdup or another non-standard function.
--Michiel.
The extension is "clearsilver" (http://www.clearsilver.net/). It's a "third-party" extension, so "not using" strdup is not a "clean" solution. (while in this case, rewritting it is quite simple, so it didn't took me much time to do it).
So the .pyd generated with distutils linked to msvcrt.dll and to msvcr71.dll
calls linked to msvcrt.dll : _close _fdopen _mkdir _mktemp _open _putenv _read _rmdir _sleep _stat _strdup _tzset _unlink _write _findclose _findfirst _findnext _fullpath
calls linked to msvcr71.dll : __dllonexit __mb_cur_max _errno _iob _isctype _pctype _stricmp _strnicmp _vsnprintf abort atoi bsearch calloc fclose fflush fgets fopen fprintf fread free fseek fwrite getenv gmtime localtime malloc memchr memcpy memmove memset mktime qsort rand realloc rename sprintf srand sscanf strcat strchr strcmp strcpy strerror strftime strlen strncat strncmp strncpy strpbrk strrchr strspn strstr strtol time tolower toupper vfprintf vprintf vsprintf
while all calls in the first section ARE exported in msvcr71.dll too (so that's not a problem of "msvcr71.dll" doesn't have them) (I suppose that msvcr71.dll export ALL the items mscvrt.dll plus some more).
And so yes, _stdup is in the list. I tried to remove the strdup calls (just in case strdup would be the *only* "trigger" to that weirdness).
So I replace all calls to strdup to strxdup, and I wrote a quick strxdup using just "strlen", "malloc" and "strncpy".
The result was ... _strdup wasn't in the list of dependences from neo_cgi.pyd to msvcrt.dll. But all others were still remaining.
And so in the list, if unlink, sleep or fdopen aren't ANSI-C, they look like POSIX to me. So I feel like using them. Am I wrong ?
So I don't know what is really happening (look like MingW32 has problems handling msvcr71.dll, but I'm far from being competent to say that), but I feel like distutils should do something to not generate .pyd that would crash python scripts.
(perhaps generate .pyd with mscvr71, then look if .pyd is linked to several run-time libs, and if yes, "downgrade" to mscvrt ?)
--AGiss
participants (2)
-
AGiss -
Michiel Jan Laurens de Hoon