Re: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
afa@codespeak.net schrieb:
Author: afa Date: Tue May 20 16:17:22 2008 New Revision: 55005
Modified: pypy/branch/win32port/pypy/lib/_ctypes/builtin.py pypy/branch/win32port/pypy/lib/msvcrt.py pypy/branch/win32port/pypy/module/_rawffi/__init__.py pypy/branch/win32port/pypy/module/_rawffi/interp_rawffi.py pypy/branch/win32port/pypy/module/_rawffi/test/test__rawffi.py pypy/branch/win32port/pypy/rlib/libffi.py pypy/branch/win32port/pypy/rlib/rwin32.py pypy/branch/win32port/pypy/rlib/test/test_libffi.py Log: Maciej was right: it's better to dynamically get the name of the msvcrt library: the same generated sources may be recompiled, e.g. in debug mode.
I hope I got it right: in pypy-c.exe, we compile a function that retrieves the dll containing the "fopen" pointer address.
+ #include <stdio.h> + + /* Get the module where the "fopen" function resides in */ + HANDLE get_libc_handle() { + MEMORY_BASIC_INFORMATION mi; + memset(&mi, 0, sizeof(mi)); + + if( !VirtualQueryEx(GetCurrentProcess(), &fopen, &mi, sizeof(mi)) ) + return 0; + + return (HMODULE)mi.AllocationBase; + } + '''],
Ha, that's the best approach I have seen so far to get the name if the msvcrt runtime library. Hope you don't mind if I steal it for 'official' ctypes (I will check if it works in win64 also before) ? -- Thanks, Thomas
Ha, that's the best approach I have seen so far to get the name if the msvcrt runtime library. Hope you don't mind if I steal it for 'official' ctypes (I will check if it works in win64 also before) ?
-- Thanks, Thomas
Cool :) Besides, can you take a look at small script here: http://codespeak.net/svn/pypy/dist/pypy/lib/ctypes_support.py? It tries to address two issues - "how to find c library" and "how to get errno", what do you think? Cheers, fijal
Maciej Fijalkowski schrieb:
Ha, that's the best approach I have seen so far to get the name if the msvcrt runtime library. Hope you don't mind if I steal it for 'official' ctypes (I will check if it works in win64 also before) ?
-- Thanks, Thomas
Cool :)
Besides, can you take a look at small script here: http://codespeak.net/svn/pypy/dist/pypy/lib/ctypes_support.py? It tries to address two issues - "how to find c library" and "how to get errno", what do you think?
Cheers, fijal
Even cooler that I can read from day to day how you reimplemented ctypes for pypy :-) "how to find c library" - is only a problem on windows (which I hope can now be solved better). On linux (not sure about *nix) simply call LoadLibrary(None) [=> dlopen(NULL) in C], and you can get all functions from all loaded shared libs from that handle. "how to get errno" - the problem with this is that errno is only valid (in the calling thread) until another stdlib library function has been called in this thread. So, since arbitrary code can run (in CPython's ctypes, at least) between the time a function in a shared lib has been called and the time you access errno there is no guarantee that it is still valid. There has been some discussion on the ctypes-users list some months ago about that, and the only reliable way to get errno for the function call seems to be to get errno immediately after the function call. The question is how to make this value available to the calling Python code. Two ways were discussed: To attach the errno value in thread local storage to the function object itself, or to pass it to the '.errcheck' function. Nothing of this has yet been implemented in ctypes. -- Thanks, Thomas
Even cooler that I can read from day to day how you reimplemented ctypes for pypy :-)
"how to find c library" - is only a problem on windows (which I hope can now be solved better). On linux (not sure about *nix) simply call LoadLibrary(None) [=> dlopen(NULL) in C], and you can get all functions from all loaded shared libs from that handle.
No. That does not sound to me like a robust solution. (We should probably move this discussion to ctypes-users though). Because you can have strange stuff in there and I really want to access libc, not something else. I think that helper in ctypes that gives you *exactly* libc, not only on windows is a good thing to have.
"how to get errno" - the problem with this is that errno is only valid (in the calling thread) until another stdlib library function has been called in this thread.
So, since arbitrary code can run (in CPython's ctypes, at least) between the time a function in a shared lib has been called and the time you access errno there is no guarantee that it is still valid. There has been some discussion on the ctypes-users list some months ago about that, and the only reliable way to get errno for the function call seems to be to get errno immediately after the function call. The question is how to make this value available to the calling Python code. Two ways were discussed: To attach the errno value in thread local storage to the function object itself, or to pass it to the '.errcheck' function. Nothing of this has yet been implemented in ctypes.
Ah! Good :) I think I should start reading ctypes mailing list. We have support for such stuff already, as we need to get errno when testing posix stuff (I can explain in details if you like :) It sounds like vastly better solution that ours, please implement it, we'll follow :) Cheers, fijal
Maciej Fijalkowski schrieb:
Even cooler that I can read from day to day how you reimplemented ctypes for pypy :-)
"how to find c library" - is only a problem on windows (which I hope can now be solved better). On linux (not sure about *nix) simply call LoadLibrary(None) [=> dlopen(NULL) in C], and you can get all functions from all loaded shared libs from that handle.
No. That does not sound to me like a robust solution. (We should probably move this discussion to ctypes-users though).
I've cc'd to ctypes.users.
Because you can have strange stuff in there and I really want to access libc, not something else. I think that helper in ctypes that gives you *exactly* libc, not only on windows is a good thing to have.
That may be, but AFAICT it doesn't work this way on linux. Once you have a loaded shared libraries (in the current process), you can access *all* exported functions from this process from it: theller@tubu610:~$ python Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from ctypes import * from ctypes.util import find_library m = CDLL(find_library("m")) m <CDLL 'libm.so.6', handle b7f652b0 at b7d867ac> c = CDLL(find_library("c")) c <CDLL 'libc.so.6', handle b7f65558 at b7d8606c> c.printf <_FuncPtr object at 0xb7da41cc> m.printf <_FuncPtr object at 0xb7da4234> c.printf("hello, world\n") hello, world 13 m.printf("hello, world\n") hello, world 13 find_library("c") 'libc.so.6' find_library("m") 'libm.so.6'
It may be possible to get different behaviour when the libraries are loaded using different RTLD_ flags.
"how to get errno" - the problem with this is that errno is only valid (in the calling thread) until another stdlib library function has been called in this thread.
So, since arbitrary code can run (in CPython's ctypes, at least) between the time a function in a shared lib has been called and the time you access errno there is no guarantee that it is still valid. There has been some discussion on the ctypes-users list some months ago about that, and the only reliable way to get errno for the function call seems to be to get errno immediately after the function call. The question is how to make this value available to the calling Python code. Two ways were discussed: To attach the errno value in thread local storage to the function object itself, or to pass it to the '.errcheck' function. Nothing of this has yet been implemented in ctypes.
Ah! Good :) I think I should start reading ctypes mailing list. We have support for such stuff already, as we need to get errno when testing posix stuff (I can explain in details if you like :)
It sounds like vastly better solution that ours, please implement it, we'll follow :)
There's an entry in the python bug tracker at http://bugs.python.org/issue1798 where you could add comments if you have any.
Cheers, fijal
-- Thanks, Thomas
I've cc'd to ctypes.users.
Thank you, subscribed
theller@tubu610:~$ python Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from ctypes import * from ctypes.util import find_library m = CDLL(find_library("m")) m <CDLL 'libm.so.6', handle b7f652b0 at b7d867ac> c = CDLL(find_library("c")) c <CDLL 'libc.so.6', handle b7f65558 at b7d8606c> c.printf <_FuncPtr object at 0xb7da41cc> m.printf <_FuncPtr object at 0xb7da4234> c.printf("hello, world\n") hello, world 13 m.printf("hello, world\n") hello, world 13 find_library("c") 'libc.so.6' find_library("m") 'libm.so.6'
It may be possible to get different behaviour when the libraries are loaded using different RTLD_ flags.
yes, it depends on RTLD_GLOBAL vs RTLD_LOCAL. Note, that unless you specify RTLD_GLOBAL, you don't get this behavior (RTLD_LOCAL is the default for C and accidentally for pypy as well). For me details on top of cpython are slightly different and it's very platform dependent. (ie libm has no qsort, but libc has sqrt, obscure). I would rather like to keep it clean and not populate namespace if not necessary, hence libc helper finder would be cool. Thomas: thanks for feedback Cheers, fijal
participants (2)
-
Maciej Fijalkowski -
Thomas Heller