[Python-3000] threading, part 2 --- + a bit of ctypes FFI worry

[Thomas Heller]

Tim Peters schrieb:
> [Josiah Carlson]
>> ...
>> Python 2.3.5 (#62, Feb  8 2005, 16:23:02) [MSC v.1200 32 bit (Intel)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import ctypes
>> >>> import threading
>> >>> import time
>> >>> def foo():
>> ...     try:
>> ...             while 1:
>> ...                     time.sleep(.01)
>> ...     finally:
>> ...             print "I quit!"
>> ...
>> >>> x = threading.Thread(target=foo)
>> >>> x.start()
>> >>> for i,j in threading._active.items():
>> ...     if j is x:
>> ...             break
>> ...
>> >>> ctypes.pythonapi.PyThreadState_SetAsyncExc(i, ctypes.py_object(Exception))
> 
> As I discovered to my chagrin when I added a similar test to the test
> suite a few days ago, that's got a subtle error on most 64-bit boxes.
> When the ctypes docs talk about passing and returning integers, they
> never explain what "integers" /means/, but it seems the docs
> implicitly have a 32-bit-only view of the world here.  In reality
> "integer" seems to mean the native C `int` type.

'ctypes.c_int' and 'ctypes.c_long' correspond to the C 'int' and 'long' types.
If you think that the docs could be clearer, please suggest changes.

> But a Python thread
> id is a native C `long` (== a Python short integer), and the code
> above fails in a baffling way on most 64-bit boxes:  the call returns
> 0 instead; i.e. the thread id isn't found, and no exception gets set.
> So I believe that needs to be:
> 
>     ctypes.pythonapi.PyThreadState_SetAsyncExc(
>         ctypes.c_long(i),
>         ctypes.py_object(Exception))
> 
> to make it portable.

Right.  A little bit more safety migt be gained by setting the argtypes attribute
of the PyThreadState_SetAsyncExc function in this way:

ctypes.pythonapi.PyThreadState_SetAsyncEx.argtypes = ctypes.c_long, ctypes.py_object

This way the wrapping of arguments is automatic.

> It's unclear to me how to write portable ctypes code in the presence
> of a gazillion integer typedefs and #defines, such as for Py_ssize_t.
> That doesn't map to a fixed C integral type cross-platform, so what
> can you do?  You're not required to answer that ;-)

This must probably be exported from the C code.  Currently ctypes has
the basic (integer) types c_byte, c_short, c_int, c_long, c_longlong, plus
their unsigned variants.  On 32-bit platforms, c_int is an alias to c_long.

Sized ints are defined: c_int8, c_int16, c_int32, c_int64, (plus the unsigned
variants again), also as aliases to the 10 basic integer types.

I *should* be possible by some checks to find out about the size
of Py_ssize_t at runtime (unless it is an configurable option)...

> Thread ids may bite us someday too.  Python casts the platform's
> notion of a thread id to C `long`, but there's no guarantee this won't
> lose information (or is even legal) on all platforms.  We'd probably
> be safer casting to, e.g., Py_uintptr_t (some thread implementions
> return an index into a kernel or library thread-info table, but at
> least some in my lifetime returned a pointer to a thread-info struct,
> and that's definitely fatter than C `long` on some boxes).
> 
>> 1
>> >>> I quit!
>> Exception in thread Thread-2:Traceback (most recent call last):
>>   File "C:\python23\lib\threading.py", line 442, in __bootstrap
>>     self.run()
>>   File "C:\python23\lib\threading.py", line 422, in run
>>     self.__target(*self.__args, **self.__kwargs)
>>   File "<stdin>", line 4, in foo
>> Exception
> 
> It's really cool that you can do this from ctypes, eh?  That's exactly
> the right level of abstraction for this attractive nuisance too ;-)

;-)

Thomas