[David Abrahams]
I can't really imagine what you're suggesting here. Code samples help.
OK :) Here is some code demonstrating my "pluggable" idea, but working backwards <wink>. (Please don't pick on the names, or where I will need to pass a param, or where I forgot to cast, etc <wink>) First, let's assume we come up with a high-level API similar to: /*** Python "auto-thread-state" API ***/ typedef void *PyATS_HANDLE; /* Get's a "cookie" for use in all subsequent auto-thread-state calls. Generally called once per application/extension. Not strictly necessary, but a vehicle to abstract PyInterpreterState should people care enough. */ PyATS_HANDLE PyAutoThreadState_Init(); /* Ensure we have Python ready to rock. This is the "slow" version that assumes nothing about Python's state, other than the handle is valid. */ int PyAutoThreadState_Ensure(PyATS_HANDLE); /* Notify the auto-thread-state mechanism that we are done - there should be one Release() per Ensure(). Again, maybe not necessary if we are super clever, but for the sake of argument ...<wink> */ void PyAutoThreadState_Release(PyATS_HANDLE); /* end of definitions */ This is almost the "holy grail" for me. Your module/app init code does: PyATS_HANDLE myhandle = PyAutoThreadState_Init() And your C function entry points do a PyAutoThreadStateEnsure()/Release() pair. That is it! Your Python extension functions generally need take no special action, including releasing the lock, as PyAutoThreadStateEnsure() is capable of coping with the fact the lock is already held by this thread. So, to my mind, that sketches out the high-level API we are discussing. Underneath the covers, Python will need TLS to implement this. We have 2 choices for the TLS: * Implement it inside Python as part of the platform threading API. This works fine in most scenarios, but may potentially let down e.g. some Mozilla xpcom users - users where Python is ported, but this TLS API is not. Those platforms could not use this new AutoThreadState API, even though the application has a functional TLS implemention provided by xpcom. * Allow the extension author to provide "pluggable" TLS. This would expand the API like so: /* Back in the "auto thread state" header struct PyTLS_FUNCS = { /* Save a PyThreadState pointer in TLS */ int (*pfnSaveThreadState)(PyThreadState *p); /* Release the pointer for the thread (as the thread dies) */ void (*pfnReleaseThreadState)(); /* Get the saved pointer for this thread */ PyThreadState *(*pfnGetThreadState)(); } /* For the Win32 extensions, I would provide the following code in my extension */ DWORD dwTlsIndex = 0; // The TLS functions we "export" back to Python. int MyTLS_SaveThreadState(PyThreadState *ts) { // allocate space for the pointer in the platform TLS PyThreadState **p = (ThreadData **)malloc(sizeof(PyThreadState *)); if (!p) return -1; *p = ts; TlsSetValue(dwTlsIndex, p); return 0; } void PyThreadState MyTLS_DropThreadState() { PyThreadState **p = (PyThreadState**)TlsGetValue(dwTlsIndex); if (!p) return; TlsSetValue(dwTlsIndex, NULL); free(p); } PyThreadState *MyTLS_FetchThreadState() { return (PyThreadState *)TlsGetValue(dwTlsIndex); } // A structure of function pointers defined by Python. Py_AutoThreadStateFunctions myfuncs = { MyTLS_SaveThreadState, MyTLS_DropThreadState, MyTLS_FetchThreadState } /* End of Win32 code */ The XPCOM code would look almost identical, except spelt PR_GetThreadPrivate, PR_SetThreadPrivate etc. I assume pthreads can also fit into this scheme.
But yeah, as I said before, happy to YAGNI it.
Not sure what "it" is supposed to be here, either.
I'm happy to YAGNI the pluggable TLS idea. I see that the number of users who would actually benefit is tiny. Keeping the TLS api completely inside Python is fine with me. Mark.