[Python-Dev] threading (GilState) question

[Michael Hudson]

I recently redid how the readline module handled threads around
callbacks into Python (the previous code was insane).

This resulted in the following bug report:

    http://www.python.org/sf/1176893

Which is correctly assigned to me as it's clearly a result of my
recent checkin.  However, I think my code is correct and the fault
lies elsewhere.

Basically, if you call PyGilState_Release before PyEval_InitThreads
you crash, because PyEval_ReleaseThread gets called while
interpreter_lock is NULL.  This is very simple to make go away -- the
problem is that there are several ways!

Point the first is that I really think this is a bug in the GilState
APIs: the readline API isn't inherently multi-threaded and so it would
be insane to call PyEval_InitThreads() in initreadline, yet it has to
cope with being called in a multithreaded situation.  If you can't use
the GilState APIs in this situation, what are they for?

Option 1) Call PyEval_ThreadsInitialized() in PyGilState_Release().
Non-invasive, but bleh.

Option 2) Call PyEval_SaveThread() instead of
PyEval_ReleaseThread()[1] in PyGilState_Release().  This is my
favourite option (PyGilState_Ensure() calls PyEval_RestoreThread which
is PyEval_SaveThread()s "mate") and I guess you can distill this long
mail into the question "why doesn't PyGilState_Release do this
already?"

Option 3) Make PyEval_ReleaseThread() not crash when interpreter_lock
== NULL.  Easy, but it's actually documented that you can't do this.

Opinions?  Am I placing too much trust into PyGilState_Release()s
existing choice of function?

Cheers,
mwh

[1] The issue of having almost-but-not-quite identical variations of
    API functions -- here

         PyEval_AcquireThread/PyEval_ReleaseThread
     vs. PyEval_RestoreThread/PyEval_SaveThread

    -- is something I can rant about at length, if anyone is
    interested :)

--
  I located the link but haven't bothered to re-read the article,
  preferring to post nonsense to usenet before checking my facts.
                                      -- Ben Wolfson, comp.lang.python